Proceedings 17th International Conference on Data Engineering (2001)
Apr. 2, 2001 to Apr. 6, 2001
George Kollios , Boston University
Dimitrios Gunopulos , UC Riverside
Nick Koudas , AT&T Laboratories
Stefan Berchtold , stb gmbh
Abstract: We investigate the use of biased sampling according to the density of the dataset, to speed up the operation of general data mining tasks, such as clustering and outlier detection in large multidimensional datasets. In density-biased sampling, the probability that a given point will be included in the sample depends on the local density of the dataset. We propose a general technique for density-biased sampling that can factor in user requirements to sample for properties of interest, and can be tuned for specific data mining tasks. This allows great flexibility, and improved accuracy of the results over simple random sampling. We describe our approach in detail, we analytically evaluate it, and show how it can be optimized for approximate clustering and outlier detection. Finally we present a thorough experimental evaluation of the proposed method, applying density-biased sampling on real and synthetic data sets, and employing clustering and outlier detection algorithms, thus highlighting the utility of our approach.
G. Kollios, D. Gunopulos, N. Koudas and S. Berchtold, "An Efficient Approximation Scheme for Data Mining Tasks," Proceedings 17th International Conference on Data Engineering(ICDE), Heidelberg, Germany, 2001, pp. 0453.