loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
15th International Conference on Pattern Recognition (ICPR'00) - Volume 2
Clustering Very Large Databases Using EM Mixture Models
Barcelona, Spain
September 03-September 08
ISBN: 0-7695-0750-6
P.S. Bradley, Microsoft Research
C.A. Reina, Microsoft Research
U.M. Fayyad, digiMine.com
Clustering very large databases is a challenge for traditional pattern recognition algorithms, e.g. the Expectation-Maximization (EM) algorithm for fitting mixture models, because of high memory and iteration requirements. Over large databases, the cost of the numerous scans required converging and large memory requirement of the algorithm becomes prohibitive. We present a decomposition of the EM algorithm requiring a small amount of memory by limiting iterations to small data subsets. The scalable EM approach requires at most one database scan and is based on identifying regions of the data that are discardable, regions that are compressible, and regions that must be maintained in memory. Data resolution is preserved to the extent possible based upon the size of the memory buffer and fit of the current model to the data. Computational tests demonstrate that the scalable scheme outperforms similarly constrained EM approaches.
Citation:
P.S. Bradley, C.A. Reina, U.M. Fayyad, "Clustering Very Large Databases Using EM Mixture Models," icpr, vol. 2, pp.2076, 15th International Conference on Pattern Recognition (ICPR'00) - Volume 2, 2000
Usage of this product signifies your acceptance of the Terms of Use.