This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Agglomerative Mean-Shift Clustering
February 2012 (vol. 24 no. 2)
pp. 209-219
Xiao-Tong Yuan, Rutgers University, Newark
Bao-Gang Hu, Institute of Automation Chinese Academy of Sciences, Beijing
Ran He, Institute of Automation Chinese Academy of Sciences, Beijing
Mean-Shift (MS) is a powerful nonparametric clustering method. Although good accuracy can be achieved, its computational cost is particularly expensive even on moderate data sets. In this paper, for the purpose of algorithmic speedup, we develop an agglomerative MS clustering method along with its performance analysis. Our method, namely Agglo-MS, is built upon an iterative query set compression mechanism which is motivated by the quadratic bounding optimization nature of MS algorithm. The whole framework can be efficiently implemented in linear running time complexity. We then extend Agglo-MS into an incremental version which performs comparably to its batch counterpart. The efficiency and accuracy of Agglo-MS are demonstrated by extensive comparing experiments on synthetic and real data sets.

[1] M. Allain, J. Idier, and Y. Goussard, "On Global and Local Convergence of Half-quadratic Algorithms," IEEE Trans. Image Processing, vol. 15, no. 5, pp. 1130-1142, May 2006.
[2] D. Barash and D. Comaniciu, "A Common Framework for Nonlinear Diffusion, Adaptive Smoothing, Bilateral Filtering and Mean Shift," Image and Vision Computing, vol. 22, no. 1, pp. 73-81, Jan. 2004.
[3] D. Beeferman and A. Berger, "Agglomerative Clustering of a Search Engine Query Log," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 407-416, 2000.
[4] M. Bilenko, S. Basu, and R. Mooney, "Integrating Constraints and Metric Learning in Semi-Supervised Clustering," Proc. Int'l Conf. Machine Learning, pp. 81-88, 2004.
[5] C.M. Bishop, Pattern Recognition and Machine Learning. Springer, 2006.
[6] D. Cai, X. He, and J. Han, "Spectral Regression for Efficient Regularized Subspace Learning," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[7] M. Carreira-Perpinan, "Mode-Finding for Mixtures of Gaussian Distributions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1318-1323, Nov. 2000.
[8] M. Carreira-Perpinan, "Continuous Latent Variable Models for Dimensionality Reduction and Sequential Data Reconstruction," PhD thesis, Univ. of Sheffield, 2001.
[9] M. Carreira-Perpinan, "Acceleration Strategies for Gaussian Mean-Shift Image Segmentation," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 1160-1167, 2006.
[10] M. Carreira-Perpinan, "Fast Nonparametric Clustering with Gaussian Blurring Mean-Shift," Proc. Int'l Conf. Machine Learning, pp. 153-160, 2006.
[11] M. Carreira-Perpinan, "Gaussian Mean-Shift is an em Algorithm," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, pp. 767-776, May 2007.
[12] H.E. Cetingul and R. Vidal, "Intrinsic Mean Shift for Clustering on Stiefel and Grassmann Manifolds," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[13] C.-C. Chang and C.-J. Lin, "Ijcnn 2001 Challenge: Generalization Ability and Text Decoding," Proc. Int'l Joint Conf. Neural Networks, 2001.
[14] Y. Cheng, "Mean Shift, Mode Seeking, and Clustering," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 7, pp. 790-799, July 1995.
[15] R.T. Collins, "Mean-Shift Blob Tracking through Scale Space," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 234-240, 2003.
[16] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[17] D. Comaniciu, V. Ramesh, and P. Meer, "Real-Time Tracking of Non-Rigid Objets Using Mean Shift," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[18] I. Davidson and S.S. Ravi, "Using Instance-Level Constraints in Agglomerative Hierarchical Clustering: Theoretical and Empirical Results," Data Mining and Knowledge Discovery, vol. 18, no. 2, pp. 257-282, Apr. 2009.
[19] L. Dragomirescu and T. Postelnicu, "A Natural Agglomerative Clusteringmethod for Biology," Biometrical J., vol. 33, no. 7, pp. 841-849, Jan. 2007.
[20] M. Fashing and C. Tomasi, "Mean Shift Is a Bound Optimization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 471-474, Mar. 2005.
[21] D. Freedman and P. Kisilev, "Fast Mean Shift by Compact Density Representation," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[22] K. Fukunaga and L. Hostetler, "The Estimation of the Gradient of a Density Function, with Application in Pattern Recognition," IEEE Trans. Information Theory, vol. 21, no. 1, pp. 32-40, Jan. 1975.
[23] M.R. Garey and D.S. Johnson, Computers and Intractability, A Guide to the Theory of NP-Completeness. Freeman, 1979.
[24] B. Georgescu, I. Shimshoni, and P. Meer, "Mean Shift Based Clustering in High Dimensions: A Texture Classification Example," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 456-463, 2003.
[25] A.G. Gray and A.W. Moore, "N-Body Problems in Statistical Learning," Proc. Neural Information Processing Systems, pp. 521-527, 2001.
[26] S. Guha, R. Rastogi, and K. Shim, "Cure: An Efficient Clustering Algorithm for Large Databases," Proc. ACM SIGMOD Int'l Conf. Management of Data, pp. 73-84, 1998.
[27] X. He, S. Yan, Y. Hu, P. Niyogi, and H. Zhang, "Face Recognition Using Laplacianfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 1-13, Mar. 2005.
[28] A.K. Jain, M.N. Murty, and P. Flynn, "Data Clustering: A Review," ACM Computing Surveys, vol. 31, no. 3, pp. 264-323, 1999.
[29] K. Lang, "Newsweeder: Learning to Filter Netnews," Proc. Int'l Conf. Machine Learning, pp. 331-339, 1995.
[30] S. Paris and F. Durand, "A Topological Approach to Hierarchical Segmentation Using Mean Shift," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognitio, 2007.
[31] D. Prokhorov, "Ijcnn 2001 Neural Network Competition," Slide Presentation in ijcnn 01, Ford Research Laboratory, 2001.
[32] S. Rao, W. Liu, J. Principe, and A. de Medeiros Martins, "Information Theoretic Mean Shift Algorithm," Proc. IEEE Signal Processing Soc. Workshop Machine Learning for Signal Processing, pp. 155-160, 2006.
[33] R. Rockfellar, Convex Analysis. Princeton Press, 1970.
[34] Y. Sheikh, E. Khan, and T. Kanade, "Mode-Seeking by Medoidshifts," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[35] C. Shen, M.J. Brooks, and A.V.D. Hengel, "Fast Global Kernel Density Mode Seeking: Applications to Localization and Tracking," IEEE Trans. Image Processing, vol. 16, no. 5, pp. 1457-1469, May 2007.
[36] R. Subbarao and P. Meer, "Nonlinear Mean Shift for Clustering over Analytic Manifolds," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1168-1175, 2006.
[37] O. Tuzel, R. Subbarao, and P. Meer, "Simultaneous Multiple 3D Motion Estimation via Mode Finding on Lie Groups," Proc. IEEE Int'l Conf. Computer Vision, pp. 18-25, 2005.
[38] A. Vedaldi and S. Soatto, "Quick Shift and Kernel Methods for Mode Seeking," Proc. European Conf. Computer Vision, vol. 4, pp. 705-718, 2008.
[39] B. Walter, K. Bala, M. Kulkarni, and K. Pingali, "Fast Agglomerative Clustering for Rendering," Proc. IEEE Symp. Interactive Ray Tracing, pp. 81-86, 2008.
[40] F. Wang and T. Li, "Gene Selection via Matrix Factorization," Proc. IEEE Int'l Symp. Bioinformatics and Bioeng., pp. 1046-1050, 2007.
[41] F. Wang and C. Zhang, "Fast Multilevel Transduction on Graphs," Proc. SIAM Int'l Conf. Data Mining, 2007.
[42] P. Wang, D. Lee, A. Gray, and J. Rehg, "Fast Mean Shift with Accurate and Stable Convergence," Proc. Int'l Conf. Artificial Intelligence and Statistics, vol. 2, pp. 604-611, 2007.
[43] W. Wang and M. Carreira-Perpinan, "Manifold Blurring Mean Shift Algorithms for Manifold Denoising," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2010.
[44] C. Yang, R. Duraiswami, N.A. Gumerov, and L. Davis, "Improved Fast Gauss Transform and Efficient Kernel Density Estimation," Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 664-671, 2003.
[45] X. Yuan and S.Z. Li, "Half Quadratic Analysis for Mean Shift: With Extension to a Sequential Data Mode-Seeking Method," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[46] X.-T. Yuan and S.Z. Li, "Stochastic Gradient Kernel Density Mode-Seeking," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2009.
[47] K. Zhang, J.T. Kwok, and M. Tang, "Accelerated Convergence Using Dynamic Mean Shift," Proc. European Conf. Computer Vision, pp. 257-268, 2006.
[48] Y. Zho and G. Karypis, "Hierarchical Clustering Algorithms for Document Datasets," Data Mining and Knowledge Discovery, vol. 10, no. 2, pp. 141-168, Mar. 2005.

Index Terms:
Mean-shift, agglomerative clustering, half-quadratic optimization, incremental clustering.
Citation:
Xiao-Tong Yuan, Bao-Gang Hu, Ran He, "Agglomerative Mean-Shift Clustering," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 2, pp. 209-219, Feb. 2012, doi:10.1109/TKDE.2010.232
Usage of this product signifies your acceptance of the Terms of Use.