Subscribe
Issue No.10 - Oct. (2013 vol.25)
pp: 2206-2216
Yun Zheng , Sch. of Inf. Sci. & Technol., Sun Yat-Sen Univ., Guangzhou, China
Pei Chen , Sch. of Inf. Sci. & Technol., Sun Yat-Sen Univ., Guangzhou, China
ABSTRACT
The exemplar-based data clustering problem can be formulated as minimizing an energy function defined on a Markov random field (MRF). However, most algorithms for optimizing the MRF energy function cannot be directly applied to the task of clustering, as the problem has a high-order energy function. In this paper, we first show that the high-order energy function for the clustering problem can be simplified as a pairwise energy function with the metric property, and consequently it can be optimized by the α-expansion move algorithm based on graph cut. Then, the original expansion move algorithm is improved in the following two aspects: 1) Instead of solving a minimal s-t graph cut problem, we show that there is an explicit and interpretable solution for minimizing the energy function in the clustering problem. Based on this interpretation, a fast α-expansion move algorithm is proposed, which is much more efficient than the graph-cut-based algorithm. 2) The fast α-expansion move algorithm is further improved by extending its move space so that a larger energy value reduction can be achieved in each iteration. Experiments on benchmark data sets show that the enhanced expansion move algorithm has a better performance, compared to other state-of-the-art exemplar-based clustering algorithms.
INDEX TERMS
Clustering algorithms, Approximation algorithms, Belief propagation, Minimization, Labeling, Random variables, Measurement,$(\alpha)$-expansion, Clustering algorithms, Approximation algorithms, Belief propagation, Minimization, Labeling, Random variables, Measurement, graph cut, Exemplar-based clustering, MRF
CITATION
Yun Zheng, Pei Chen, "Clustering based on enhanced α-expansion move", IEEE Transactions on Knowledge & Data Engineering, vol.25, no. 10, pp. 2206-2216, Oct. 2013, doi:10.1109/TKDE.2012.202
REFERENCES
 [1] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001. [2] M. Brusco and H. Köhn, "Comment on 'Clustering by Passing Messages between Data Points'," Science, vol. 319, no. 5864, p. 726, 2008. [3] W. Feng, J. Jia, and Z.-Q. Liu, "Self-Validated Labeling of Markov Random Fields for Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1871-1887, Oct. 2010. [4] B. Frey and D. Dueck, "Clustering by Passing Messages between Data Points," Science, vol. 315, no. 5814, pp. 972-976, 2007. [5] B. Frey and D. Dueck, "Mixture Modeling by Affinity Propagation," Advances in Neural Information Processing Systems, vol. 18, pp. 379-386, 2006. [6] S. Guha, A. Meyerson, N. Mishra, R. Motwani, and L. O'Callaghan, "Clustering Data Streams: Theory and Practice," IEEE Trans. Knowledge and Data Eng., vol. 153, pp. 515-528, May/June 2003. [7] D.S. Hochbaum and D.B. Shmoys, "A Best Possible Heuristic for the K-Center Problem," Math. of Operations Research, vol. 10, no. 2, pp. 180-184, 1985. [8] A.K. Jain, "Data Clustering: 50 Years Beyond K-means," Pattern Recognition Letters, vol. 31, no. 8, pp. 651-666, 2010. [9] D. Jiang, C. Tang, and A. Zhang, "Cluster Analysis for Gene Expression Data: A Survey," IEEE Trans. Knowledge and Data Eng., vol. 16, no. 11, pp. 1370-1386, Nov. 2004. [10] V. Kolmogorov and C. Rother, "Comparison of Energy Minimization Algorithms for Highly Connected Graphs," Proc. European Conf. Computer Vision (ECCV), vol. 3952, pp. 1-15, 2006. [11] V. Kolmogorov and R. Zabih, "What Energy Functions Can be Minimized via Graph Cuts?," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004. [12] N. Komodakis, N. Paragios, and G. Tziritas, "Clustering via LP-Based Stabilities," Advances in Neural Information Processing Systems, vol. 21, pp. 865-872, 2008. [13] D. Lashkari and P. Golland, "Convex Clustering with Exemplar-Based Models," Advances in Neural Information Processing Systems, vol. 20, pp. 825-832, 2007. [14] N. Lazic, "Message Passing Algorithms for Facility Location Problems," PhD dissertation, Univ. of Toronto, 2011. [15] C. Leiserson, R. Rivest, and C. Stein, Introduction to Algorithms. MIT Press, 2001. [16] M. Mahdian, "Facility Location and the Analysis of Algorithms Through Factor-Revealing Programs," PhD dissertation, MIT, 2004. [17] M. Mahdian, E. Markakis, A. Saberi, and V. Vazirani, "Greedy Facility Location Algorithms Analyzed Using Dual Fitting with Factor-Revealing LP," J. ACM, vol. 50, pp. 127-137, 2001. [18] M. Mahdian, Y. Ye, and J. Zhang, "Approximation Algorithms for Metric Facility Location Problems," SIAM J. Computing, vol. 36, no. 2, pp. 411-432, 2006. [19] K. Murphy, Y. Weiss, and M. Jordan, "Loopy Belief Propagation for Approximate Inference: An Empirical Study," Proc. 15th Conf. Uncertainty in Artificial Intelligence (AI), 1999. [20] A. Ng, M. Jordan, and Y. Weiss, "On Spectral Clustering: Analysis and an Algorithm," Advances in Neural Information Processing Systems, vol. 2, pp. 849-856, 2002. [21] F. Nie, C. Ding, D. Luo, and H. Huang, "Improved Minmax Cut Graph Clustering with Nonnegative Relaxation," Proc. European Conf. Machine Learning and Knowledge Discovery in Databases, pp. 451-466, 2010. [22] F. Nie, Z. Zeng, I.W. Tsang, D. Xu, and C. Zhang, "Spectral Embedded Clustering: A Framework for In-Sample and Out-Of-Sample Spectral Clustering," IEEE Trans. Neural Networks, no. 22, pp. 1796-1808, 2011. [23] A.M. Rush, M. Collins, T. Jaakkola, D. Sontag, and M.I.T. Csail, "Dual Decomposition for Parsing with Non-Projective Head Automata," Proc. Conf. Empirical Methods in Natural Language Processing, pp. 1288-1298, 2010. [24] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June 2008. [25] D. Tarlow, R. Zemel, and B. Frey, "Flexible Priors for Exemplar-Based Clustering," Proc. Uncertainty in Artificial Intelligence (AI), 2008. [26] S. Theodoridis and K. Koutroumbas, Pattern Recognition, third ed. Academic Press, Inc., 2006. [27] N. Tishby, F. Pereira, and W. Bialek, "The Information Bottleneck Method," Proc. 37th Ann. Allerton Conf. Comm., Control, and Computing, 1999.