CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2013 vol.35 Issue No.09 - Sept.

Subscribe

Issue No.09 - Sept. (2013 vol.35)

pp: 2223-2237

Chang-Dong Wang , Sch. of Inf. Sci. & Technol., Sun Yat-sen Univ., Guangzhou, China

Jian-Huang Lai , Sch. of Inf. Sci. & Technol., Sun Yat-sen Univ., Guangzhou, China

Ching Y. Suen , Centre for Pattern Recognition & Machine Intell. (CENPARMI), Concordia Univ., Montreal, QC, Canada

Jun-Yong Zhu , Sch. of Math. & Comput. Sci., Sun Yat-sen Univ., Guangzhou, China

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2013.28

ABSTRACT

The affinity propagation (AP) clustering algorithm has received much attention in the past few years. AP is appealing because it is efficient, insensitive to initialization, and it produces clusters at a lower error rate than other exemplar-based methods. However, its single-exemplar model becomes inadequate when applied to model multisubclasses in some situations such as scene analysis and character recognition. To remedy this deficiency, we have extended the single-exemplar model to a multi-exemplar one to create a new multi-exemplar affinity propagation (MEAP) algorithm. This new model automatically determines the number of exemplars in each cluster associated with a super exemplar to approximate the subclasses in the category. Solving the model is NP--hard and we tackle it with the max-sum belief propagation to produce neighborhood maximum clusters, with no need to specify beforehand the number of clusters, multi-exemplars, and superexemplars. Also, utilizing the sparsity in the data, we are able to reduce substantially the computational time and storage. Experimental studies have shown MEAP's significant improvements over other algorithms on unsupervised image categorization and the clustering of handwritten digits.

INDEX TERMS

Clustering algorithms, Belief propagation, Couplings, Computational modeling, Kernel, Clustering methods, Educational institutions,max-product belief propagation, Clustering, multi-exemplar, affinity propagation, factor graph

CITATION

Chang-Dong Wang, Jian-Huang Lai, Ching Y. Suen, Jun-Yong Zhu, "Multi-Exemplar Affinity Propagation",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.35, no. 9, pp. 2223-2237, Sept. 2013, doi:10.1109/TPAMI.2013.28REFERENCES

- [1] B.J. Frey and D. Dueck, "Clustering by Passing Messages between Data Points,"
Science, vol. 315, pp. 972-976, http://www.psi. toronto.eduindex.php?q=affinity%20propagation , 2007.- [2] F.R. Kschischang, B.J. Frey, and H.-A. Loeliger, "Factor Graphs and the Sum-Product Algorithm,"
IEEE Trans. Information Theory, vol. 47, no. 2, pp. 498-519, Feb. 2001.- [3] Y. Weiss and W.T. Freeman, "On the Optimality of Solutions of the Max-Product Belief-Propagation Algorithm in Arbitrary Graphs,"
IEEE Trans. Information Theory, vol. 47, no. 2, pp. 736-744, Feb. 2001.- [4] J. MacQueen, "Some Methods for Classification and Analysis of Multivariate Observations,"
Proc. 15th Berkeley Symp. Math. Statistics and Probability, vol. 1, pp. 281-297, 1967.- [5] A.K. Jain, "Data Clustering: 50 Years Beyond K-Means,"
Pattern Recognition Letters, vol. 31, pp. 651-666, 2010.- [6] D. Dueck, B.J. Frey, N. Jojic, V. Jojic, G. Giaever, A. Emili, G. Musso, and R. Hegele, "Constructing Treatment Portfolios Using Affinity Propagation,"
Proc. 12th Ann. Int'l Conf. Research in Computational Molecular Biology, pp. 360-371, 2008.- [7] T.-H. Huang, K.-Y. Cheng, and Y.-Y. Chuang, "A Collaborative Benchmark for Region of Interest Detection Algorithms,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 296-303, 2009.- [8] R. Verma and P. Wang, "On Detecting Subtle Pathology via Tissue Clustering of Multi-Parametric Data Using Affinity Propagation,"
Proc. 11th IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.- [9] D. Dueck and B.J. Frey, "Non-Metric Affinity Propagation for Unsupervised Image Categorization,"
Proc. 11th IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.- [10] Z.-Q. Zhao, J. Gao, H. Glotin, and X. Wu, "A Matrix Modular Neural Network Based on Task Decomposition with Subspace Division by Adaptive Affinity Propagation Clustering,"
Applied Math. Modelling, vol. 34, pp. 3884-3895, 2010.- [11] M.L. Sumedha and M. Weigt, "Unsupervised and Semi-Supervised Clustering by Message Passing: Soft-Constraint Affinity Propagation,"
European Physical J. B, vol. 66, pp. 125-135, 2008.- [12] D. Tarlow, R.S. Zemel, and B.J. Frey, "Flexible Priors for Exemplar-Based Clustering,"
Proc. 24th Conf. Uncertainty in Artificial Intelligence, pp. 537-545, 2008.- [13] X. Zhang, C. Furtlehner, and M. Sebag, "Data Streaming with Affinity Propagation,"
Proc. European Conf. Machine Learning and Knowledge Discovery in Databases, pp. 628-643, 2008.- [14] I.E. Givoni and B.J. Frey, "Semi-Supervised Affinity Propagation with Instance-Level Constraints,"
Proc. Conf. Artificial Intelligence and Statistics, pp. 161-168, 2009.- [15] J. Xiao, J. Wang, P. Tan, and L. Quan, "Joint Affinity Propagation for Multiple View Segmentation,"
Proc. IEEE Int'l Conf. Computer Vision, pp. 1-7, 2007.- [16] I.E. Givoni, C. Chung, and B.J. Frey, "Hierarchical Affinity Propagation,"
Proc. 24th Conf. Uncertainty in Artificial Intelligence, pp. 238-246, 2011.- [17] L. Fei-Fei and P. Perona, "A Bayesian Hierarchical Model for Learning Natural Scene Categories,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 524-531, 2005.- [18] M. Zhu and A.M. Martinez, "Subclass Discriminant Analysis,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1274-1286, Aug. 2006.- [19] H.I. Avi-Itzhak, J.A.V. Mieghem, and L. Rub, "Multiple Subclass Pattern Recognition: A Maximin Correlation Approach,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 4, pp. 418-431, Apr. 1995.- [20] A.F.R. Rahman and M. Fairhurst, "Multi-Prototype Classification: Improved Modelling of the Variability of Handwritten Data Using Statistical Clustering Algorithms,"
Electronics Letters, vol. 33, pp. 1208-1210, 1997.- [21] B. Schölkopf, A. Smola, and K.-R. Müller, "Nonlinear Component Analysis as a Kernel Eigenvalue Problem,"
Neural Computation, vol. 10, pp. 1299-1319, 1998.- [22] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.- [23] C.-D. Wang, J.-H. Lai, and J.-Y. Zhu, "A Conscience On-Line Learning Approach for Kernel-Based Clustering,"
Proc. 10th Int'l Conf. Data Mining, pp. 531-540, 2010.- [24] I.S. Dhillon, Y. Guan, and B. Kulis, "Weighted Graph Cuts Without Eigenvectors: A Multilevel Approach,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1944-1957, Nov. 2007.- [25] S.K. Zhou and R. Chellappa, "Multiple-Exemplar Discriminant Analysis for Face Recognition,"
Proc. 17th Int'l Conf. Pattern Recognition, pp. 191-194, 2004.- [26] J. Reisinger and R.J. Mooney, "Multi-Prototype Vector-Space Models of Word Meaning,"
Proc. Ann. Conf. North Am. Chapter of the Assoc. for Computational Linguistics, pp. 109-117, 2010.- [27] Q. Zhu, Y. Cai, and L. Liu, "A Multiple Hyper-Ellipsoidal Subclass Model for an Evolutionary Classifier,"
Pattern Recognition, vol. 34, pp. 547-560, 2001.- [28] F. Aiolli and A. Sperduti, "Multiclass Classification with Multi-Prototype Support Vector Machines,"
J. Machine Learning Research, vol. 6, pp. 817-850, 2005.- [29] S. Guha, R. Rastogi, and K. Shim, "CURE: An Efficient Clustering Algorithm for Large Databases,"
Information Systems, vol. 26, no. 1, pp. 35-58, 2001.- [30] M. Liu, X. Jiang, and A.C. Kot, "A Multi-Prototype Clustering Algorithm,"
Pattern Recognition, vol. 42, pp. 689-698, 2009.- [31] T. Luo, C. Zhong, H. Li, and X. Sun, "A Multi-Prototype Clustering Algorithm Based on Minimum Spanning Tree,"
Proc. Seventh Int'l Conf. Fuzzy Systems and Knowledge Discovery, pp. 1602-1607, 2010.- [32] D. Dueck, "Affinity Propagation: Clustering Data by Passing Messages," PhD dissertation, Univ. of Toronto, 2009.
- [33] I.E. Givoni and B.J. Frey, "A Binary Variable Model for Affinity Propagation,"
Neural Computation, vol. 21, no. 6, pp. 1589-1600, June 2009.- [34] J. Shawe-Taylor and N. Cristianini,
Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.- [35] L. Xu, J. Neufeld, B. Larson, and D. Schuurmans, "Maximum Margin Clustering,"
Proc. Conf. Neural Information Processing Systems, 2004.- [36] M.J. Lyons, S. Akamatsu, M. Kamachi, and J. Gyoba, "Coding Facial Expressions with Gabor Wavelets,"
Proc. Third IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 200-205, 1998.- [37] L. Fei-Fei, R. Fergus, and P. Perona, "Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, pp. 178-188, 2004.- [38] D. You, O.C. Hamsici, and A.M. Martinez, "Kernel Optimization in Discriminant Analysis,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 631-638, Mar. 2011.- [39] K. Grauman and T. Darrell, "Unsupervised Learning of Categories from Sets of Partially Matching Image Features,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, pp. 2596-2603, 2006.- [40] K. Mikolajczyk, B. Leibe, and B. Schiele, "Multiple Object Class Detection with a Generative Model,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshop, pp. 26-36, 2006.- [41] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints,"
Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.- [42] A. Strehl and J. Ghosh, "Cluster Ensembles—A Knowledge Reuse Framework for Combining Multiple Partitions,"
J. Machine Learning Research, vol. 3, pp. 583-617, 2002.- [43] A. Strehl, J. Ghosh, and R.J. Mooney, "Impact of Similarity Measures on Web-Page Clustering,"
Proc. AAAI Workshop AI for Web Search, pp. 58-64, 2000.- [44] L. Hubert and P. Arabie, "Comparing Partitions,"
J. Classification, vol. 2, pp. 193-218, 1985.- [45] M. Meil, "Comparing Clusterings—An Axiomatic View,"
Proc. 22nd Int'l Conf. Machine Learning, pp. 577-584, 2005.- [46] P. Hansen and N. Mladenović, "Variable Neighborhood Search for the p-Median,"
Location Science, vol. 5, no. 4, pp. 207-226, Dec. 1997.- [47] J.J. Hull, "A Database for Handwritten Text Recognition Research,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 5, pp. 550-554, May 1994.- [48] Y. LeCun, L. Bottou, Y. Bengio, and P. Haffner, "Gradient-Based Learning Applied to Document Recognition,"
Proc. IEEE, vol. 86, no. 11, pp. 2278-2324, Nov. 1998. |