This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Semi-Supervised Maximum Margin Clustering with Pairwise Constraints
May 2012 (vol. 24 no. 5)
pp. 926-939
ASCII Text
x 
 
Yiu-Ming Cheung, Hong Zeng, "Semi-Supervised Maximum Margin Clustering with Pairwise Constraints," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 5, pp. 926-939, May, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2011.68,
author = { Yiu-Ming Cheung and Hong Zeng},
title = {Semi-Supervised Maximum Margin Clustering with Pairwise Constraints},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {24},
number = {5},
issn = {1041-4347},
year = {2012},
pages = {926-939},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2011.68},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - Semi-Supervised Maximum Margin Clustering with Pairwise Constraints
IS - 5
SN - 1041-4347
SP926
EP939
EPD - 926-939
A1 - Yiu-Ming Cheung,
A1 - Hong Zeng,
PY - 2012
KW - quadratic programming
KW - concave programming
KW - convex programming
KW - gradient methods
KW - learning (artificial intelligence)
KW - pattern clustering
KW - semisupervised maximum margin clustering
KW - pairwise constraint
KW - k-means clustering method
KW - spectral clustering method
KW - performance enhancement
KW - maximum margin framework
KW - supervised learning
KW - maximum margin idea
KW - loss function
KW - optimization problem
KW - nonconvex problem
KW - constrained concave-convex procedure
KW - convex quadratic program
KW - subgradient projection optimization method
KW - Clustering algorithms
KW - Robustness
KW - Estimation
KW - Labeling
KW - Partitioning algorithms
KW - Optimization methods
KW - constrained concave-convex procedure.
KW - Semi-supervised clustering
KW - pairwise constraints
KW - maximum margin clustering
VL - 24
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
Yiu-Ming Cheung, Dept. of Comput. Sci., Hong Kong Baptist Univ., Hong Kong, China
Hong Zeng, Sch. of Instrum. Sci. & Eng., Southeast Univ., Nanjing, China
The pairwise constraints specifying whether a pair of samples should be grouped together or not have been successfully incorporated into the conventional clustering methods such as k-means and spectral clustering for the performance enhancement. Nevertheless, the issue of pairwise constraints has not been well studied in the recently proposed maximum margin clustering (MMC), which extends the maximum margin framework in supervised learning for clustering and often shows a promising performance. This paper therefore proposes a pairwise constrained MMC algorithm. Based on the maximum margin idea in MMC, we propose a set of effective loss functions for discouraging the violation of given pairwise constraints. For the resulting optimization problem, we show that the original nonconvex problem in our approach can be decomposed into a sequence of convex quadratic program problems via constrained concave-convex procedure (CCCP). Subsequently, we present an efficient subgradient projection optimization method to solve each convex problem in the CCCP sequence. Experiments on a number of real-world data sets show that the proposed constrained MMC algorithm is scalable and outperforms the existing constrained MMC approach as well as the typical semi-supervised clustering counterparts.

[1] B. Kulis, S. Basu, I. Dhillon, and R. Mooney, "Semi-Supervised Graph Clustering: A Kernel Approach," Proc. Int'l Conf. Machine Learning, pp. 457-464, 2005.
[2] M. Ester, R. Ge, B.J. Gao, Z. Hu, and B. Ben-Moshe, "Joint Cluster Analysis of Attribute Data and Relationship Data: The Connected K-Center Problem," Proc. SIAM Int'l Conf. Data Mining, pp. 25-46, 2006.
[3] E.P. Xing, A.Y. Ng, M.I. Jordan, and S. Russell, "Distance Metric Learning with Application to Clustering with Side-Information," Advances in Neural Information Processing Systems, vol. 15, pp. 521-528, 2003.
[4] N. Shental, T. Hertz, D. Weinshall, and M. Pavel, "Adjustment Learning and Relevant Component Analysis," Proc. European Conf. Computer Vision, pp. 776-792, 2002.
[5] S.C.H. Hoi, W. Liu, M.R. Lyu, and W.Y. Ma, "Learning Distance Metrics with Contextual Constraints for Image Retrieval," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 2072-2078, 2006.
[6] M. Bilenko, S. Basu, and R.J. Mooney, "Integrating Constraints and Metric Learning in Semi-Supervised Clustering," Proc. Int'l Conf. Machine Learning, pp. 81-88, 2004.
[7] K. Wagstaff, C. Cardie, and S. Schroedl, "Constrained K-Means Clustering with Background Knowledge," Proc. Int'l Conf. Machine Learning, pp. 577-584, 2001.
[8] S. Basu, M. Bilenko, and R.J. Mooney, "A Probabilistic Framework for Semi-Supervised Clustering," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 59-68, 2004.
[9] N. Shental, A. Bar-Hillel, D. Weinshall, "Computing Gaussian Mixture Models with EM Using Equivalence Constraints," Advances in Neural Information Processing Systems, vol. 16, pp. 465-472, 2004.
[10] M. Law, A. Topchy, and A.K. Jain, "Model-Based Clustering with Probabilistic Constraints," Proc. SIAM Int'l Conf. Data Mining, pp. 641-645, 2005.
[11] Z. Lu and M.A. Carreira-Perpinan, "Constrained Spectral Clustering through Affinity Propagation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[12] N. Kumar and K. Kummamuru, "Semisupervised Clustering with Metric Learning Using Relative Comparisons," IEEE Trans. Knowledge and Data Eng., vol. 20, no. 4, pp. 496-503, Apr. 2008.
[13] L. Xu, J. Neufeld, B. Larson, and D. Schuurmans, "Maximum Margin Clustering," Advances in Neural Information Processing Systems, vol. 17, pp. 1537-1544, 2005.
[14] H. Valizadegan and R. Jin, "Generalized Maximum Margin Clustering and Unsupervised Kernel Learning," Advances in Neural Information Processing Systems, vol. 19, pp. 1417-1424, 2007.
[15] K. Zhang, I.W. Tsang, and J.T. Kwok, "Maximum Margin Clustering Made Practical," Proc. Int'l Conf. Machine Learning, pp. 1119-1126, 2007.
[16] B. Zhao, F. Wang, and C. Zhang, "Efficient Multiclass Maximum Margin Clustering," Proc. Int'l Conf. Machine Learning, pp. 1248-1255, 2008.
[17] Y.F. Li, I.W. Tsang, J.T. Kwok, and Z.H. Zhou, "Tighter and Convex Maximum Margin Clustering," Proc. Int'l Conf. Artificial Intelligence and Statistics, pp. 344-351, 2009.
[18] Y. Hu, J. Wang, N. Yu, and X.S. Hua, "Maximum Margin Clustering with Pairwise Constraints," Proc. IEEE Int'l Conf. Data Mining, pp. 253-262, 2008.
[19] S.C.H. Hoi, R. Jin, and M.R. Lyu, "Learning Nonparametric Kernel Matrices from Pairwise Constraints," Proc. Int'l Conf. Machine Learning, pp. 361-368, 2007.
[20] K. Crammer and Y. Singer, "On the Algorithmic Implementation of Multiclass Kernel-Based Vector Machines," J. Machine Learning Research, vol. 2, pp. 265-292, 2002.
[21] L. Xu and D. Schuurmans, "Unsupervised and Semi-Supervised Multi-Class Support Vector Machines," Proc. Nat'l Conf. Artificial Intelligence, pp. 904-910, 2005.
[22] R. Yan, J. Zhang, J. Yang, and A. Hauptmann, "A Discriminative Learning Framework with Pairwise Constraints for Video Object Classification," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 578-593, Apr. 2006.
[23] N. Nguyen and R. Caruana, "Improving Classification with Pairwise Constraints: A Margin-Based Approach," Proc. European Conf. Machine Learning and Knowledge Discovery in Databases, pp. 113-124, 2008.
[24] T. Joachims, "Training Linear SVMs in Linear Time," Proc. ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 217-226, 2006.
[25] A.L. Yuille and A. Rangarajan, "The Concave-Convex Procedure," Neural Computation, vol. 15, no. 4, pp. 915-936, 2003.
[26] A.J. Smola, S.V.N. Vishwanathan, and T. Hofmann, "Kernel Methods for Missing Variables," Proc. Int'l Workshop Artificial Intelligence and Statistics, pp. 325-332, 2005.
[27] K. Zhang, I.W. Tsang, and J.T. Kwok, "Maximum Margin Clustering Made Practical," IEEE Trans. Neural Networks, vol. 20, no. 4, pp. 583-596, Apr. 2009.
[28] R. Collobert, F. Sinz, J. Weston, and L. Bottou, "Large Scale Transductive SVMs," J. Machine Learning Research, vol. 7, pp. 1687-1712, 2006.
[29] M. Karlen, J. Weston, A. Erkan, and R. Collobert, "Large Scale Manifold Transduction," Proc. Int'l Conf. Machine Learning, pp. 448-455, 2008.
[30] A. Zien, U. Brefeld, and T. Scheffer, "Transductive Support Vector Machines for Structured Variables," Proc. Int'l Conf. Machine Learning, pp. 1183-1190, 2007.
[31] K. Weinberger, J. Blitzer, and L. Saul, "Distance Metric Learning for Large Margin Nearest Neighbor Classification," Advances in Neural Information Processing Systems, vol. 18, pp. 1473-1480, 2006.
[32] S. Shalev-Shwartz, Y. Singer, and N. Srebro, "Pegasos: Primal Wstimated Sub-Gradient Solver for SVM," Proc. Int'l Conf. Machine Learning, pp. 807-814, 2007.
[33] S. Boyd and L. Vandenberghe, Convex Optimization. Cambridge Univ. Press, 2004.
[34] D.P. Bertsekas, Convex Analysis and Optimization. Athena Scientific, 2003.
[35] W. Hersh, C. Buckley, T.J. Leone, and D. Hickam, "OHSUMED: An Interactive Retrieval Evaluation and New Large Test Collection for Research," Proc. Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 192-201, 1994.
[36] A. Strehl and J. Ghosh, "Cluster Ensembles-A Knowledge Reuse Framework for Combining Multiple Partitions," J. Machine Learning Research, vol. 3, pp. 583-617, 2003.

Index Terms:
quadratic programming,concave programming,convex programming,gradient methods,learning (artificial intelligence),pattern clustering,semisupervised maximum margin clustering,pairwise constraint,k-means clustering method,spectral clustering method,performance enhancement,maximum margin framework,supervised learning,maximum margin idea,loss function,optimization problem,nonconvex problem,constrained concave-convex procedure,convex quadratic program,subgradient projection optimization method,Clustering algorithms,Robustness,Estimation,Labeling,Partitioning algorithms,Optimization methods,constrained concave-convex procedure.,Semi-supervised clustering,pairwise constraints,maximum margin clustering
Citation:
Yiu-Ming Cheung, Hong Zeng, "Semi-Supervised Maximum Margin Clustering with Pairwise Constraints," IEEE Transactions on Knowledge and Data Engineering, vol. 24, no. 5, pp. 926-939, May 2012, doi:10.1109/TKDE.2011.68
Usage of this product signifies your acceptance of the Terms of Use.