This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Clustering with Local and Global Regularization
December 2009 (vol. 21 no. 12)
pp. 1665-1678
ASCII Text
x 
 
Fei Wang, Changshui Zhang, Tao Li, "Clustering with Local and Global Regularization," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 12, pp. 1665-1678, December, 2009.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2009.40,
author = {Fei Wang and Changshui Zhang and Tao Li},
title = {Clustering with Local and Global Regularization},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {21},
number = {12},
issn = {1041-4347},
year = {2009},
pages = {1665-1678},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2009.40},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - Clustering with Local and Global Regularization
IS - 12
SN - 1041-4347
SP1665
EP1678
EPD - 1665-1678
A1 - Fei Wang,
A1 - Changshui Zhang,
A1 - Tao Li,
PY - 2009
KW - Clustering
KW - local learning
KW - smoothness
KW - regularization.
VL - 21
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
Fei Wang, Tsinghua University, Beijing
Changshui Zhang, Tsinghua University, Beijing
Tao Li, Florida International University, Miami
Clustering is an old research topic in data mining and machine learning. Most of the traditional clustering methods can be categorized as local or global ones. In this paper, a novel clustering method that can explore both the local and global information in the data set is proposed. The method, Clustering with Local and Global Regularization (CLGR), aims to minimize a cost function that properly trades off the local and global costs. We show that such an optimization problem can be solved by the eigenvalue decomposition of a sparse symmetric matrix, which can be done efficiently using iterative methods. Finally, the experimental results on several data sets are presented to show the effectiveness of our method.

[1] A. Argyriou, M. Herbster, and M. Pontil, “Combining Graph Laplacians for Semi-Supervised Learning,” Proc. Conf. Neural Information Processing Systems, 2005.
[2] M. Belkin and P. Niyogi, “Laplacian Eigenmaps for Dimensionality Reduction and Data Representation,” Neural Computation, vol. 15, no. 6, pp. 1373-1396, 2003.
[3] M. Belkin and P. Niyogi, “Semi-Supervised Learning on Riemannian Manifolds,” Machine Learning, vol. 56, pp. 209-239, 2004.
[4] M. Belkin and P. Niyogi, “Towards a Theoretical Foundation for Laplacian-Based Manifold Methods,” Proc. 18th Conf. Learning Theory (COLT), 2005.
[5] M. Belkin, P. Niyogi, and V. Sindhwani, “Manifold Regularization: A Geometric Framework for Learning from Labeled and Unlabeled Examples,” J. Machine Learning Research, vol. 7, pp.2399-2434, 2006.
[6] L. Bottou and V. Vapnik, “Local Learning Algorithms,” Neural Computation, vol. 4, pp. 888-900, 1992.
[7] P.K. Chan, D.F. Schlag, and J.Y. Zien, “Spectral K-Way Ratio-Cut Partitioning and Clustering,” IEEE Trans. Computer-Aided Design of Integrated Circuits and Systems, vol. 13, no. 9, pp. 1088-1096, Sept. 1994.
[8] O. Chapelle, M. Chi, and A. Zien, “A Continuation Method for Semi-Supervised SVMs,” Proc. 23rd Int'l Conf. Machine Learning, pp. 185-192, 2006.
[9] J. Chen, Z. Zhao, J. Ye, and L. Huan, “Nonlinear Adaptive Distance Metric Learning for Clustering,” Proc. 13th ACM Special Interest Group Conf. Knowledge Discovery and Data Mining (SIGKDD), pp. 123-132, 2007.
[10] G. Dai and D.-Y. Yeung, “Kernel Selection for Semi-Supervised Kernel Machines,” Proc. 24th Int'l Conf. Machine Learning (ICML '07), pp. 185-192, 2007.
[11] C. Ding, X. He, H. Zha, M. Gu, and H.D. Simon, “A Min-Max Cut Algorithm for Graph Partitioning and Data Clustering,” Proc. First Int'l Conf. Data Mining (ICDM '01), pp. 107-114, 2001.
[12] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. John Wiley & Sons, 2001.
[13] A.S. Georghiades, P.N. Belhumeur, and D.J. Kriegman, “From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[14] G.H. Golub and C.F. Van Loan, Matrix Computations, third ed. The Johns Hopkins Univ. Press, 1996.
[15] J. Han and M. Kamber, Data Mining. Morgan Kaufmann, 2001.
[16] M. Hein, J.Y. Audibert, and U. von Luxburg, “From Graphs to Manifolds—Weak and Strong Pointwise Consistency of Graph Laplacians,” Proc. 18th Conf. Learning Theory (COLT '05), pp. 470-485, 2005.
[17] J. He, M. Lan, C.-L. Tan, S.-Y. Sung, and H.-B. Low, “Initialization of Cluster Refinement Algorithms: A Review and Comparative Study,” Proc. Int'l Joint Conf. Neural Networks, 2004.
[18] A. Jain and R. Dubes, Algorithms for Clustering Data. Prentice-Hall, 1988.
[19] B. Kernighan and S. Lin, “An Efficient Heuristic Procedure for Partitioning Graphs,” The Bell System Technical J., vol. 49, no. 2, pp.291-307, 1970.
[20] L. Zelnik-Manor and P. Perona, “Self-Tuning Spectral Clustering,” Proc. Conf. Neural Information Processing Systems, 2005.
[21] A. McCallum, K. Nigam, J. Rennie, and K. Seymore, “Automating the Contruction of Internet Portals with Machine Learning,” Information Retrieval J., vol. 3, pp. 127-163, 2000.
[22] S.A. Nene, S.K. Nayar, and J. Murase, “Columbia Object Image Library (COIL-20),” Technical Report CUCS-005-96, Columbia Univ., Feb. 1996.
[23] A.Y. Ng, M.I. Jordan, and Y. Weiss, “On Spectral Clustering: Analysis and an Algorithm,” Proc. Conf. Neural Information Processing Systems, 2002.
[24] S.T. Roweis and L.K. Saul, “Noninear Dimensionality Reduction by Locally Linear Embedding,” Science, vol. 290, pp. 2323-2326, 2000.
[25] B. Schölkopf and A. Smola, Learning with Kernels. The MIT Press, 2002.
[26] J. Shawe-Taylor and N. Cristianini, Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.
[27] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[28] T. Sim, S. Baker, and M. Bsat, “The CMU Pose, Illumination, and Expression Database,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[29] A. Strehl and J. Ghosh, “Cluster Ensembles—A Knowledge Reuse Framework for Combining Multiple Partitions,” J. Machine Learning Research, vol. 3, pp. 583-617, 2002.
[30] V.N. Vapnik, The Nature of Statistical Learning Theory. Springer-Verlag, 1995.
[31] F. Wang, C. Zhang, and T. Li, “Regularized Clustering for Documents,” Proc. 30th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval (SIGIR '07), 2007.
[32] F. Wang and C. Zhang, “Label Propagation through Linear Neighborhoods,” Proc. 23rd Int'l Conf. Machine Learning, 2006.
[33] Y. Weiss, “Segmentation Using Eigenvectors: A Unifying View,” Proc. IEEE Int'l Conf. Computer Vision, pp. 975-982, 1999.
[34] M. Wu and B. Schölkopf, “A Local Learning Approach for Clustering,” Proc. Conf. Neural Information Processing Systems, 2006.
[35] M. Wu and B. Schölkopf, “Transductive Classification via Local Learning Regularization,” Proc. 11th Int'l Conf. Artificial Intelligence and Statistics (AISTATS '07), pp. 628-635, 2007.
[36] S.X. Yu and J. Shi, “Multiclass Spectral Clustering,” Proc. Int'l Conf. Computer Vision, 2003.
[37] L. Zelnik-Manor and P. Perona, “Self-Tuning Spectral Clustering,” Proc. Conf. Neural Information Processing Systems, 2005.
[38] H. Zha, X. He, C. Ding, M. Gu, and H. Simon, “Spectral Relaxation for K-Means Clustering,” Proc. Conf. Neural Information Processing Systems, 2001.
[39] T. Zhang and J.F. Oles, “Text Categorization Based on Regularized Linear Classification Methods,” Information Retrieval, vol. 4, pp. 5-31, 2001.
[40] D. Zhou and B. Schölkopf, “Learning from Labeled and Unlabeled Data Using Random Walks,” Proc. 26th DAGM Symp. Pattern Recognition, pp. 237-244, 2004.
[41] D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Schölkopf, “Learning with Local and Global Consistency,” Proc. Conf. Neural Information Processing Systems, 2004.
[42] X. Zhu, Z. Ghahramani, and J. Lafferty, “Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions,” Proc. 20th Int'l Conf. Machine Learning, 2003.
[43] X. Zhu, J. Lafferty, and Z. Ghahramani, “Semi-Supervised Learning: From Gaussian Fields to Gaussian Process,” Computer Science Technical Report CMU-CS-03-175, Carnegie Mellon Univ., 2003.
[44] X. Zhu and A. Goldberg, “Kernel Regression with Order Preferences,” Proc. 22nd AAAI Conf. Artificial Intelligence (AAAI), 2007.

Index Terms:
Clustering, local learning, smoothness, regularization.
Citation:
Fei Wang, Changshui Zhang, Tao Li, "Clustering with Local and Global Regularization," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 12, pp. 1665-1678, Dec. 2009, doi:10.1109/TKDE.2009.40
Usage of this product signifies your acceptance of the Terms of Use.