This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Locally Consistent Concept Factorization for Document Clustering
June 2011 (vol. 23 no. 6)
pp. 902-913
ASCII Text
x 
 
Deng Cai, Xiaofei He, Jiawei Han, "Locally Consistent Concept Factorization for Document Clustering," IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 6, pp. 902-913, June, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TKDE.2010.165,
author = {Deng Cai and Xiaofei He and Jiawei Han},
title = {Locally Consistent Concept Factorization for Document Clustering},
journal ={IEEE Transactions on Knowledge and Data Engineering},
volume = {23},
number = {6},
issn = {1041-4347},
year = {2011},
pages = {902-913},
doi = {http://doi.ieeecomputersociety.org/10.1109/TKDE.2010.165},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Knowledge and Data Engineering
TI - Locally Consistent Concept Factorization for Document Clustering
IS - 6
SN - 1041-4347
SP902
EP913
EPD - 902-913
A1 - Deng Cai,
A1 - Xiaofei He,
A1 - Jiawei Han,
PY - 2011
KW - Nonnegative matrix factorization
KW - concept factorization
KW - graph Laplacian
KW - manifold regularization
KW - clustering.
VL - 23
JA - IEEE Transactions on Knowledge and Data Engineering
ER -
 
Deng Cai, Zhejiang University, Hangzhou
Xiaofei He, Zhejiang University, Hangzhou
Jiawei Han, University of Illinois at Urbana Champaign, Urbana
Previous studies have demonstrated that document clustering performance can be improved significantly in lower dimensional linear subspaces. Recently, matrix factorization-based techniques, such as Nonnegative Matrix Factorization (NMF) and Concept Factorization (CF), have yielded impressive results. However, both of them effectively see only the global euclidean geometry, whereas the local manifold geometry is not fully considered. In this paper, we propose a new approach to extract the document concepts which are consistent with the manifold geometry such that each concept corresponds to a connected component. Central to our approach is a graph model which captures the local geometry of the document submanifold. Thus, we call it Locally Consistent Concept Factorization (LCCF). By using the graph Laplacian to smooth the document-to-concept mapping, LCCF can extract concepts with respect to the intrinsic manifold structure and thus documents associated with the same concept can be well clustered. The experimental results on TDT2 and Reuters-21578 have shown that the proposed approach provides a better representation and achieves better clustering results in terms of accuracy and mutual information.

[1] M. Belkin and P. Niyogi, "Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering," Advances in Neural Information Processing Systems 14, pp. 585-591, MIT Press, 2001.
[2] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Examples," J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.
[3] Y. Bengio, J.-F. Paiement, P. Vincent, O. Delalleau, N.L. Roux, and M. Ouimet, "Out-of-Sample Extensions for LLE, Isomap, MDS, Eigenmaps, and Spectral Clustering," Advances in Neural Information Processing Systems 16, MIT Press, 2003.
[4] J.-P. Brunet, P. Tamayo, T.R. Golub, and J.P. Mesirov, "Metagenes and Molecular Pattern Discovery Using Matrix Factorization," Proc. Nat'l Academy of Sciences USA vol. 101, no. 12, pp. 4164-4169, 2004.
[5] D. Cai, X. He, and J. Han, "Document Clustering Using Locality Preserving Indexing," IEEE Trans. Knowledge and Data Eng., vol. 17, no. 12, pp. 1624-1637, Dec. 2005.
[6] D. Cai, X. He, X. Wu, and J. Han, "Non-Negative Matrix Factorization on Manifold," Proc. Int'l Conf. Data Mining (ICDM '08), 2008.
[7] F.R.K. Chung, Spectral Graph Theory, vol. 92, Am. Math. Soc., 1997.
[8] T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein, Introduction to Algorithms, second ed. MIT Press, 2001.
[9] S.C. Deerwester, S.T. Dumais, T.K. Landauer, G.W. Furnas, and R.A. harshman, "Indexing by Latent Semantic Analysis," J. Am. Soc. of Information Science, vol. 416, pp. 391-407, 1990.
[10] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the EM Algorithm," J. Royal Statistical Soc., Series B (Methodological), vol. 391, pp. 1-38, 1977.
[11] I.S. Dhillon, Y. Guan, and B. Kulis, "Kernel K-Means: Spectral Clustering and Normalized Cuts," Proc. 10th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining (KDD '04), pp. 551-556, 2004.
[12] J. Kivinen and M.K. Warmuth, "Additive versus Exponentiated Gradient Updates for Linear Prediction," Proc. 27th Ann. ACM Symp. Theory of Computing (STOC '95), pp. 209-218, 1995.
[13] D.D. Lee and H.S. Seung, "Learning the Parts of Objects by Non-Negative Matrix Factorization," Nature, vol. 401, pp. 788-791, 1999.
[14] D.D. Lee and H.S. Seung, "Algorithms for Non-Negative Matrix Factorization," Advances in Neural Information Processing Systems 13, MIT Press, 2001.
[15] X. Li and Y. Pang, "Deterministic Column-Based Matrix Decomposition," IEEE Trans. Knowledge and Data Eng., vol. 22, no. 1, pp. 145-149, Jan. 2010.
[16] N.K. Logothetis and D.L. Sheinberg, "Visual Object Recognition," Ann. Rev. of Neuroscience, vol. 19, pp. 577-621, 1996.
[17] L. Lovasz and M. Plummer, Matching Theory. Akadémiai Kiadó, 1986.
[18] A.Y. Ng, M. Jordan, and Y. Weiss, "On Spectral Clustering: Analysis and an Algorithm," Advances in Neural Information Processing Systems 14, pp. 849-856, MIT Press, 2001.
[19] S. Roweis and L. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
[20] F. Sha, Y. Lin, L.K. Saul, and D.D. Lee, "Multiplicative Updates for Nonnegative Quadratic Programming," Neural Computation, vol. 198, pp. 2004-2031, 2007.
[21] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[22] G.W. Stewart, Matrix Algorithms, Vol. 1: Basic Decompositions. SIAM, 1998.
[23] J. Tenenbaum, V. de Silva, and J. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
[24] W. Xu and Y. Gong, "Document Clustering by Concept Factorization," Proc. ACM SIGIR '04, pp. 202-209, July 2004.
[25] W. Xu, X. Liu, and Y. Gong, "Document Clustering Based on Non-Negative Matrix Factorization," Proc. ACM SIGIR '03, pp. 267-273, Aug. 2003.
[26] Y. Yuan, X. Li, Y. Pang, X. Lu, and D. Tao, "Binary Sparse Nonnegative Matrix Factorization," IEEE Trans. Circuits and Systems for Video Technology, vol. 19, no. 5, pp. 772-777, May 2009.
[27] H. Zha, C. Ding, M. Gu, X. He, and H. Simon, "Spectral Relaxation for K-Means Clustering," Advances in Neural Information Processing Systems 14, pp. 1057-1064, MIT Press, 2001.
[28] D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Schölkopf, "Learning with Local and Global Consistency," Advances in Neural Information Processing Systems 16, MIT Press, 2003.

Index Terms:
Nonnegative matrix factorization, concept factorization, graph Laplacian, manifold regularization, clustering.
Citation:
Deng Cai, Xiaofei He, Jiawei Han, "Locally Consistent Concept Factorization for Document Clustering," IEEE Transactions on Knowledge and Data Engineering, vol. 23, no. 6, pp. 902-913, June 2011, doi:10.1109/TKDE.2010.165
Usage of this product signifies your acceptance of the Terms of Use.