The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2011 vol.33)
pp: 1548-1560
Xiaofei He , Zhejiang University, Hangzhou
Jiawei Han , University of Illinois at Urbana-Champaign, Urbana
Thomas S. Huang , University of Illinois at Urbana-Champaign, Urbana
ABSTRACT
Matrix factorization techniques have been frequently applied in information retrieval, computer vision, and pattern recognition. Among them, Nonnegative Matrix Factorization (NMF) has received considerable attention due to its psychological and physiological interpretation of naturally occurring data whose representation may be parts based in the human brain. On the other hand, from the geometric perspective, the data is usually sampled from a low-dimensional manifold embedded in a high-dimensional ambient space. One then hopes to find a compact representation,which uncovers the hidden semantics and simultaneously respects the intrinsic geometric structure. In this paper, we propose a novel algorithm, called Graph Regularized Nonnegative Matrix Factorization (GNMF), for this purpose. In GNMF, an affinity graph is constructed to encode the geometrical information and we seek a matrix factorization, which respects the graph structure. Our empirical study shows encouraging results of the proposed algorithm in comparison to the state-of-the-art algorithms on real-world problems.
INDEX TERMS
Nonnegative matrix factorization, graph Laplacian, manifold regularization, clustering.
CITATION
Xiaofei He, Jiawei Han, Thomas S. Huang, "Graph Regularized Nonnegative Matrix Factorization for Data Representation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 8, pp. 1548-1560, August 2011, doi:10.1109/TPAMI.2010.231
REFERENCES
[1] M. Belkin and P. Niyogi, "Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering," Advances in Neural Information Processing Systems 14, pp. 585-591, MIT Press, 2001.
[2] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Examples," J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.
[3] J.-P. Brunet, P. Tamayo, T.R. Golub, and J.P. Mesirov, "Metagenes and Molecular Pattern Discovery Using Matrix Factorization," Nat'l Academy of Sciences, vol. 101, no. 12, pp. 4164-4169, 2004.
[4] D. Cai, X. He, and J. Han, "Document Clustering Using Locality Preserving Indexing," IEEE Trans. Knowledge and Data Eng., vol. 17, no. 12, pp. 1624-1637, Dec. 2005.
[5] D. Cai, X. He, X. Wang, H. Bao, and J. Han, "Locality Preserving Nonnegative Matrix Factorization," Proc. Int'l Joint Conf. Artificial Intelligence, 2009.
[6] D. Cai, X. He, X. Wu, and J. Han, "Non-Negative Matrix Factorization on Manifold," Proc. Int'l Conf. Data Mining, 2008.
[7] D. Cai, X. Wang, and X. He, "Probabilistic Dyadic Data Analysis with Local and Global Consistency," Proc. 26th Ann. Int'l Conf. Machine Learning, pp. 105-112, 2009.
[8] M. Catral, L. Han, M. Neumann, and R. Plemmons, "On Reduced Rank Nonnegative Matrix Factorization for Symmetric Nonnegative Matrices," Linear Algebra and Its Applications, vol. 393, pp. 107-126, 2004.
[9] F.R.K. Chung, Spectral Graph Theory. Am. Math. Soc., 1997.
[10] T.H. Cormen, C.E. Leiserson, R.L. Rivest, and C. Stein, Introduction to Algorithms, second ed. MIT Press and McGraw-Hill, 2001.
[11] S.C. Deerwester, S.T. Dumais, T.K. Landauer, G.W. Furnas, and R.A. harshman, "Indexing by Latent Semantic Analysis," J. Am. Soc. of Information Science, vol. 41, no. 6, pp. 391-407, 1990.
[12] A.P. Dempster, N.M. Laird, and D.B. Rubin, "Maximum Likelihood from Incomplete Data via the Em Algorithm," J. Royal Statistical Soc. Series B (Methodological), vol. 39, no. 1, pp. 1-38, 1977.
[13] C. Ding, T. Li, and W. Peng, "Nonnegative Matrix Factorization and Probabilistic Latent Semantic Indexing: Equivalence, Chi-Square Statistic, and a Hybrid Method," Proc. Nat'l Conf. American Association for Artificial Intelligence, 2006.
[14] D. Donoho and V. Stodden, "When Does Non-Negative Matrix Factorization Give a Correct Decomposition into Parts?" Advances in Neural Information Processing Systems 16, MIT Press, 2003.
[15] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification, second ed. Wiley-Interscience, 2000.
[16] L. Finesso and P. Spreij, "Nonnegative Matrix Factorization and I-Divergence Alternating Minimization," Linear Algebra and Its Applications, vol. 416, nos. 2/3, pp. 270-287, 2006.
[17] E. Gaussier and C. Goutte, "Relation between PLSA and NFM and Implications," Proc. 28th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 601-602, 2005.
[18] R. Hadsell, S. Chopra, and Y. LeCun, "Dimensionality Reduction by Learning an Invariant Mapping," Proc. IEEE CS. Conf. Computer Vision and Pattern Recognition, pp. 1735-1742, 2006.
[19] X. He and P. Niyogi, "Locality Preserving Projections," Advances in Neural Information Processing Systems 16, MIT Press, 2003.
[20] M.R. Hestenes and E. Stiefel, "Methods of Conjugate Gradients for Solving Linear Systems," J. Research of the Nat'l Bureau of Standards, vol. 49, no. 6, pp. 409-436, 1952.
[21] T. Hofmann, "Unsupervised Learning by Probabilistic Latent Semantic Analysis," Machine Learning, vol. 42, nos. 1/2, pp. 177-196, 2001.
[22] P.O. Hoyer, "Non-Negative Sparse Coding," Proc. IEEE Workshop Neural Networks for Signal Processing, pp. 557-565, 2002.
[23] P.O. Hoyer, "Non-Negative Matrix Factorization with Sparseness Constraints," J. Machine Learning Research, vol. 5, pp. 1457-1469, 2004.
[24] I.T. Jolliffe, Principal Component Analysis. Springer-Verlag, 1989.
[25] J. Kivinen and M.K. Warmuth, "Additive versus Exponentiated Gradient Updates for Linear Prediction," Proc. Ann. ACM Symp. Theory of Computing, pp. 209-218, 1995.
[26] D.D. Lee and H.S. Seung, "Learning the Parts of Objects by Non-Negative Matrix Factorization," Nature, vol. 401, pp. 788-791, 1999.
[27] D.D. Lee and H.S. Seung, "Algorithms for Non-Negative Matrix Factorization," Advances in Neural Information Processing Systems 13, MIT Press, 2001.
[28] J.M. Lee, Introduction to Smooth Manifolds. Springer-Verlag, 2002.
[29] S.Z. Li, X. Hou, H. Zhang, and Q. Cheng, "Learning Spatially Localized, Parts-Based Representation," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 207-212, 2001.
[30] C.-J. Lin, "On the Convergence of Multiplicative Update Algorithms for Non-Negative Matrix Factorization," IEEE Trans. Neural Networks, vol. 18, no. 6, pp. 1589-1596, Nov. 2007.
[31] N.K. Logothetis and D.L. Sheinberg, "Visual Object Recognition," Ann. Rev. of Neuroscience, vol. 19, pp. 577-621, 1996.
[32] A.Y. Ng, M. Jordan, and Y. Weiss, "On Spectral Clustering: Analysis and an Algorithm," Advances in Neural Information Processing Systems 14, pp. 849-856, MIT Press, 2001.
[33] P. Paatero and U. Tapper, "Positive Matrix Factorization: A Non-Negative Factor Model with Optimal Utilization of Error Estimates of Data Values," Environmetrics, vol. 5, no. 2, pp. 111-126, 1994.
[34] S.E. Palmer, "Hierarchical Structure in Perceptual Representation," Cognitive Psychology, vol. 9, pp. 441-474, 1977.
[35] S. Roweis and L. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, no. 5500, pp. 2323-2326, 2000.
[36] H.S. Seung and D.D. Lee, "The Manifold Ways of Perception," Science, vol. 290, no. 12, pp. 2268/2269, 2000.
[37] F. Shahnaza, M.W. Berrya, V. Paucab, and R.J. Plemmonsb, "Document Clustering Using Nonnegative Matrix Factorization," Information Processing and Management, vol. 42, no. 2, pp. 373-386, 2006.
[38] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[39] J. Tenenbaum, V. de Silva, and J. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, no. 5500, pp. 2319-2323, 2000.
[40] M. Turk and A. Pentland, "Eigenfaces for Recognition," J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[41] E. Wachsmuth, M.W. Oram, and D.I. Perrett, "Recognition of Objects and Their Component Parts: Responses of Single Units in the Temporal Cortex of the Macaque," Cerebral Cortex, vol. 4, pp. 509-522, 1994.
[42] W. Xu, X. Liu, and Y. Gong, "Document Clustering Based on Non-Negative Matrix Factorization," Proc. Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 267-273, Aug. 2003.
[43] L. Zelnik-Manor and P. Perona, "Self-Tuning Spectral Clustering," Advances in Neural Information Processing Systems 17, pp. 1601-1608, MIT Press, 2004.
[44] H. Zha, C. Ding, M. Gu, X. He, and H. Simon, "Spectral Relaxation for k-Means Clustering," Advances in Neural Information Processing Systems 14, pp. 1057-1064, MIT Press, 2001.
[45] D. Zhou, O. Bousquet, T. Lal, J. Weston, and B. Schölkopf, "Learning with Local and Global Consistency," Advances in Neural Information Processing Systems 16, MIT Press, 2003.
[46] X. Zhu and J. Lafferty, "Harmonic Mixtures: Combining Mixture Models and Graph-Based Methods for Inductive and Scalable Semi-Supervised Learning," Proc. 22nd Int'l Conf. Machine Learning, pp. 1052-1059, 2005.
27 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool