This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Constrained Nonnegative Matrix Factorization for Image Representation
July 2012 (vol. 34 no. 7)
pp. 1299-1311
Haifeng Liu, Zhejiang University, Hangzhou
Zhaohui Wu, Zhejiang University, Hangzhou
Deng Cai, Zhejiang University, Hangzhou
Thomas S. Huang, University of Illinois at Urbana-Champaign, Urbana
Nonnegative matrix factorization (NMF) is a popular technique for finding parts-based, linear representations of nonnegative data. It has been successfully applied in a wide range of applications such as pattern recognition, information retrieval, and computer vision. However, NMF is essentially an unsupervised method and cannot make use of label information. In this paper, we propose a novel semi-supervised matrix decomposition method, called Constrained Nonnegative Matrix Factorization (CNMF), which incorporates the label information as additional constraints. Specifically, we show how explicitly combining label information improves the discriminating power of the resulting matrix decomposition. We explore the proposed CNMF method with two cost function formulations and provide the corresponding update solutions for the optimization problems. Empirical experiments demonstrate the effectiveness of our novel algorithm in comparison to the state-of-the-art approaches through a set of evaluations based on real-world applications.

[1] D. Cai, X. He, X. Wu, and J. Han, "Non-Negative Matrix Factorization on Manifold," Proc. Eighth IEEE Int'l Conf. Data Mining, pp. 63-72, 2008.
[2] D.D. Lee and H.S. Seung, "Learning the Parts of Objects by Non-Negative Matrix Factorization," Nature, vol. 401, pp. 788-791, 1999.
[3] T. Li and C. Ding, "The Relationships among Various Nonnegative Matrix Factorization Methods for Clustering," Proc. Sixth IEEE Int'l Conf. Data Mining, pp. 362-371, 2006.
[4] X. He and P. Niyogi, "Locality Preserving Projections," Advances in Neural Information Processing Systems, vol. 16, MIT Press, 2003.
[5] X. He, S. Yan, Y. Hu, P. Niyogi, and H.-J. Zhang, "Face Recognition Using Laplacianfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 328-340, Mar. 2005.
[6] W. Liu, D. Tao, and J. Liu, "Transductive Component Analysis," Proc. IEEE Int'l Conf. Data Mining, pp. 433-442, 2008.
[7] D. Tao, X. Li, X. Wu, and S.J. Maybank, "Geometric Mean for Subspace Selection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 260-274, Feb. 2009.
[8] D. Tao, X. Li, X. Wu, and S.J. Maybank, "General Averaged Divergence Analysis," Proc. IEEE Int'l Conf. Data Mining, 2007.
[9] I. Jolliffe, Principle Component Analysis. Springer-Verlag, 1986.
[10] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. Wiley-Interscience, 2000.
[11] A. Gersho and R.M. Gray, Vector Quantization and Signal Compression. Kluwer Academic Press, 1992.
[12] S. Li, X. Hou, H. Zhang, and Q. Cheng, "Learning Spatially Localized, Parts-Based Representation," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 207-212, 2001.
[13] W. Xu, X. Liu, and Y. Gong, "Document Clustering Based on Non-Negative Matrix Factorization," Proc. Ann. ACM SIGIR Conf. Research and Development in Information Retrieval, 2003.
[14] F. Shahnaz, M.W. Berry, V. Pauca, and R.J. Plemmons, "Document Clustering Using Nonnegative Matrix Factorization," Information Processing and Management, vol. 42, pp. 373-386, 2006.
[15] M. Belkin, V. Sindhwani, and P. Niyogi, "Manifold Regularization: A Geometric Framework for Learning from Examples," J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.
[16] D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Schölkopf, "Learning with Local and Global Consistency," Advances in Neural Information Processing Systems, vol. 16, MIT Press, 2003.
[17] X. Zhu, Z. Ghahramani, and J. Lafferty, "Semi-Supervised Learning Using Gaussian Fields and Harmonic Functions," Proc. 20th Int'l Conf. Machine Learning, 2003.
[18] W. Buntine, "Variational Extensions to EM and Multinomial PCA," Proc. 13th European Conf. Machine Learning, 2002.
[19] E. Gaussier and C. Goutte, "Relation between PLSA and NMF and Implications," Proc. Ann. ACM SIGIR Conf. Research and Development in Information Retrieval, pp. 601-602, 2005.
[20] C. Ding, T. Li, and W. Peng, "Nonnegative Matrix Factorization and Probabilistic Latent Semantic Indexing: Equivalence Chi-Square Statistics and a Hybrid Method," Proc. AAAI Conf. Artificial Intelligence, 2006.
[21] P.O. Hoyer, "Non-Negative Matrix Factorization with Sparseness Constraints," J. Machine Learning Research, vol. 5, pp. 1457-1469, 2004.
[22] D. Dueck, Q.D. Morris, and B.J. Frey, "Probabilistic Sparse Matrix Factorization," Technical Report PSI-2004-23, Univ. of Toronto, 2004.
[23] P. Paatero and U. Tapper, "Positive Matrix Factorization: A Non-Negative Factor Model with Optimal Utilization of Error Estimates of Data Values," Environmetrics, vol. 5, pp. 111-126, 1994.
[24] D.D. Lee and H.S. Seung, "Algorithms for Non-Negative Matrix Factorization," Advances in Neural Information Processing Systems, MIT Press, 2001.
[25] W. Liu, N. Zheng, and X. Lu, "Non-Negative Matrix Factorization for Visual Coding," Proc. IEEE Int'l Conf. Acoustics, Speech and Signal Processing, 2003.
[26] C. Ding, T. Li, and M.I. Jordan, "Convex and Semi-Nonnegative Matrix Factorizations," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 45-55, Jan. 2010.
[27] C. Ding, X. He, and H.D. Simon, "On the Equivalence of Nonnegative Matrix Factorization and Spectral Clustering," Proc. SIAM Data Mining Conf., pp. 606-610, 2005.
[28] C.-J. Lin, "Projected Gradient Methods for Nonnegative Matrix Factorization," Neural Computation, vol. 19, no. 10, pp. 2756-2779, 2007.
[29] C.-J. Lin, "On the Convergence of Multiplicative Update Algorithms for Nonnegative Matrix Factorization," IEEE Trans. Neural Networks, vol. 18, no. 6, pp. 1589-1596, Nov. 2007.
[30] M. Heiler, C. Schnörr, P. Bennett, and E. Parrado-Hernandez, "Learning Sparse Representations by Non-Negative Matrix Factorization and Sequential Cone Programming," J. Machine Learning Research, vol. 7, pp. 1385-1407, 2006.
[31] A. Dempster, N. Laird, and D. Rubin, "Maximum Likelihood from Incomplete Data via the Em Algorithm," J. Royal Statistics Soc., vol. 39, pp. 1-38, 1977.
[32] L. Saul and F. Pereira, "Aggregate and Mixed-Order Markov Models for Statistical Language Processing," Proc. Second Conf. Empirical Methods in Nature Language Processing, pp. 81-89, 1977.
[33] J. Kivinen and M. Warmuth, "Additive versus Exponentiated Gradient Updates for Linear Prediction," J. Information and Computation, vol. 132, pp. 1-64, 1997.
[34] L. Lovasz and M. Plummer, Matching Theory. North Holland, 1986.
[35] A. Shashua and T. Hazan, "Non-Negative Tensor Factorization with Applications to Statistics and Computer Vision," Proc. Int'l Conf. Machine Learning, pp. 793-800, 2005.
[36] W. Xu and Y. Gong, "Document Clustering by Concept Factorization," Proc. Ann. ACM SIGIR Conf. Research and Development in Information Retrieval, 2004.
[37] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[38] J. Philbin, O. Chum, M. Isard, J. Sivic, and A. Zisserman, "Object Retrieval with Large Vocabularies and Fast Spatial Matching," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2007.

Index Terms:
Nonnegative matrix factorization, semi-supervised learning, dimension reduction, clustering.
Citation:
Haifeng Liu, Zhaohui Wu, Deng Cai, Thomas S. Huang, "Constrained Nonnegative Matrix Factorization for Image Representation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 7, pp. 1299-1311, July 2012, doi:10.1109/TPAMI.2011.217
Usage of this product signifies your acceptance of the Terms of Use.