CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2013 vol.35 Issue No.07 - July

Subscribe

Issue No.07 - July (2013 vol.35)

pp: 1717-1729

Zhenyue Zhang , Dept. of Math., Zhejiang Univ., Hangzhou, China

Keke Zhao , Dept. of Math., Zhejiang Univ., Hangzhou, China

ABSTRACT

This paper proposes a new model of low-rank matrix factorization that incorporates manifold regularization to the matrix factorization. Superior to the graph-regularized nonnegative matrix factorization, this new regularization model has globally optimal and closed-form solutions. A direct algorithm (for data with small number of points) and an alternate iterative algorithm with inexact inner iteration (for large scale data) are proposed to solve the new model. A convergence analysis establishes the global convergence of the iterative algorithm. The efficiency and precision of the algorithm are demonstrated numerically through applications to six real-world datasets on clustering and classification. Performance comparison with existing algorithms shows the effectiveness of the proposed method for low-rank factorization in general.

INDEX TERMS

Approximation methods, Sparse matrices, Manifolds, Symmetric matrices, Vectors, Matrix decomposition, Algorithm design and analysis,manifold learning, Matrix factorization, graph regularization, classification, clustering

CITATION

Zhenyue Zhang, Keke Zhao, "Low-Rank Matrix Approximation with Manifold Regularization",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.35, no. 7, pp. 1717-1729, July 2013, doi:10.1109/TPAMI.2012.274REFERENCES

- [1] T. Ahonen, A. Hadid, and M. Pietikäinen, "Face Description with Local Binary Patterns: Application to Face Recognition,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, Dec. 2006.- [2] M. Belkin and P. Niyogi, "Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering,"
Proc. Advances in Neural Information Processing Systems, vol. 14, pp. 585-591, 2001.- [3] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Examples,"
J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.- [4] D. Cai, X. He, J. Han, and T.S. Huang, "Graph Regularized Nonnegative Matrix Factorization for Data Representation,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1548-1560, Aug. 2011.- [5] J.F. Canny, "GaP: A Factor Model for Discrete Data,"
Proc. ACM Conf. Information Retrieval, pp. 122-129, 2004.- [6] Y. Chen, D. Pavlov, M. Kapralov, and J.F. Canny, "Factor Modeling for Advertisement Targeting,"
Proc. Advances in Neural Information Processing Systems, vol. 22, pp. 324-332, 2009.- [7] J.E. Cohen and U.G. Rothblum, "Nonnegative Ranks, Decompositions, and Factorizations of Nonnegative Matrices,"
Linear Algebra and Its Applications, vol. 190, pp. 149-168, 1993.- [8] B. Dong, M.T. Lin, and M.T. Chu, "Nonnegative Rank Factorization via Rank Reduction," http://www4.ncsu.edu/mtchu/Research/Papers NRF04d.pdf, 2013.
- [9] R. Fan, K. Chang, C. Hsieh, X. Wang, and C. Lin, "LIBLINEAR: A Library for Large Linear Classification,"
J. Machine Learning Research, vol. 9, pp. 1871-1874, 2008.- [10] G.H. Golub and C.F. Van Loan,
Matrix Computations. The Johns Hopkins Univ. Press, 1996.- [11] N. Guan, D. Tao, Z. Luo, and B. Yuan, "Manifold Regularized Discriminative Nonnegative Matrix Factorization with Fast Gradient Descent,"
IEEE Trans. Image Processing, vol. 20, no. 7, pp. 2030-2048, July 2011.- [12] X. He and P. Niyogi, "Locality Preserving Projections,"
Proc. Advances in Neural Information Processing Systems 16, 2004.- [13] J. Kim and H. Park, "Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons,"
SIAM J. Scientific Computing, vol. 33, no. 6, pp. 3261-3281, 2011.- [14] Y. Koren, "Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model,"
Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 426-434, 2008.- [15] Y. Koren, R. Bell, and C. Volinsky, "Matrix Factorization Techniques for Recommender Systems,"
Computer, vol. 42, no. 8, pp. 30-37, Aug. 2009.- [16] D. Lee and H. Seung, "Algorithms for Nonnegative Matrix Factorization,"
Proc. Advances in Neural Information Processing Systems vol. 13, pp. 556-562, 2001.- [17] D. Kuang, C. Ding, and H. Park, "Symmetric Nonnegative Matrix Factorization for Graph Clustering,"
Proc. SIAM Int'l Conf. Data Mining, pp. 106-117, 2012.- [18] W. Li and D. Yeung, "Relation Regularized Matrix Factorization,"
Proc. 21st Int'l Joint Conf. Artificial Intelligence, 2009.- [19] Q. Lu and L. Getoor, "Link-Based Classification,"
Proc. 20th Int'l Conf. Machine Learning, Aug. 2003.- [20] H. Ma, M.R. Lyu, and I. King, "Learning to Recommend with Trust and Distrust Relationships,"
Proc. Third ACM Conf. Recommender Systems, vol. 8, pp. 189-196, 2009.- [21] S.T. Roweis and L.K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding,"
Science, vol. 290, pp. 2323-2326, 2000.- [22] Y. Saad,
Iterative Methods for Sparse Linear Systems, second ed. SIAM, 2003.- [23] J.B. Tenenbaum, V. de Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction,"
Science, vol. 290, pp. 2319-2323, 2000.- [24] W. Xu, X. Liu, and Y. Gong, "Document Clustering Based on Non-Negative Matrix Factorization,"
Proc. Int'l Conf. Research and Development in Information Retrieval, pp. 267-273, Aug. 2003.- [25] H. Zha and Z. Zhang, "Spectral Properties of the Alignment Matrices in Manifold Learning,"
SIAM Rev., vol. 51, no. 3, pp. 545-566, 2009.- [26] Z. Zhang and H. Zha, "Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment,"
SIAM J. Scientific Computing, vol. 26, no. 1, pp. 313-338, 2005.- [27] Z. Zhang, H. Zha, and M. Zhang, "Spectral Methods for Semi-Supervised Manifold Learning,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.- [28] Z. Zhang, K. Zhao, and H. Zha, "Inducible Regularization for Low-Rank Matrix Factorizations for Collaborative Filtering,"
Neurocomputing, vol. 97, pp. 52-62, 2012.- [29] D. Zhou, S. Zhu, K. Yu, X. Song, B.L. Tseng, H. Zha, and C. Giles, "Learning Multiple Graphs for Document Recommendations,"
Proc. 17th Int'l Conf. World Wide Web, pp. 141-150, 2008.- [30] S. Zhu, K. Yu, Y. Chi, and Y. Gong, "Combining Content and Link for Classification Using Matrix Factorization,"
Proc. 30th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, 2007.- [31] Y. Wen, A. Ray, and S. Phoha, "Hilbert Space Formulation of Symbolic Systems for Signal Representation and Analysis,"
Signal Processing, to appear. |