The Community for Technology Leaders
RSS Icon
Issue No.07 - July (2013 vol.35)
pp: 1717-1729
Zhenyue Zhang , Dept. of Math., Zhejiang Univ., Hangzhou, China
Keke Zhao , Dept. of Math., Zhejiang Univ., Hangzhou, China
This paper proposes a new model of low-rank matrix factorization that incorporates manifold regularization to the matrix factorization. Superior to the graph-regularized nonnegative matrix factorization, this new regularization model has globally optimal and closed-form solutions. A direct algorithm (for data with small number of points) and an alternate iterative algorithm with inexact inner iteration (for large scale data) are proposed to solve the new model. A convergence analysis establishes the global convergence of the iterative algorithm. The efficiency and precision of the algorithm are demonstrated numerically through applications to six real-world datasets on clustering and classification. Performance comparison with existing algorithms shows the effectiveness of the proposed method for low-rank factorization in general.
Approximation methods, Sparse matrices, Manifolds, Symmetric matrices, Vectors, Matrix decomposition, Algorithm design and analysis,manifold learning, Matrix factorization, graph regularization, classification, clustering
Zhenyue Zhang, Keke Zhao, "Low-Rank Matrix Approximation with Manifold Regularization", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1717-1729, July 2013, doi:10.1109/TPAMI.2012.274
[1] T. Ahonen, A. Hadid, and M. Pietikäinen, "Face Description with Local Binary Patterns: Application to Face Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037-2041, Dec. 2006.
[2] M. Belkin and P. Niyogi, "Laplacian Eigenmaps and Spectral Techniques for Embedding and Clustering," Proc. Advances in Neural Information Processing Systems, vol. 14, pp. 585-591, 2001.
[3] M. Belkin, P. Niyogi, and V. Sindhwani, "Manifold Regularization: A Geometric Framework for Learning from Examples," J. Machine Learning Research, vol. 7, pp. 2399-2434, 2006.
[4] D. Cai, X. He, J. Han, and T.S. Huang, "Graph Regularized Nonnegative Matrix Factorization for Data Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1548-1560, Aug. 2011.
[5] J.F. Canny, "GaP: A Factor Model for Discrete Data," Proc. ACM Conf. Information Retrieval, pp. 122-129, 2004.
[6] Y. Chen, D. Pavlov, M. Kapralov, and J.F. Canny, "Factor Modeling for Advertisement Targeting," Proc. Advances in Neural Information Processing Systems, vol. 22, pp. 324-332, 2009.
[7] J.E. Cohen and U.G. Rothblum, "Nonnegative Ranks, Decompositions, and Factorizations of Nonnegative Matrices," Linear Algebra and Its Applications, vol. 190, pp. 149-168, 1993.
[8] B. Dong, M.T. Lin, and M.T. Chu, "Nonnegative Rank Factorization via Rank Reduction," NRF04d.pdf, 2013.
[9] R. Fan, K. Chang, C. Hsieh, X. Wang, and C. Lin, "LIBLINEAR: A Library for Large Linear Classification," J. Machine Learning Research, vol. 9, pp. 1871-1874, 2008.
[10] G.H. Golub and C.F. Van Loan, Matrix Computations. The Johns Hopkins Univ. Press, 1996.
[11] N. Guan, D. Tao, Z. Luo, and B. Yuan, "Manifold Regularized Discriminative Nonnegative Matrix Factorization with Fast Gradient Descent," IEEE Trans. Image Processing, vol. 20, no. 7, pp. 2030-2048, July 2011.
[12] X. He and P. Niyogi, "Locality Preserving Projections," Proc. Advances in Neural Information Processing Systems 16, 2004.
[13] J. Kim and H. Park, "Fast Nonnegative Matrix Factorization: An Active-Set-Like Method and Comparisons," SIAM J. Scientific Computing, vol. 33, no. 6, pp. 3261-3281, 2011.
[14] Y. Koren, "Factorization Meets the Neighborhood: A Multifaceted Collaborative Filtering Model," Proc. 14th ACM SIGKDD Int'l Conf. Knowledge Discovery and Data Mining, pp. 426-434, 2008.
[15] Y. Koren, R. Bell, and C. Volinsky, "Matrix Factorization Techniques for Recommender Systems," Computer, vol. 42, no. 8, pp. 30-37, Aug. 2009.
[16] D. Lee and H. Seung, "Algorithms for Nonnegative Matrix Factorization," Proc. Advances in Neural Information Processing Systems vol. 13, pp. 556-562, 2001.
[17] D. Kuang, C. Ding, and H. Park, "Symmetric Nonnegative Matrix Factorization for Graph Clustering," Proc. SIAM Int'l Conf. Data Mining, pp. 106-117, 2012.
[18] W. Li and D. Yeung, "Relation Regularized Matrix Factorization," Proc. 21st Int'l Joint Conf. Artificial Intelligence, 2009.
[19] Q. Lu and L. Getoor, "Link-Based Classification," Proc. 20th Int'l Conf. Machine Learning, Aug. 2003.
[20] H. Ma, M.R. Lyu, and I. King, "Learning to Recommend with Trust and Distrust Relationships," Proc. Third ACM Conf. Recommender Systems, vol. 8, pp. 189-196, 2009.
[21] S.T. Roweis and L.K. Saul, "Nonlinear Dimensionality Reduction by Locally Linear Embedding," Science, vol. 290, pp. 2323-2326, 2000.
[22] Y. Saad, Iterative Methods for Sparse Linear Systems, second ed. SIAM, 2003.
[23] J.B. Tenenbaum, V. de Silva, and J.C. Langford, "A Global Geometric Framework for Nonlinear Dimensionality Reduction," Science, vol. 290, pp. 2319-2323, 2000.
[24] W. Xu, X. Liu, and Y. Gong, "Document Clustering Based on Non-Negative Matrix Factorization," Proc. Int'l Conf. Research and Development in Information Retrieval, pp. 267-273, Aug. 2003.
[25] H. Zha and Z. Zhang, "Spectral Properties of the Alignment Matrices in Manifold Learning," SIAM Rev., vol. 51, no. 3, pp. 545-566, 2009.
[26] Z. Zhang and H. Zha, "Principal Manifolds and Nonlinear Dimension Reduction via Local Tangent Space Alignment," SIAM J. Scientific Computing, vol. 26, no. 1, pp. 313-338, 2005.
[27] Z. Zhang, H. Zha, and M. Zhang, "Spectral Methods for Semi-Supervised Manifold Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[28] Z. Zhang, K. Zhao, and H. Zha, "Inducible Regularization for Low-Rank Matrix Factorizations for Collaborative Filtering," Neurocomputing, vol. 97, pp. 52-62, 2012.
[29] D. Zhou, S. Zhu, K. Yu, X. Song, B.L. Tseng, H. Zha, and C. Giles, "Learning Multiple Graphs for Document Recommendations," Proc. 17th Int'l Conf. World Wide Web, pp. 141-150, 2008.
[30] S. Zhu, K. Yu, Y. Chi, and Y. Gong, "Combining Content and Link for Classification Using Matrix Factorization," Proc. 30th Ann. Int'l ACM SIGIR Conf. Research and Development in Information Retrieval, 2007.
[31] Y. Wen, A. Ray, and S. Phoha, "Hilbert Space Formulation of Symbolic Systems for Signal Representation and Analysis," Signal Processing, to appear.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool