The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2014 vol.36)
pp: 577-591
Yi-Lei Chen , Dept. of Comput. Sci., Nat. Tsing Hua Univ., Hsinchu, Taiwan
Chiou-Ting Hsu , Dept. of Comput. Sci., Nat. Tsing Hua Univ., Hsinchu, Taiwan
Hong-Yuan Mark Liao , Inst. of Inf. Sci., Acad. Sinica, Taipei, Taiwan
ABSTRACT
The success of research on matrix completion is evident in a variety of real-world applications. Tensor completion, which is a high-order extension of matrix completion, has also generated a great deal of research interest in recent years. Given a tensor with incomplete entries, existing methods use either factorization or completion schemes to recover the missing parts. However, as the number of missing entries increases, factorization schemes may overfit the model because of incorrectly predefined ranks, while completion schemes may fail to interpret the model factors. In this paper, we introduce a novel concept: complete the missing entries and simultaneously capture the underlying model structure. To this end, we propose a method called simultaneous tensor decomposition and completion (STDC) that combines a rank minimization technique with Tucker model decomposition. Moreover, as the model structure is implicitly included in the Tucker model, we use factor priors, which are usually known a priori in real-world tensor objects, to characterize the underlying joint-manifold drawn from the model factors. By exploiting this auxiliary information, our method leverages two classic schemes and accurately estimates the model factors and missing entries. We conducted experiments to empirically verify the convergence of our algorithm on synthetic data and evaluate its effectiveness on various kinds of real-world data. The results demonstrate the efficacy of the proposed method and its potential usage in tensor-based applications. It also outperforms state-of-the-art methods on multilinear model analysis and visual data completion tasks.
INDEX TERMS
Tensile stress, Equations, Matrix decomposition, Mathematical model, Approximation methods, Visualization, Brain modeling,multilinear model analysis, Tensor completion, Tucker decomposition, factor priors
CITATION
Yi-Lei Chen, Chiou-Ting Hsu, Hong-Yuan Mark Liao, "Simultaneous Tensor Decomposition and Completion Using Factor Priors", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.36, no. 3, pp. 577-591, March 2014, doi:10.1109/TPAMI.2013.164
REFERENCES
[1] T. Ding, M. Sznaier, and O.I. Camps, "A Rank Minimization Approach to Video Inpainting," Proc. 11th IEEE Int'l Conf. Computer Vision (ICCV), pp. 1-8, 2007.
[2] Y.X. Wang and Y.J. Zhang, "Image Inpainting via Weighted Sparse Non-Negative Matrix Factorization," Proc. 18th IEEE Int'l Conf. Image Processing (ICIP), pp. 3409-3412, 2011.
[3] H. Ji, C. Liu, Z. Shen, and Y. Xu, "Robust Video Denoising Using Low Rank Matrix Completion," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 1791-1798, 2010.
[4] Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, "RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 763-770, 2010.
[5] Z. Zhang, A. Ganesh, X. Liang, and Y. Ma, "TILT: Transform Invariant Low-Rank Textures," Int'l J. Computer Vision, vol. 99, no. 1, pp. 1-24, 2012.
[6] C. Lang, G. Liu, J. Yu, and S. Yan, "Saliency Detection by Multitask Sparsity Pursuit," IEEE Trans. Image Processing, vol. 21, no. 3, pp. 1327-1338, Mar. 2012.
[7] L. Zhuang, H. Gao, J. Huang, and N. Yu, "Semi-Supervised Classification via Low-Rank Graph," Proc. Sixth Int'l Conf. Image and Graphics, pp. 511-516, 2011.
[8] G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, "Robust Recovery of Subspace Structures by Low-Rank Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 171-184, Jan. 2013.
[9] K. Li, Q. Dai, W. Xu, and J. Jiang, "Three-Dimensional Motion Estimation via Matrix Completion," IEEE Trans. Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 42, no. 2, pp. 539-551, Apr. 2012.
[10] X. Zhou, C. Yang, and W. Yu, "Moving Object Detection by Detecting Contiguous Outliers in the Low-Rank Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 3, pp. 597-610, Mar. 2013.
[11] O. Oreifej, X. Lin, and M. Shah, "Simultaneous Video Stabilization and Moving Object Detection in Turbulence," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no. 2, pp. 450-462, Feb. 2013.
[12] R.O. Duda, P.E. Hart, and D.G. Stork, Pattern Classification. Wiley Interscience, 2000.
[13] D.D. Lee and H.S. Seung, "Learning the Parts of Objects by Nonnegative Matrix Factorization," Nature, vol. 401, pp. 788-791, 1999.
[14] S. Ma, D. Goldfarb, and L. Chen, "Fixed Point and Bregman Iterative Methods for Matrix Rank Minimization," Math. Programming, vol. 128, no. 1, pp. 321-353, 2009.
[15] J.F. Cai, E.J. Candes, and Z. Shen, "A Singular Value Algorithm for Matrix Completion," SIAM J. Optimization, vol. 20, no. 4, pp. 1956-1982, 2010.
[16] X. Lu, T. Gong, P. Yan, Y. Yuan, and X. Li, "Robust Alternative Minimization for Matrix Completion," IEEE Trans. Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 42, no. 3, pp. 939-949, June 2012.
[17] B. Recht, M. Fazel, and P. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010.
[18] W.J. Li and D.Y. Yeung, "Relation Regularized Matrix Factorization," Proc. 21st Int'l Joint Conf. Artificial Intelligence, pp. 1126-1131, 2009.
[19] A.D. Bue, J. Xavier, L. Agapito, and M. Paladini, "Bilinear Modeling via Augmented Lagrange Multipliers (BALM)," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 8, pp. 1496-1508, Aug. 2012.
[20] J. Chen and Y. Saad, "On the Tensor SVD and the Optimal Low Rank Orthogonal Approximation of Tensors," SIAM J. Matrix Analysis and Applications, vol. 30, no. 4, pp. 1709-1734, 2009.
[21] E. Acar, D.M. Dunlavy, T.G. Kolda, and M. Morup, "Scalable Tensor Factorization for Incomplete Data," Chemometrics and Intelligent Laboratory Systems, vol. 106, pp. 41-56, 2011.
[22] X. Geng, K. Smith-Miles, Z.H. Zhou, and L. Wang, "Face Image Modeling by Multilinear Subspace Analysis with Missing Values," IEEE Trans. Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 41, no. 3, pp. 881-892, June 2011.
[23] J. Liu, P. Wonka, and J. Ye, "Tensor Completion for Estimating Missing Values in Visual Data," Proc. 12th IEEE Int'l Conf. Computer Vision (ICCV), 2009.
[24] Y. Li, J. Yan, Y. Zhou, and J. Yang, "Optimum Subspace Learning and Error Correction for Tensors," Proc. 11th European Conf. Computer Vision (ECCV), 2010.
[25] S. Gandy, B. Recht, and I. Yamada, "Tensor Completion and Low-N-Rank Tensor Recovery via Convex Optimization," Inverse Problems, vol. 27, no. 2, 2011.
[26] R. Tomioka, K. Hayashi, and H. Kashima, "Estimation of Low-Rank Tensors via Convex Optimization," Arxiv Preprint arXiv:1010.0789, 2011.
[27] J. Liu, P. Musialski, P. Wonka, and J. Ye, "Tensor Completion for Estimating Missing Values in Visual Data," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 35, no.1, pp. 208-220, Jan. 2013.
[28] M. Signoretto, Q.T. Dinh, L.D. Lathauwer, and J.A.K. Suykens, "Learning with Tensors: A Framework Based on Convex Optimization and Spectral Regularization," Machine Learning, May 2013.
[29] L.D. Lathauwer, B.D. Moor, and J. Vandewalle, "A Multilinear Singular Value Decomposition," SIAM J. Matrix Analysis and Applications, vol. 21, no. 4, pp. 1254-1278, 2000.
[30] R.P. Adams, G.E. Dahl, and I. Murray, "Incorporating Side Information in Probabilistic Matrix Factorization with Gaussian Process," Proc. 26th Conf. Uncertainty in Artificial Intelligence, pp. 1-9, 2010.
[31] R. Salakhutdinov and A. Mnih, "Probabilistic Matrix Factorization," Proc. Advances in Neural Information Processing Systems (NIPS), pp. 1257-1264, 2008.
[32] A. Narita, K. Hayashi, R. Tomioka, and H. Kashima, "Tensor Factorization Using Auxiliary Information," Proc. European Conf. Machine Learning and Knowledge Discovery, pp. 501-516, 2011.
[33] Y.L. Chen and C.T. Hsu, "Multilinear Graph Embedding: Representation and Regularization for Images," IEEE Trans. Image Processing, to appear.
[34] N. Gillis and F. Glineur, "Low-Rank Matrix Approximation with Weights or Missing Data Is NP Hard," SIAM J. Matrix Analysis Applications, vol. 32, no. 4, pp. 1149-1165, 2011.
[35] D. Bertsekas, Constrained Optimization and Lagrange Multiplier Method. Academic Press, 1982.
[36] Z. Lin, M. Chen, L. Wu, and Y. Ma, "The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices," Technical Report UILU-ENG-09-2215, Univ. of Illinois at Urbana-Champaign, 2009.
[37] B. He, M. Tao, and X. Yuan, "Alternating Direction Method with Gaussian Back Substitution for Separable Convex Programming," SIAM J. Optimization, vol. 22, no. 2, pp. 313-340, 2012.
[38] Y. Shen, Z. Wen, and Y. Zhang, "Augmented Lagrangian Alternating Direction Method for Matrix Separation Based on Low-Rank Factorization," technical report, Rice Univ., 2011.
[39] G. Liu and S. Yan, "Active Subspace: Towards Scalable Low-Rank Learning," Neural Computing, vol. 24, no. 12, 2012.
[40] S. Yan, D. Xu, B. Zhang, H. Zhang, Q. Yang, and S. Lin, "Graph Embedding and Extensions: A General Framework for Dimensionality Reduction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 40-51, Jan. 2007.
[41] T. Sim, S. Baker, and M. Bsat, "The CMU Pose, Illumination, and Expression Database," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 12, pp. 1615-1618, Dec. 2003.
[42] J. Yang and X. Yuan, "Linearized Augmented Lagrangian and Alternating Direction Methods for Nuclear Norm Minimization," Math. Computation, vol. 82, pp. 301-329, 2013.
[43] M. Turk and A. Pentland, "Face Recognition Using Eigenfaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pp. 586-591, 1991.
[44] X. He, S. Yan, Y. Hu, P. Niyogi, and H. Zhang, "Face Recognition Using Laplacianfaces," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3 , pp. 328-340, Mar. 2005.
[45] P.N. Belhumeur, J.P. Hespanha, and D.J. Kriegman, "Eigenfaces vs. Fisherfaces: Recognition Using Class Specific Linear Projection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 711-720, July 1997.
[46] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.
[47] S.H. Chan, R. Khoshabeh, K.B. Gibson, P.E. Gill, and T.Q. Nguyen, "An Augmented Lagrangian Method for Total Variation Video Restoration," IEEE Trans. Image Processing, vol. 20, no. 11, pp. 3097-3111, Nov. 2011.
78 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool