Subscribe
Issue No.01 - Jan. (2013 vol.35)
pp: 171-184
Guangcan Liu , Dept. of Comput. Sci. & Eng., Shanghai Jiao Tong Univ., Shanghai, China
Zhouchen Lin , Key Lab. of Machine Perception (MOE), Peking Univ., Beijing, China
Shuicheng Yan , Dept. of Electr. & Comput. Eng., Nat. Univ. of Singapore, Singapore, Singapore
Ju Sun , Dept. of Electr. Eng., Columbia Univ., New York, NY, USA
Yong Yu , Dept. of Comput. Sci. & Eng., Shanghai Jiao Tong Univ., Shanghai, China
Yi Ma , Visual Comput. Group, Microsoft Res. Asia, Beijing, China
ABSTRACT
In this paper, we address the subspace clustering problem. Given a set of data samples (vectors) approximately drawn from a union of multiple subspaces, our goal is to cluster the samples into their respective subspaces and remove possible outliers as well. To this end, we propose a novel objective function named Low-Rank Representation (LRR), which seeks the lowest rank representation among all the candidates that can represent the data samples as linear combinations of the bases in a given dictionary. It is shown that the convex program associated with LRR solves the subspace clustering problem in the following sense: When the data is clean, we prove that LRR exactly recovers the true subspace structures; when the data are contaminated by outliers, we prove that under certain conditions LRR can exactly recover the row space of the original data and detect the outlier as well; for data corrupted by arbitrary sparse errors, LRR can also approximately recover the row space with theoretical guarantees. Since the subspace membership is provably determined by the row space, these further imply that LRR can perform robust subspace clustering and error correction in an efficient and effective way.
INDEX TERMS
Robustness, Noise, Dictionaries, Optimization, Polynomials, Data models, Vectors,outlier detection, Low-rank representation, subspace clustering, segmentation
CITATION
Guangcan Liu, Zhouchen Lin, Shuicheng Yan, Ju Sun, Yong Yu, Yi Ma, "Robust Recovery of Subspace Structures by Low-Rank Representation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 171-184, Jan. 2013, doi:10.1109/TPAMI.2012.88
REFERENCES
 [1] W. Gear, "Multibody Grouping from Motion Images," Int'l J. Computer Vision, vol. 29, no. 2, pp. 133-150, 1998. [2] J. Yan and M. Pollefeys, "A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-Rigid, Degenerate and Non-Degenerate," Proc. European Conf. Computer Vision, vol. 4, pp. 94-106, 2006. [3] S. Rao, R. Tron, R. Vidal, and Y. Ma, "Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1832-1845, Oct. 2010. [4] G. Liu and S. Yan, "Latent Low-Rank Representation for Subspace Segmentation and Feature Extraction," Proc. IEEE Int'l Conf. Computer Vision, 2011. [5] Y. Ma, H. Derksen, W. Hong, and J. Wright, "Segmentation of Multivariate Mixed Data via Lossy Data Coding and Compression," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1546-1562, Sept. 2007. [6] E. Candès and Y. Plan, "Matrix Completion with Noise," Proc. IEEE, vol. 98, no. 6, pp. 925-936, June 2010. [7] E. Candès, X. Li, Y. Ma, and J. Wright, "Robust Principal Component Analysis?" J. ACM, 2009. [8] J. Ho, M. Yang, J. Lim, K. Lee, and D. Kriegman, "Clustering Appearances of Objects under Varying Illumination Conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 11-18, 2003. [9] G. Liu, Z. Lin, X. Tang, and Y. Yu, "Unsupervised Object Segmentation with a Hybrid Graph Model (HGM)," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 910-924, May 2010. [10] M. Fischler and R. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. ACM, vol. 24, no. 6, pp. 381-395, 1981. [11] C. Zhang and R. Bitmead, "Subspace System Identification for Training-Based MIMO Channel Estimation," Automatica, vol. 41, no. 9, pp. 1623-1632, 2005. [12] J. Costeira and T. Kanade, "A Multibody Factorization Method for Independently Moving Objects," Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998. [13] E. Elhamifar and R. Vidal, "Sparse Subspace Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2790-2797, 2009. [14] G. Liu, Z. Lin, and Y. Yu, "Robust Subspace Segmentation by Low-Rank Representation," Proc. Int'l Conf. Machine Learning, pp. 663-670, 2010. [15] M. Soltanolkotabi and E. Candès, "A Geometric Analysis of Subspace Clustering with Outliers," arXiv:1112.4258v2, 2011. [16] H. Xu, C. Caramanis, and S. Sanghavi, "Robust PCA via Outlier Pursuit," Systems Advances in Neural Information Processing Systems, vol. 23, pp. 2496-2504, 2010. [17] M. Fazel, "Matrix Rank Minimization with Applications," PhD thesis, 2002. [18] B. Chen, G. Liu, Z. Huang, and S. Yan, "Multi-Task Low-Rank Affinities Pursuit for Image Segmentation," Proc. IEEE Int'l Conf. Computer Vision, 2011. [19] C. Lang, G. Liu, J. Yu, and S. Yan, "Saliency Detection by Multi-Task Sparsity Pursuit," IEEE Trans. Image Processing, vol. 21, no. 3, pp. 1327-1338, Mar. 2012. [20] Y. Eldar and M. Mishali, "Robust Recovery of Signals from a Structured Union of Subspaces," IEEE Trans. Information Theory, vol. 55, no. 11, pp. 5302-5316, Nov. 2009. [21] A. Gruber and Y. Weiss, "Multibody Factorization with Uncertainty and Missing Data Using the EM Algorithm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 707-714, 2004. [22] T. Zhang, A. Szlam, and G. Lerman, "Median K-Flats for Hybrid Linear Modeling with Many Outliers," Proc. Workshop Subspace Methods, 2009. [23] A. Yang, S. Rao, and Y. Ma, "Robust Statistical Estimation and Segmentation of Multiple Subspaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006. [24] Y. Ma, A. Yang, H. Derksen, and R. Fossum, "Estimation of Subspace Arrangements with Applications in Modeling and Segmenting Mixed Data," SIAM Rev., vol. 50, no. 3, pp. 413-458, 2008. [25] S. Rao, A. Yang, S. Sastry, and Y. Ma, "Robust Algebraic Segmentation of Mixed Rigid-Body and Planar Motions in Two Views," Int'l J. Computer Vision, vol. 88, no. 3, pp. 425-446, 2010. [26] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000. [27] G. Chen and G. Lerman, "Spectral Curvature Clustering (SCC)," Int'l J. Computer Vision, vol. 81, pp. 317-330, 2009. [28] G. Chen and G. Lerman, "Foundations of a Multi-Way Spectral Clustering Framework for Hybrid Linear Modeling," Foundations of Computational Math., vol. 9, pp. 517-558, 2009. [29] T. Zhang, A. Szlam, Y. Wang, and G. Lerman, "Hybrid Linear Modeling via Local Best-Fit Flats," arXiv:1010.3460, 2011. [30] E. Arias-Castro, G. Chen, and G. Lerman, "Spectral Clustering Based on Local Linear Approximations," Electronic J. Statistics, vol. 5, pp. 1537-1587, 2011. [31] F. Lauer and C. Schnórr, "Spectral Clustering of Linear Subspaces for Motion Segmentation," Proc. IEEE Int'l Conf. Computer Vision, 2009. [32] D. Donoho, "For Most Large Underdetermined Systems of Linear Equations the Minimal $\ell_1$ -Norm Solution Is Also the Sparsest Solution," Comm. Pure and Applied Math., vol. 59, pp. 797-829, 2004. [33] B. Nasihatkon and R. Hartley, "Graph Connectivity in Sparse Subspace Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2137-2144, 2011. [34] G. Lerman and T. Zhang, "Robust Recovery of Multiple Subspaces by Geometric $\ell_p$ Minimization," arXiv:1104.3770, 2011. [35] Z. Zhang, X. Liang, A. Ganesh, and Y. Ma, "TILT: Transform Invariant Low-Rank Textures," Proc. Asian Conf. Computer Vision, 2010. [36] Z. Lin, M. Chen, and Y. Ma, "The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices," UIUC Technical Report UILU-ENG-09-2215, 2009. [37] D. Bertsekas, Constrained Optimization and Lagrange Multiplier Methods. Academic Press, 1982. [38] J. Cai, E. Candès, and Z. Shen, "A Singular Value Thresholding Algorithm for Matrix Completion," SIAM J. Optimization, vol. 20, no. 4, pp. 1956-1982, 2010. [39] J. Yang, W. Yin, Y. Zhang, and Y. Wang, "A Fast Algorithm for Edge-Preserving Variational Multichannel Image Restoration," SIAM J. Imaging Sciences, vol. 2, no. 2, pp. 569-592, 2009. [40] Y. Zhang, "Recent Advances in Alternating Direction Methods: Practice and Theory," tutorial, 2010. [41] J. Eckstein and D. Bertsekas, "On the Douglas-Rachford Splitting Method and the Proximal Point Algorithm for Maximal Monotone Operators," Math. Programming, vol. 55, pp. 293-318, 1992. [42] S. Wei and Z. Lin, "Analysis and Improvement of Low Rank Representation for Subspace Segmentation," arXiv:1107.1561, 2010. [43] G. Liu, H. Xu, and S. Yan, "Exact Subspace Segmentation and Outlier Detection by Low-Rank Representation," Proc. Int'l Conf. Artificial Intelligence and Statistics, 2012. [44] R. Vidal, Y. Ma, and J. Piazzi, "A New GPCA Algorithm for Clustering Subspaces by Fitting, Differentiating and Dividing Polynomials," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 510-517, 2004. [45] K. Huang, Y. Ma, and R. Vidal, "Minimum Effective Dimension for Mixtures of Subspaces: A Robust GPCA Algorithm and Its Applications," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 631-638, 2004. [46] R. Tron and R. Vidal, "A Benchmark for the Comparison of 3-D Motion Segmentation Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007. [47] K. Lee, J. Ho, and D. Kriegman, "Acquiring Linear Subspaces for Face Recognition Under Variable Lighting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 684-698, May 2005. [48] F.-F. Li, R. Fergus, and P. Perona, "Learning Generative Visual Models from Few Training Examples: An Incremental Bayesian Approach Tested on 101 Object Categories," Proc. Conf. Computer Vision and Pattern Recognition, pp. 178-188. 2004, [49] Y. Sugaya and K. Kanatani, "Multi-Stage Unsupervised Learning for Multi-Body Motion Segmentation," IEICE Trans. Information Systems, pp. 1935-1942, 2004. [50] A. Goh and R. Vidal, "Segmenting Motions of Different Types by Unsupervised Manifold Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007. [51] P. Favaro, R. Vidal, and A. Ravichandran, "A Closed Form Solution to Robust Subspace Estimation and Clustering," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.