This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories
October 2010 (vol. 32 no. 10)
pp. 1832-1845
Shankar Rao, HRL Laboratories, LLC, Malibu
Roberto Tron, Johns Hopkins University, Baltimore
René Vidal, Johns Hopkins University, Baltimore
Yi Ma, University of Illinois at Urbana-Champaign, Urbana and Microsoft Research Asia, Beijing
In this paper, we study the problem of segmenting tracked feature point trajectories of multiple moving objects in an image sequence. Using the affine camera model, this problem can be cast as the problem of segmenting samples drawn from multiple linear subspaces. In practice, due to limitations of the tracker, occlusions, and the presence of nonrigid objects in the scene, the obtained motion trajectories may contain grossly mistracked features, missing entries, or corrupted entries. In this paper, we develop a robust subspace separation scheme that deals with these practical issues in a unified mathematical framework. Our methods draw strong connections between lossy compression, rank minimization, and sparse representation. We test our methods extensively on the Hopkins155 motion segmentation database and other motion sequences with outliers and missing data. We compare the performance of our methods to state-of-the-art motion segmentation methods based on expectation-maximization and spectral clustering. For data without outliers or missing information, the results of our methods are on par with the state-of-the-art results and, in many cases, exceed them. In addition, our methods give surprisingly good performance in the presence of the three types of pathological trajectories mentioned above. All code and results are publicly available at http://perception.csl.uiuc.edu/coding/motion/.

[1] A. Barron, J. Rissanen, and B. Yu, "The Minimum Description Length Principle in Coding and Modeling," IEEE Trans. Information Theory, vol. 44, no. 6, pp. 2743-2760, Oct. 1998.
[2] E. Candes and B. Recht, "Exact Matrix Completion via Convex Optimization," Foundations of Computational Math., 2009.
[3] E. Candes, M. Rudelson, R. Vershynin, and T. Tao, "Error Correction via Linear Programming," Proc. IEEE Symp. Foundations of Computer Science, pp. 295-308, 2005.
[4] E. Candes and T. Tao, "Decoding by Linear Programming," IEEE Trans. Information Theory, vol. 51, no. 12, pp. 4203-4215, Dec. 2005.
[5] J. Costeira and T. Kanade, "A Multibody Factorization Method for Independently Moving Objects," Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998.
[6] M. Datar, N. Immorlica, P. Indyk, and V. Mirrokni, "Locality-Sensitive Hashing Scheme Based of $p$ -Stable Distributions," Proc. ACM Symp. Computational Geometry, 2004.
[7] F. De la Torre and M.J. Black, "Robust Principal Component Analysis for Computer Vision," Proc. Int'l Conf. Computer Vision, pp. 362-369, 2001.
[8] D.L. Donoho, "For Most Large Underdetermined Systems of Linear Equations the Minimal $\ell^1$ -Norm Solution Is Also the Sparsest Solution," Comm. Pure and Applied Math., vol. 59, no. 6, pp. 797-829, Mar. 2006.
[9] D.L. Donoho and J. Tanner, "Counting Faces of Randomly Projected Polytopes When the Projection Radically Lowers Dimension," J. Am. Math. Soc., vol. 22, no. 1, pp. 1-53, 2009.
[10] Z. Fan, J. Zhou, and Y. Wu, "Multibody Grouping by Inference of Multiple Subspaces from High Dimensional Data Using Oriented-Frames," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 90-105, Jan. 2006.
[11] M. Fazel, H. Hindi, and S. Boyd, "Log-Det Heuristic for Matrix Rank Minimization with Applications to Hankel and Euclidean Distance Matrices," Proc. Am. Control Conf., pp. 2156-2162, June 2003.
[12] C. Gear, "Multibody Grouping from Motion Images," Int'l J. Computer Vision, vol. 29, no. 2, pp. 133-150, 1998.
[13] M. Grant and S. Boyd, "CVX: MATLAB Software for Disciplined Convex Programming [Web Page and Software]," http://www.stanford.edu/~boydcvx/, Nov. 2007.
[14] A. Gruber and Y. Weiss, "Multibody Factorization with Uncertainty and Missing Data Using the EM Algorithm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 769-775, 2004.
[15] R. Hartley and F. Schaffalitzky, "PowerFactorization: An Approach to Affine Reconstruction with Missing and Uncertain Data," Proc. Australia-Japan Advanced Workshop Computer Vision, 2003.
[16] N. Ichimura, "Motion Segmentation Based on Factorization and Discriminant Criterion," Proc. IEEE Int'l Conf. Computer Vision, pp. 600-605, 1999.
[17] K. Kanatani, "Motion Segmentation by Subspace Separation and Model Selection," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 586-591, 2001.
[18] K. Kanatani and Y. Sugaya, "Multi-Stage Optimization for Multi-Body Motion Segmentation," Proc. Australia-Japan Advanced Workshop Computer Vision, 2003.
[19] Q. Ke and T. Kanade, "Robust $\ell^1$ -Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 739-746, 2005.
[20] Y. Ma, H. Derksen, W. Hong, and J. Wright, "Segmentation of Multivariate Mixed Data via Lossy Coding and Compression," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1546-1562, Sept. 2007.
[21] B. Recht, M. Fazel, and P.A. Parillo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization," SIAM Rev., 2007.
[22] K. Schindler, D. Suter, and H. Wang, "A Model-Selection Framework for Multibody Structure-and-Motion of Image Sequences," Int'l J. Computer Vision, vol. 79, no. 2, pp. 159-177, Aug. 2008.
[23] P. Torr, "Geometric Motion Segmentation and Model Selection," Philosophical Trans. Royal Soc. of London, vol. 356, pp. 1321-1340, 1998.
[24] R. Tron and R. Vidal, "A Benchmark for the Comparison of 3D Motion Segmentation Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[25] L. Vandenberghe and S. Boyd, "Semidefinite Programming," SIAM Rev., vol. 38, no. 1, pp. 49-95, Mar. 1996.
[26] R. Vidal, Y. Ma, and S. Sastry, "Generalized Principal Component Analysis (GPCA)," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1945-1959, Dec. 2005.
[27] R. Vidal, R. Tron, and R. Hartley, "Multiframe Motion Segmentation with Missing Data Using PowerFactorization and GPCA," Int'l J. Computer Vision, vol. 79, no. 1, pp. 85-105, 2007.
[28] J. Wright and Y. Ma, "Dense Error Correction via $\ell^1$ Minimization," IEEE Trans. Information Theory, 2008.
[29] J. Wright, Y. Ma, Y. Tao, Z. Lin, and H.-Y. Shum, "Classification via Minimum Incremental Coding Length," SIAM J. Imaging Sciences, vol. 2, no. 2, pp. 367-395, 2009.
[30] J. Wright, A.Y. Yang, A. Ganesh, S.S. Sastry, and Y. Ma, "Robust Face Recognition via Sparse Representation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210-227, Feb. 2009.
[31] J. Yan and M. Pollefeys, "A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-Rigid, Degenerate and Non-Degenerate," Proc. European Conf. Computer Vision, pp. 94-106, 2006.
[32] A.Y. Yang, S. Rao, and Y. Ma, "Robust Statistical Estimation and Segmentation of Multiple Subspaces," Proc. IEEE Computer Vision and Pattern Recognition Workshop 25 Years of RANSAC, 2006.
[33] L. Zelnik-Manor and M. Irani, "Degeneracies, Dependencies and Their Implications in Multi-Body and Multi-Sequence Factorization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 287-293, 2003.

Index Terms:
Motion segmentation, subspace separation, lossy compression, incomplete data, error correction, sparse representation, matrix rank minimization.
Citation:
Shankar Rao, Roberto Tron, René Vidal, Yi Ma, "Motion Segmentation in the Presence of Outlying, Incomplete, or Corrupted Trajectories," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1832-1845, Oct. 2010, doi:10.1109/TPAMI.2009.191
Usage of this product signifies your acceptance of the Terms of Use.