This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video
May 2008 (vol. 30 no. 5)
pp. 865-877
Recovering articulated shape and motion, especially human body motion, from video is a challenging problem with a wide range of applications in medical study, sport analysis and animation, etc. Previous work on articulated motion recovery generally requires prior knowledge of the kinematic chain and usually does not concern the recovery of the articulated shape. The non-rigidity of some articulated part, e.g. human body motion with nonrigid facial motion, is completely ignored. We propose a factorization-based approach to recover the shape, motion and kinematic chain of an articulated object with nonrigid parts altogether directly from video sequences under a unified framework. The proposed approach is based on our modeling of the articulated non-rigid motion as a set of intersecting motion subspaces. A motion subspace is the linear subspace of the trajectories of an object. It can model a rigid or non-rigid motion. The intersection of two motion subspaces of linked parts models the motion of an articulated joint or axis. Our approach consists of algorithms for motion segmentation, kinematic chain building, and shape recovery. It handles outliers and can be automated. We test our approach through synthetic and real experiments and demonstrate how to recover articulated structure with non-rigid parts via a single-view camera without prior knowledge of its kinematic chain.

[1] A. Bartoli and S.I. Olsen, “A Batch Algorithm for Implicit Non-Rigid Shape and Motion Recovery,” Proc. ICCV Workshop Dynamical Models for Computer Vision, 2005.
[2] T. Boult and L. Brown, “Factorization-Based Segmentation of Motions,” Proc. IEEE Workshop Visual Motion, 1991.
[3] M. Brand, “Morphable 3D Models from Video,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 456-463, 2001.
[4] M. Brand and R. Bhotika, “Flexible Flow for 3D Nonrigid Tracking and Shape Recovery,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 315-322, Dec. 2001.
[5] M. Brand, “A Direct Method for 3D Factorization of Nonrigid Motion Observed in 2D,” Proc. IEEE CS Int'l Conf. Computer Vision and Pattern Recognition, June 2005.
[6] C. Bregler, A. Hertzmann, and H. Biermann, “Recovering Non-Rigid 3D Shape from Image Streams,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2000.
[7] C. Bregler and J. Malik, “Tracking People with Twists and Exponential Maps,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 8-15, 1998.
[8] J.P. Costeira and T. Kanade, “A Multibody Factorization Method for Independently Moving Objects,” Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998.
[9] G. Ferrari-Trecate, M. Muselli, D. Liberati, and M. Morari, “A Clustering Technique for the Identification of Piecewise Affine and Hybrid Systems,” Automatica, vol. 39, pp. 205-217, 2003.
[10] C.W. Gear, “Feature Grouping in Moving Objects,” Proc. Workshop Motion of Non-Rigid and Articulated Objects, 1994.
[11] G. Golub and A. van Loan, Matrix Computations. Johns Hopkins Univ. Press, 1996.
[12] M. Han and T. Kanade, “Reconstruction of a Scene with Multiple Linearly Moving Objects,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2000.
[13] D. Hogg, “Model-Based Vision: A Program to See a Walking Person,” Image and Vision Computing, vol. 1, no. 1, pp. 5-20, 1983.
[14] N. Ichimura, “Motion Segmentation Based on Factorization Method and Discriminant Criterion,” Proc. Int'l Conf. Computer Vision, pp. 600-605, 1999.
[15] K. Kanatani, “Motion Segmentation by Subspace Separation and Model Selection: Model Selection and Reliability Evaluation,” Int'l. J. Image and Graphics, vol. 2, no. 2, pp. 179-197, 2002.
[16] A. Ng, M. Jordan, and Y. Weiss, “On Spectral Clustering: Analysis and an Algorithm,” Advances in Neural Information Processing Systems 14, MIT Press, 2002.
[17] C. Poelman and T. Kanade, “A Paraperspective Factorization Method for Shape and Motion Recovery,” Technical Report CMU-CS-92-208, Computer Science Dept., Carnegie Mellon Univ., Oct. 1992.
[18] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[19] D. Sinclair, L. Paletta, and A. Pinz, “Euclidean Structure Recovery through Articulated Motion,” Proc. 10th Scandinavian Conf. Image Analysis, pp. 991-998, 1997.
[20] C. Sminchisescu and B. Triggs, “Covariance Scaled Sampling for Monocular 3D Body Tracking,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 447-454, 2001.
[21] C.J. Taylor, “Reconstruction of Articulated Objects from Point Correspondences in a Single Image,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, p. 677, 2000.
[22] C. Tomasi and T. Kanade, “Shape and Motion from Image Streams under Orthography: A Factorization Method,” Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[23] P.H.S. Torr, “Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting,” Int'l J. Computer Vision, vol. 50, no. 1, pp. 35-61, 2002.
[24] L. Torresani, D.B. Yang, E.J. Alexander, and C. Bregler, “Tracking and Modeling Non-Rigid Objects with Rank Constraints,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 493-500, 2001.
[25] L. Torresani and A. Hertzmann, “Automatic Non-Rigid 3D Modeling from Video,” Proc. Eighth European Conf. Computer Vision, 2004.
[26] P. Tresadern and I. Reid, “Articulated Structure from Motion by Factorization,” Proc. IEEE CS Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[27] R. Vidal and R. Hartley, “Motion Segmentation with Missing Data Using PowerFactorization and GPCA,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[28] R. Vidal, Y. Ma, and S. Sastry, “Generalized Principal Component Analysis (GPCA),” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2003.
[29] R. Vidal, Y. Ma, and J. Piazzi, “A New GPCA Algorithm for Clustering Subspaces by Fitting, Differentiating and Dividing Polynomials,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June-July 2004.
[30] Y. Weiss, “Segmentation Using Eigenvectors: A Unifying View,” Proc. Int'l Conf. Computer Vision (ICCV '99), pp. 975-982, Sept. 1999.
[31] J. Xiao, J. Chai, and T. Kanade, “A Closed-Form Solution to Non-Rigid Shape and Motion Recovery,” Proc. Eighth European Conf. Computer Vision, 2004.
[32] J. Yan and M. Pollefeys, “A Factorization-Based Approach to Articulated Motion Recovery,” Proc. IEEE CS Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[33] J. Yan and M. Pollefeys, “A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-Rigid, Degenerate and Non-Degenerate,” Proc. Ninth European Conf. Computer Vision, 2006.
[34] J. Yan and M. Pollefeys, “Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[35] J. Yan and M. Pollefeys, “Articulated Motion Segmentation Using RANSAC with Priors,” Proc. ICCV Workshop Dynamical Vision, 2005.
[36] L. Zelnik-Manor and M. Irani, “Degeneracies, Dependencies and Their Implications in Multi-Body and Multi-Sequence Factorizations,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2003.
[37] H. Zhou and T. Huang, “Recovering Articulated Motion with a Hierarchical Factorization Method,” Proc. Fifth Int'l Workshop Gesture-Based Comm. in Human-Computer Interaction, pp.140-151, 2003.

Index Terms:
computer vision, 3D scene analysis, motion, shape, articulated, non-rigid, kinematic chain, factorization method
Citation:
Jingyu Yan, Marc Pollefeys, "A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 865-877, May 2008, doi:10.1109/TPAMI.2007.70739
Usage of this product signifies your acceptance of the Terms of Use.