The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2008 vol.30)
pp: 865-877
Jingyu Yan , IEEE
ABSTRACT
Recovering articulated shape and motion, especially human body motion, from video is a challenging problem with a wide range of applications in medical study, sport analysis and animation, etc. Previous work on articulated motion recovery generally requires prior knowledge of the kinematic chain and usually does not concern the recovery of the articulated shape. The non-rigidity of some articulated part, e.g. human body motion with nonrigid facial motion, is completely ignored. We propose a factorization-based approach to recover the shape, motion and kinematic chain of an articulated object with nonrigid parts altogether directly from video sequences under a unified framework. The proposed approach is based on our modeling of the articulated non-rigid motion as a set of intersecting motion subspaces. A motion subspace is the linear subspace of the trajectories of an object. It can model a rigid or non-rigid motion. The intersection of two motion subspaces of linked parts models the motion of an articulated joint or axis. Our approach consists of algorithms for motion segmentation, kinematic chain building, and shape recovery. It handles outliers and can be automated. We test our approach through synthetic and real experiments and demonstrate how to recover articulated structure with non-rigid parts via a single-view camera without prior knowledge of its kinematic chain.
INDEX TERMS
computer vision, 3D scene analysis, motion, shape, articulated, non-rigid, kinematic chain, factorization method
CITATION
Jingyu Yan, Marc Pollefeys, "A Factorization-Based Approach for Articulated Nonrigid Shape, Motion and Kinematic Chain Recovery From Video", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 5, pp. 865-877, May 2008, doi:10.1109/TPAMI.2007.70739
REFERENCES
[1] A. Bartoli and S.I. Olsen, “A Batch Algorithm for Implicit Non-Rigid Shape and Motion Recovery,” Proc. ICCV Workshop Dynamical Models for Computer Vision, 2005.
[2] T. Boult and L. Brown, “Factorization-Based Segmentation of Motions,” Proc. IEEE Workshop Visual Motion, 1991.
[3] M. Brand, “Morphable 3D Models from Video,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 456-463, 2001.
[4] M. Brand and R. Bhotika, “Flexible Flow for 3D Nonrigid Tracking and Shape Recovery,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 315-322, Dec. 2001.
[5] M. Brand, “A Direct Method for 3D Factorization of Nonrigid Motion Observed in 2D,” Proc. IEEE CS Int'l Conf. Computer Vision and Pattern Recognition, June 2005.
[6] C. Bregler, A. Hertzmann, and H. Biermann, “Recovering Non-Rigid 3D Shape from Image Streams,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2000.
[7] C. Bregler and J. Malik, “Tracking People with Twists and Exponential Maps,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 8-15, 1998.
[8] J.P. Costeira and T. Kanade, “A Multibody Factorization Method for Independently Moving Objects,” Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998.
[9] G. Ferrari-Trecate, M. Muselli, D. Liberati, and M. Morari, “A Clustering Technique for the Identification of Piecewise Affine and Hybrid Systems,” Automatica, vol. 39, pp. 205-217, 2003.
[10] C.W. Gear, “Feature Grouping in Moving Objects,” Proc. Workshop Motion of Non-Rigid and Articulated Objects, 1994.
[11] G. Golub and A. van Loan, Matrix Computations. Johns Hopkins Univ. Press, 1996.
[12] M. Han and T. Kanade, “Reconstruction of a Scene with Multiple Linearly Moving Objects,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2000.
[13] D. Hogg, “Model-Based Vision: A Program to See a Walking Person,” Image and Vision Computing, vol. 1, no. 1, pp. 5-20, 1983.
[14] N. Ichimura, “Motion Segmentation Based on Factorization Method and Discriminant Criterion,” Proc. Int'l Conf. Computer Vision, pp. 600-605, 1999.
[15] K. Kanatani, “Motion Segmentation by Subspace Separation and Model Selection: Model Selection and Reliability Evaluation,” Int'l. J. Image and Graphics, vol. 2, no. 2, pp. 179-197, 2002.
[16] A. Ng, M. Jordan, and Y. Weiss, “On Spectral Clustering: Analysis and an Algorithm,” Advances in Neural Information Processing Systems 14, MIT Press, 2002.
[17] C. Poelman and T. Kanade, “A Paraperspective Factorization Method for Shape and Motion Recovery,” Technical Report CMU-CS-92-208, Computer Science Dept., Carnegie Mellon Univ., Oct. 1992.
[18] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[19] D. Sinclair, L. Paletta, and A. Pinz, “Euclidean Structure Recovery through Articulated Motion,” Proc. 10th Scandinavian Conf. Image Analysis, pp. 991-998, 1997.
[20] C. Sminchisescu and B. Triggs, “Covariance Scaled Sampling for Monocular 3D Body Tracking,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 447-454, 2001.
[21] C.J. Taylor, “Reconstruction of Articulated Objects from Point Correspondences in a Single Image,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, p. 677, 2000.
[22] C. Tomasi and T. Kanade, “Shape and Motion from Image Streams under Orthography: A Factorization Method,” Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[23] P.H.S. Torr, “Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting,” Int'l J. Computer Vision, vol. 50, no. 1, pp. 35-61, 2002.
[24] L. Torresani, D.B. Yang, E.J. Alexander, and C. Bregler, “Tracking and Modeling Non-Rigid Objects with Rank Constraints,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 493-500, 2001.
[25] L. Torresani and A. Hertzmann, “Automatic Non-Rigid 3D Modeling from Video,” Proc. Eighth European Conf. Computer Vision, 2004.
[26] P. Tresadern and I. Reid, “Articulated Structure from Motion by Factorization,” Proc. IEEE CS Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[27] R. Vidal and R. Hartley, “Motion Segmentation with Missing Data Using PowerFactorization and GPCA,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2004.
[28] R. Vidal, Y. Ma, and S. Sastry, “Generalized Principal Component Analysis (GPCA),” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 2003.
[29] R. Vidal, Y. Ma, and J. Piazzi, “A New GPCA Algorithm for Clustering Subspaces by Fitting, Differentiating and Dividing Polynomials,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June-July 2004.
[30] Y. Weiss, “Segmentation Using Eigenvectors: A Unifying View,” Proc. Int'l Conf. Computer Vision (ICCV '99), pp. 975-982, Sept. 1999.
[31] J. Xiao, J. Chai, and T. Kanade, “A Closed-Form Solution to Non-Rigid Shape and Motion Recovery,” Proc. Eighth European Conf. Computer Vision, 2004.
[32] J. Yan and M. Pollefeys, “A Factorization-Based Approach to Articulated Motion Recovery,” Proc. IEEE CS Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[33] J. Yan and M. Pollefeys, “A General Framework for Motion Segmentation: Independent, Articulated, Rigid, Non-Rigid, Degenerate and Non-Degenerate,” Proc. Ninth European Conf. Computer Vision, 2006.
[34] J. Yan and M. Pollefeys, “Automatic Kinematic Chain Building from Feature Trajectories of Articulated Objects,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2006.
[35] J. Yan and M. Pollefeys, “Articulated Motion Segmentation Using RANSAC with Priors,” Proc. ICCV Workshop Dynamical Vision, 2005.
[36] L. Zelnik-Manor and M. Irani, “Degeneracies, Dependencies and Their Implications in Multi-Body and Multi-Sequence Factorizations,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2003.
[37] H. Zhou and T. Huang, “Recovering Articulated Motion with a Hierarchical Factorization Method,” Proc. Fifth Int'l Workshop Gesture-Based Comm. in Human-Computer Interaction, pp.140-151, 2003.
635 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool