The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2008 vol.30)
pp: 214-227
ABSTRACT
We propose a geometric approach to 3-D motion segmentation from point correspondences in three perspective views. We demonstrate that after applying a polynomial embedding to the point correspondences they become related by the socalled multibody trilinear constraint and its associated multibody trifocal tensor, which are natural generalizations of the trilinear constraint and the trifocal tensor to multiple motions. We derive a rank constraint on the embedded correspondences, from which one can estimate the number of independent motions as well as linearly solve for the multibody trifocal tensor. We then show how to compute the epipolar lines associated with each image point from the common root of a set of univariate polynomials and the epipoles by solving a pair of plane clustering problems using Generalized PCA (GPCA). The individual trifocal tensors are then obtained from the second order derivatives of the multibody trilinear constraint. Given epipolar lines and epipoles, or trifocal tensors, one can immediately obtain an initial clustering of the correspondences. We use this clustering to initialize an iterative algorithm that alternates between the computation of the trifocal tensors and the segmentation of the correspondences. We test our algorithm on various synthetic and real scenes, and compare with other algebraic and iterative algorithms.
INDEX TERMS
multibody structure from motion, 3-D motion segmentation, multibody trilinear constraint, multibody trifocal tensor and Generalized PCA (GPCA)
CITATION
Rene Vidal, "Three-View Multibody Structure from Motion", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 2, pp. 214-227, February 2008, doi:10.1109/TPAMI.2007.1179
REFERENCES
[1] T.E. Boult and L.G. Brown, “Factorization-Based Segmentation of Motions,” Proc. IEEE Workshop Motion Understanding, pp. 179-186, 1991.
[2] J. Costeira and T. Kanade, “A Multibody Factorization Method for Independently Moving Objects,” Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, 1998.
[3] A. Dempster, N. Laird, and D. Rubin, “Maximum Likelihood from Incomplete Data via the EM Algorithm,” J. Royal Statistical Soc. B, vol. 39, pp. 1-38, 1977.
[4] R. Duda, P. Hart, and D. Stork, Pattern Classification, second ed. John Wiley & Sons, 2000.
[5] Z. Fan, J. Zhou, and Y. Wu, “Multibody Grouping by Inference of Multiple Subspaces from High-Dimensional Data Using Oriented-Frames,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 91-105, Jan. 2006.
[6] X. Feng and P. Perona, “Scene Segmentation from 3D Motion,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 225-231, 1998.
[7] M.A. Fischler and R.C. Bolles, “RANSAC Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography,” Comm. ACM, vol. 26, pp. 381-395, 1981.
[8] A. Fitzgibbon and A. Zisserman, “Multibody Structure and Motion: 3D Reconstruction of Independently Moving Objects,” Proc. Sixth European Conf. Computer Vision, pp. 891-906, 2000.
[9] C.W. Gear, “Multibody Grouping from Motion Images,” Int'l J.Computer Vision, vol. 29, no. 2, pp. 133-150, 1998.
[10] A. Gruber and Y. Weiss, “Multibody Factorization with Uncertainty and Missing Data Using the EM Algorithm,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 707-714, 2004.
[11] M. Han and T. Kanade, “Reconstruction of a Scene with Multiple Linearly Moving Objects,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 542-549, 2000.
[12] M. Han and T. Kanade, “Multiple Motion Scene Reconstruction from Uncalibrated Views,” Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 1, pp. 163-170, 2001.
[13] R. Hartley and R. Vidal, “The Multibody Trifocal Tensor: Motion Segmentation from 3 Perspective Views,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 769-775, 2004.
[14] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[15] K. Kanatani, “Motion Segmentation by Subspace Separation and Model Selection,” Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 586-591, 2001.
[16] K. Kanatani, “Evaluation and Selection of Models for Motion Segmentation,” Proc. Fifth Asian Conf. Computer Vision, pp. 7-12, 2002.
[17] K. Kanatani and C. Matsunaga, “Estimating the Number of Independent Motions for Multibody Motion Segmentation,” Proc. Seventh European Conf. Computer Vision, pp. 25-31, 2002.
[18] K. Kanatani and Y. Sugaya, “Multi-Stage Optimization for Multi-Body Motion Segmentation,” Proc. Australia-Japan Advanced Workshop Computer Vision, pp. 335-349, 2003.
[19] Y. Ma, K. Huang, R. Vidal, J. Košecká, and S. Sastry, “Rank Conditions on the Multiple View Matrix,” Int'l J. Computer Vision, vol. 59, no. 2, pp. 115-137, 2004.
[20] Y. Ma, S. Soatto, J. Kosecka, and S. Sastry, An Invitation to 3D Vision: From Images to Geometric Models. Springer, 2003.
[21] A. Shashua and A. Levin, “Multi-Frame Infinitesimal Motion Model for the Reconstruction of (Dynamic) Scenes with Multiple Linearly Moving Objects,” Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 592-599, 2001.
[22] A. Shashua and L. Wolf, “Homography Tensors: On Algebraic Entities that Represent Three Views of Static or Moving Planar Points,” Proc. Sixth European Conf. Computer Vision, vol. 1, pp. 507-521, 2000.
[23] F. Shi, J. Wang, J. Zhang, and Y. Liu, “Motion Segmentation of Multiple Translating Objects Using Line Correspondences,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 315-320, 2005.
[24] P. Sturm, “Structure and Motion for Dynamic Scenes—The Case of Points Moving in Planes,” Proc. Seventh European Conf. Computer Vision, pp. 867-882, 2002.
[25] P. Torr, R. Szeliski, and P. Anandan, “An Integrated Bayesian Approach to Layer Extraction from Image Sequences,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 297-303, Mar. 2001.
[26] P.H.S. Torr, “Geometric Motion Segmentation and Model Selection,” Philosophical Trans. Royal Soc. London, vol. 356, no. 1740, pp.1321-1340, 1998.
[27] R. Vidal and R. Hartley, “Motion Segmentation with Missing Data by PowerFactorization and Generalized PCA,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 310-316, 2004.
[28] R. Vidal and Y. Ma, “A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation,” Proc. Eighth European Conf. Computer Vision, pp. 1-15, 2004.
[29] R. Vidal, Y. Ma, and J. Piazzi, “A New GPCA Algorithm for Clustering Subspaces by Fitting, Differentiating and Dividing Polynomials,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 510-517, 2004.
[30] R. Vidal, Y. Ma, and S. Sastry, “Generalized Principal Component Analysis (GPCA),” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 621-628, 2003.
[31] R. Vidal, Y. Ma, and S. Sastry, “Generalized Principal Component Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1-15, Dec. 2005.
[32] R. Vidal, Y. Ma, S. Soatto, and S. Sastry, “Two-View Multibody Structure from Motion,” Int'l J. Computer Vision, vol. 68, no. 1, pp.7-25, 2006.
[33] L. Wolf and A. Shashua, “Affine 3-D Reconstruction from Two Projective Images of Independently Translating Planes,” Proc. Eighth IEEE Int'l Conf. Computer Vision, pp. 238-244, 2001.
[34] L. Wolf and A. Shashua, “Two-Body Segmentation from Two Perspective Views,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 263-270, 2001.
[35] Y. Wu, Z. Zhang, T.S. Huang, and J.Y. Lin, “Multibody Grouping via Orthogonal Subspace Decomposition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 252-257, 2001.
[36] L. Zelnik-Manor and M. Irani, “Degeneracies, Dependencies and Their Implications in Multi-Body and Multi-Sequence Factorization,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 287-293, 2003.
32 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool