This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Two-View Multibody Structure-and-Motion with Outliers through Model Selection
June 2006 (vol. 28 no. 6)
pp. 983-995
Multibody structure-and-motion (MSaM) is the problem to establish the multiple-view geometry of several views of a 3D scene taken at different times, where the scene consists of multiple rigid objects moving relative to each other. We examine the case of two views. The setting is the following: Given are a set of corresponding image points in two images, which originate from an unknown number of moving scene objects, each giving rise to a motion model. Furthermore, the measurement noise is unknown, and there are a number of gross errors, which are outliers to all models. The task is to find an optimal set of motion models for the measurements. It is solved through Monte-Carlo sampling, careful statistical analysis of the sampled set of motion models, and simultaneous selection of multiple motion models to best explain the measurements. The framework is not restricted to any particular model selection mechanism because it is developed from a Bayesian viewpoint: Different model selection criteria are seen as different priors for the set of moving objects, which allow one to bias the selection procedure for different purposes.

[1] H. Akaike, “Information Theory and an Extension of the Maximum Likelihood Principle,” Proc. Second Int'l Symp. Information Theory, pp. 267-281, 1973.
[2] S. Avidan and A. Shashua, “Trajectory Triangulation: 3D Reconstruction of Moving Points from a Monocular Image Sequence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 4, pp. 348-357, Apr. 2000.
[3] G.L. Bretthorst, “An Introduction to Model Selection Using Probability Theory as Logic,” Maximum Entropy and Bayesian Methods, G.R. Heidbreder, ed., pp. 1-42, Kluwer Academic Publishers, 1996.
[4] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[5] O. Faugeras, Q.-T. Luong, and T. Papadopoulo, The Geometry of Multiple Images. MIT Press, 2001.
[6] O.D. Faugeras, “What Can Be Seen in 3D with an Uncalibrated Stereo Rig?” Proc. Second European Conf. Computer Vision, pp. 563-578, 1992.
[7] D.A. Forsyth and J. Ponce, Computer Vision— A Modern Approach. Prentice Hall, Inc., 2003.
[8] E.I. George and D.P. Foster, “Calibration and Empirical Bayes Variable Selection,” Biometrika, vol. 87, no. 4, pp. 731-747, 2000.
[9] F. Glover and M. Laguna, “Tabu Search,” Modern Heuristic Techniques for Combinatorial Problems, 1993.
[10] M. Han and T. Kanade, “Reconstruction of Scenes with Multiple Linearly Moving Objects,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 542-549, 2000.
[11] R. Hartley, “Estimation of Relative Camera Positions for Uncalibrated Cameras,” Proc. Second European Conf. Computer Vision, pp. 579-587, 1992.
[12] R. Hartley, “Projective Reconstruction and Invariants from Multiple Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, pp. 1036-1041, 1994.
[13] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[14] K. Huang, Y. Ma, and R. Vidal, “Minimum Effective Dimension for Mixtures of Subspaces: A Robust GPCA Algorithm and Its Applications,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 631-638, 2004.
[15] P.J. Huber, Robust Statistics. John Wiley and Sons, 1981.
[16] M. Irani and P. Anandan, “A Unified Approach to Moving Object Detection in 2D and 3D Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 6, pp. 577-589, June 1998.
[17] K. Kanatani, Statistical Optimation for Geometric Computation: Theorie and Practice. Elsevier, 1996.
[18] K. Kanatani, “Geometric Information Criterion for Model Selection,” Int'l J. Computer Vision, vol. 26, no. 3, pp. 171-189, 1998.
[19] K. Kanatani, “Uncertainty Modeling and Model Selection for Geometric Inference,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1307-1319, 2004.
[20] A. Leonardis, A. Gupta, and R. Bajcsy, “Segmentation of Range Images as the Search for Geometric Parametric Models,” Int'l J. Computer Vision, vol. 14, no. 1, pp. 253-277, 1995.
[21] C. Longuet-Higgins, “A Computer Algorithm for Reconstructing a Scene from Two Projections,” Nature, vol. 293, pp. 133-135, 1981.
[22] Y. Ma, J. Kosecka, S. Soatto, and S. Sastry, An Invitation to 3-D Vision. Springer Verlag, 2003.
[23] J. Matas, O. Chum, M. Urban, and T. Pajdla, “Robust Wide Baseline Stereo from Maximally Stable Extremal Regions,” Proc. 13th British Machine Vision Conf., vol. 1, pp. 384-393, 2002.
[24] C. Matsunaga and K. Kanatani, “Calibration of a Moving Camera Using a Planer Pattern: Optimal Computation, Reliability Evaluation and Stabilization by Model Selection,” Proc. Sixth European Conf. Computer Vision, vol. 2, 2000.
[25] J. Rissanen, “Modeling by Shortest Data Description,” Automatica, vol. 14, pp. 465-471, 1978.
[26] P.J. Rousseeuw and A.M. Leroy, Robust Regression and Outlier Detection. John Wiley and Sons, 1987.
[27] K. Schindler, “Spatially Consistent 3D Motion Segmentation,” Proc. Int'l IEEE Conf. Image Processing, 2005.
[28] G. Schwartz, “Estimating the Dimension of a Model,” Annals of Statistics, vol. 6, pp. 497-511, 1978.
[29] C.E. Shannon, “A Mathematical Theory of Communication,” Bell Systems Technical J., vol. 27, pp. 379-423, 1948.
[30] A. Shashua and A. Levin, “Multiframe Infinitesimal Motion Model for the Reconstruction of (Dynamic) Scenes with Multiple Linearly Moving Objects,” Proc. Eighth Int'l Conf. Computer Vision, pp. 592-599, 2001.
[31] M. Stricker and A. Leonardis, “ExSel++: A General Framework to Extract Parametric Models,” Proc. Computer Analysis of Images and Patterns, pp. 90-97, 1995.
[32] P. Sturm, “Structure and Motion of Dynamic Scenes— The Case of Points Moving in Planes,” Proc. Seventh European Conf. Computer Vision, pp. 867-882, 2002.
[33] C. Tomasi and T. Kanade, “Detection and Tracking of Point Features,” Technical Report CMU-CS-91-132, Carnegie Mellon Univ., 1991.
[34] W.-S. Tong, C.-K. Tang, and G. Medioni, “Simultaneous Two-View Epipolar Geometry Estimation and Motion Segmentation by 4D Tensor Voting,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1167-1184, Sept. 2004.
[35] P.H.S. Torr, “Geometric Motion Segmentation and Model Selection,” Philosophical Trans. Royal Soc. London A, vol. 356, no. 1740, pp. 1321-1340, 1998.
[36] P.H.S. Torr, “Model Selection for Structure and Motion Recovery from Multiple Images,” Technical Report MSR-TR-99-16, Microsoft Research, 1999.
[37] P.H.S. Torr, “Bayesian Model Estimation and Selection for Epipolar Geometry and Generic Manifold Fitting,” Int'l J. Computer Vision, vol. 50, no. 1, pp. 35-61, 2002.
[38] R. Vidal and Y. Ma, “A Unified Algebraic Approach to 2-D and 3-D Motion Segmentation,” Proc. Eighth European Conf. Computer Vision, pp. 1-15, 2004.
[39] R. Vidal and S. Sastry, “Optimal Segmentation of Dynamic Scenes from Two Perspective Views,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[40] R. Vidal, S. Soatto, Y. Ma, and S. Sastry, “Segmentation of Dynamic Scenes from the Multibody Fundamental Matrix,” Proc. ECCV Workshop Visual Modeling of Dynamic Scenes, 2002.
[41] C.S. Wallace and D.M. Boulton, “An Information Measure for Classification,” Computer J., vol. 11, no. 2, pp. 185-194, 1968.
[42] M.P. Wand and M. Jones, Kernel Smoothing. Chapman and Hall, 1995.
[43] H. Wang and D. Suter, “MDPE: A Very Robust Estimator for Model Fitting and Range Image Segmentation,” Int'l J. Computer Vision, vol. 59, no. 2, pp. 139-166, 2004.
[44] H. Wang and D. Suter, “Robust Fitting by Adaptive-Scale Residual Consensus,” Proc. Eighth European Conf. Computer Vision, pp. 107-118, 2004.
[45] L. Wolf and A. Shashua, “Two-Body Segmentation from Two Perspective Views,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 263-270, 2001.
[46] Z. Zhang, “Parameter Estimation Techniques: A Tutorial with Application to Conic Fitting,” Image and Vision Computing, vol. 15, no. 1, pp. 59-76, 1997.

Index Terms:
Dynamic scenes, structure-and-motion, model selection, 3D motion segmentation.
Citation:
Konrad Schindler, David Suter, "Two-View Multibody Structure-and-Motion with Outliers through Model Selection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 6, pp. 983-995, June 2006, doi:10.1109/TPAMI.2006.130
Usage of this product signifies your acceptance of the Terms of Use.