This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Incremental Model-Based Estimation Using Geometric Constraints
May 2005 (vol. 27 no. 5)
pp. 727-738
We present a model-based framework for incremental, adaptive object shape estimation and tracking in monocular image sequences. Parametric structure and motion estimation methods usually assume a fixed class of shape representation (splines, deformable superquadrics, etc.) that is initialized prior to tracking. Since the model shape coverage is fixed a priori, the incremental recovery of structure is decoupled from tracking, thereby limiting both processes in their scope and robustness. In this work, we describe a model-based framework that supports the automatic detection and integration of low-level geometric primitives (lines) incrementally. Such primitives are not explicitly captured in the initial model, but are moving consistently with its image motion. The consistency tests used to identify new structure are based on trinocular constraints between geometric primitives. The method allows not only an increase in the model scope, but also improves tracking accuracy by including the newly recovered features in its state estimation. The formulation is a step toward automatic model building, since it allows both weaker assumptions on the availability of a prior shape representation and on the number of features that would otherwise be necessary for entirely bottom-up reconstruction. We demonstrate the proposed approach on two separate image-based tracking domains, each involving complex 3D object structure and motion.

[1] M. Armstrong and A. Zisserman, “Robust Object Tracking,” Proc. Asian Conf. Computer Vision, 1995.
[2] A. Azarbayejani and A. Pentland, “Recursive Estimation of Motion, Structure and Focal Length,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 562-575, June 1995.
[3] C. Bregler, A. Hertzmann, and H. Biermann, “Recovering Non-Rigid 3D Shape from Image Streams,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2000.
[4] T. Cham and J. Rehg, “A Multiple Hypothesis Approach to Figure Tracking,” IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 239-245, 1999.
[5] M. Chan, D. Metaxas, and S. Dickinson, “Physics-Based Tracking of 3-D Objects in 2-D Image Sequences,” Proc. 12th Int'l Conf. Pattern Recognition, pp. 326-330, Oct. 1994.
[6] J. Clarke, S. Carlsson, and A Zisserman, “Detecting and Tracking Linear Features Efficiently,” Proc. Seventh British Machine Vision Conf., 1996.
[7] J. Crowley, P. Stelmaszyk, T. Skordas, and P. Pugget, “Measurements and Integration of 3-D Structures by Tracking Edge Lines,” Int'l J. Computer Vision, 1992.
[8] D. DeCarlo and D. Metaxas, “Blended Deformable Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 4, pp. 443-448, Apr. 1996.
[9] D. DeCarlo and D. Metaxas, “Combining Information in Deformable Models Using Hard Constraints,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1999.
[10] S. Dickinson and D. Metaxas, “Integrating Qualitative and Quantitative Shape Recovery,” Int'l J. Computer Vision, 1994.
[11] S. Dickinson and D. Metaxas, “Using Aspect Graphs to Control the Recovery and Tracking of Deformable Models,” Int'l J. Pattern Recognition and Artificial Intelligence, vol. 11, no. 1, pp. 115-142, Jan. 1997.
[12] S. Dickinson, A. Pentland, and A. Rosenfeld, “Shape Recovery Using Distributed Aspect Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 174-198, Feb. 1992.
[13] T. Drummond and R. Cipolla, “Real-Time Tracking of Multiple Articulated Structures in Multiple Views,” Proc. European Conf. Computer Vision, 2000.
[14] D. Dube and A. Mitiche, “The Incremental Rigidity Scheme for Structure from Motion: The Line-Based Formulation,” Proc. European Conf. Computer Vision, pp. 292-296, 1990.
[15] R. Fletcher, Practical Methods of Optimization. John Wiley, 1987.
[16] P. Fua and G. Leclerc, “Taking Advantage of Image-Based and Geometry-Based Constraints to Recover 3-D Surfaces,” Computer Vision and Image Understanding, 1996.
[17] R. Hartley, “Lines and Points in Three Views and the Trifocal Tensor,” Int'l J. Computer Vision, 1997.
[18] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[19] T. Heap and D. Hogg, “Wormholes in Shape Space: Tracking Through Discontinuities Changes in Shape,” Proc. IEEE Int'l Conf. Computer Vision, pp. 334-349, 1998.
[20] K.B. Horn, “Relative Orientation,” Int'l J. Computer Vision, 1990.
[21] T.S. Huang and A.N. Netravali, “Motion and Structure from Feature Correspondences: A Review,” Proc. IEEE, 1994.
[22] K. Kanantani, Statistical Optimization for Geometric Computation: Theory and Practice. Elsevier, 1996.
[23] Y. Keselman and S. Dickinson, “Generic Model Abstraction from Examples,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[24] D. Lowe, “Three-Dimensional Object Recognition from Single Two-Dimensional Images,” Artificial Intelligence, vol. 31, no. 3, 1987.
[25] D. Lowe, “Fitting Parameterized Three-Dimensional Models to Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 5, pp. 441-450, May 1991.
[26] D.P. McReynolds and D. Lowe, “Rigidity Checking of 3D Point Correspondences under Perspective Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, pp. 1174-1185, 1997.
[27] A. Mitiche and J. Aggarwal, “Line-Based Computation of Structure and Motion Using Angular Invariance,” Proc. IEEE Workshop Motion, 1986.
[28] A. Pentland and B. Horowitz, “Recovery of Non-Rigid Motion and Structure,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 730-742, July 1991.
[29] A. Pentland and S. Sclaroff, “Closed Form Solutions for Physically-Based Shape Modeling and Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 715-729, July 1991.
[30] D. Samaras and D. Metaxas, “Incorporating Illumination Constraints in Deformable Models,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1998.
[31] S. Seitz, “Implicit Scene Reconstruction from Probability Density Functions,” Proc. DARPA Image Understanding Workshop, 1998.
[32] A. Shashua, “Algebraic Functions for Recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 779-789, Aug. 1995.
[33] C. Sminchisescu, “Estimation Algorithms for Ambiguous Visual Models–Three-Dimensional Human Modeling and Motion Reconstruction in Monocular Video Sequences,” PhD thesis, Ins. Nat'l Politechnique de Grenoble (INRIA), July 2002.
[34] C. Sminchisescu and B. Triggs, “Estimating Articulated Human Motion with Covariance Scaled Sampling,” Int'l J. Robotics Research, vol. 22, no. 6, pp. 371-393, 2003.
[35] C. Sminchisescu and B. Triggs, “Mapping Minima and Transitions in Visual Models,” Intl J. Computer Vision, vol. 61, no. 1, 2005.
[36] F. Solina and R. Bajcsy, “Recovery of Parametric Models from Range Images: The Case of Superquadrics with Local and Global Deformations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, pp. 131-147, 1990.
[37] M. Spestakis and J. Aloimonos, “A Multi-Frame Approach to Visual Motion Perception,” Intl J. Computer Vision, 1991.
[38] R. Szeliski and S.B. Kang, “Recovery 3-D Shape and Motion from Image Streams Using Non-Linear Least-Squares,” Technical Report DEC TR, 1994.
[39] C. Taylor and D. Kriegman, “Structure and Motion from Line Segments in Multiple Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 11, pp. 1021-1032, Nov. 1996.
[40] D. Terzopoulos and D. Metaxas, “Dynamic 3D Models with Local and Global Deformations: Deformable Superquadrics,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 703-714, July 1991.
[41] D. Terzopoulos, A. Witkin, and M. Kass, “Constraints on Deformable Models: Recovering 3-D Shape and Non-Rigid Motion,” Artificial Intelligence, vol. 36, no. 1, 1988.
[42] C. Tomasi and T. Kanade, “Shape and Motion from Image Streams under Orthography: A Factorization Method,” Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[43] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, “Bundle Adjustment— A Modern Synthesis,” Vision Algorithms: Theory and Practice, 2000.
[44] S. Ullman, “Maximizing Rigidity: The Incremental Recovery of 3-D Structure from Rigid and Nonrigid Motion,” Perception, vol. 13, pp. 255-274, 1984.
[45] T. Vieville and O. Faugeras, “Feed-Forward Recovery of Motion and Structure from a Sequence of 2-D Line Matches,” Proc. IEEE Int'l Conf. Computer Vision, 1990.
[46] B. Yen and T. Huang, “Determining 3-D Motion and Structure of a Rigid Body Using Straight Line Correspondences,” ASI NATO Series, 1983.

Index Terms:
Shape recovery, object tracking, parametric models, geometric constraints, bundle adjustment, optimization.
Citation:
Cristian Sminchisescu, Dimitris Metaxas, Sven Dickinson, "Incremental Model-Based Estimation Using Geometric Constraints," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 727-738, May 2005, doi:10.1109/TPAMI.2005.104
Usage of this product signifies your acceptance of the Terms of Use.