This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Interactive Optimization of 3D Shape and 2D Correspondence Using Multiple Geometric Constraints via POCS
April 2002 (vol. 24 no. 4)
pp. 562-569

The traditional approach of handling motion tracking and structure from motion (SFM) independently in successive steps exhibits inherent limitations in terms of achievable precision and incorporation of prior geometric constraints about the scene. This paper proposes a projections onto convex sets (POCS) framework for iterative refinement of the measurement matrix in the well-known factorization method to incorporate multiple geometric constraints about the scene, thereby improving the accuracy of both 2D feature point tracking and 3D structure estimates. Regularities in the scene, such as points on line and plane and parallel lines and planes, that can be interactively identified and marked at each POCS iteration, enforce rank and parallelism constraints on appropriately defined local measurement matrices, one for each constraint. The POCS framework allows for the integration of the information in each of these local measurement matrices into a single measurement matrix that is closest to the initial observed measurement matrix in Frobenius norm, which is then factored in the usual manner. Experimental results demonstrate that the proposed interactive POCS framework consistently improves both 2D correspondences and 3D shape/motion estimates and similar results can not be achieved by enforcing these constraints as either post or preprocessing.

[1] C. Tomasi and T. Kanade, "Shape and Motion From Image Streams Under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[2] D. Weinshall and C. Tomasi, "Linear and Incremental Acquisition of Invariant Shape Models from Image Sequences," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 512-517, May 1995.
[3] C. Poelman and T. Kanade, A Paraperspective Factorization Method for Shape and Motion Recovery IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 3, pp. 206-219, Mar. 1997.
[4] L. Quan, Self-Calibration of an Affine Camera from Multiple Views Int'l J. Computer Vision, vol. 19, no. 1, pp. 93-105, July 1996.
[5] P. Sturm and B. Triggs, “A Factorization Based Algorithm for Multi-Image Projective Structure and Motion,” Proc. European Conf. Computer Vision '96, vol. 2, pp. 709-720, 1996.
[6] D. Morris and T. Kanade, “A Unified Factorization Algorithm for Points, Line Segments and Planes with Uncertainty Models,” Proc. Int'l Conf. Computer Vision '98, pp. 696-702, Jan. 1998.
[7] M. Irani and P. Anandan, “Factorization with Uncertainty,” Proc. European Conf. Computer Vision '2000, pp. 539-553, 2000.
[8] Z. Sun, V. Ramesh, and A. Murat Tekalp, “Error Characterization of the Factorization Method,” Computer Vision and Image Understanding, vol. 82, pp. 110–137, May 2001.
[9] T. Broida and R. Chellappa, "Estimating the Kinematics and Structure of a Rigid Object From a Sequence of Monocular Frames," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 6, pp. 497-513, June 1991.
[10] P.A. Beardsley, A. Zisserman, and D.W. Murray, “Sequential Updating of Projective and Affine Structure from Motion,” Int'l J. Computer Vision, vol. 23, pp. 235–259, June 1997.
[11] A. Azarbayejani and A. Pentland, "Recursive Estimation of Motion, Structure and Focal Length," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 562-575, June 1995.
[12] J. Weng, N. Ahuja, and T. Huang, "Optimal Motion and Structure Estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 864-884, 1993.
[13] R. Szeliski and S. Kang, “Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares,” J. Visual Comm. and Image Representation, vol. 5, pp. 10–28, Mar. 1994.
[14] B Triggs, P.F. McLauchlan, R.I. Hartley, and A.W. Fitzgibbon, “Special Sessions—Bundle Adjustment: A Modern Synthesis,” Lecture Notes in Computer Science, vol. 1883, pp. 298-372, 2000.
[15] T. Huang and A. Netravali,“Motion and structure from feature correspondences: A review,” Proc. IEEE, vol. 82, pp. 252-268, Feb. 1994.
[16] P.E. Debevec, C.J. Taylor, and J. Malik, “Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach,” Proc. SIGGRAPH '96, pp. 11-20, Aug. 1996.
[17] D. Liebowitz, A. Criminisi, and A. Zisserman, “Creating Architectural Models from Images,” Proc. EuroGraphics Conf., vol. 18, pp. 39–50, Sept. 1999.
[18] R. Szeliski and P. Torr, “Geometrically Constrained Structure from Motion: Points on Planes,” Proc. European Workshop 3D Structure from Multiple Images of Large-Scale Environments (SMILE), pp. 171-186, June 1998.
[19] H. Stark and Y. Yang, Vector Space Projections: A Numerical Approach to Signal and Image Processing, Neural Nets, and Optics. John Wiley&Sons, 1998.
[20] D. Youla and H. Webb, “Image Restoration by the Method of Convex Projections: Part 1—Theory,” IEEE Trans. Medical Imaging, vol. 1, pp. 81–94, Oct. 1982.
[21] M.I. Sezan and H. Stark, “Image Restoration by the Method of Convex Projections: Part 2—Application and Numerical Results,” IEEE Trans. Medical Imaging, vol. 1, pp. 95–101, Oct. 1982.

Index Terms:
geometric constrained shape recovery, structure from motion, interactive optimization, the factorization approach, projections onto convex sets (POCS)
Citation:
Z. Sun, A.M. Tekalp, N. Navab, V. Ramesh, "Interactive Optimization of 3D Shape and 2D Correspondence Using Multiple Geometric Constraints via POCS," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 562-569, April 2002, doi:10.1109/34.993563
Usage of this product signifies your acceptance of the Terms of Use.