This Article 
 Bibliographic References 
 Add to: 
Projective Structure from Uncalibrated Images: Structure From Motion and Recognition
August 1994 (vol. 16 no. 8)
pp. 778-790

Address the problem of reconstructing 3-D space in a projective framework from two or more views, and the problem of artificially generating novel views of the scene from two given views (reprojection). The author describes an invariance relation that provides a new description of structure, which the author calls projective depth, that is captured by a single equation relating image point correspondences across two or more views and the homographics of two arbitrary virtual planes. The framework is based on knowledge of correspondence of features across views, is linear and extremely simple, and the computations of structure readily extend to overdetermination using multiple views. Experimental results demonstrate a high degree of accuracy in both tasks: reconstruction and reprojection.

[1] G. Adiv, "Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field,"IEEE Trans. Patt. Anal. Mach. Intell., vol. 11, pp. 477-489, 1989.
[2] J. Aloimonos and C. M. Brown, "On the kinetic depth effect,"Biological Cybernetics, vol. 60, pp. 445-455, 1989.
[3] P. Anandan, "A unified perspective on computational techniques for the measurement of visual motion," inProc. Image Understanding Workshop. San Mateo, CA: Morgan Kaufmann, 1987, pp. 219-230.
[4] E.B. Barrett, P.M. Payton, N.N. Haag, and M. H. Brill, "General methods for determining projective invariants in imagery,"Comput. Vision Graphics Image Proc., Jan. 1991.
[5] J. R. Bergen and R. Hingorani, "'Hierarchical motion-based frame rate conversion," Tech. Rep., David Sarnoff Res. Center, 1990.
[6] T. J. Broida, S. Chandrashekhar, and R. Chellappa, "Recursive estimation of 3-D kinematics and structure from noisy monocular image sequences,"IEEE Trans. Aerosp. Electron. Syst., vol. AES-26, pp. 639-656, Aug. 1990.
[7] D. C. Brown, "Close-range camera calibration,"Photogrammetric Eng., vol. 37, pp. 855-866, 1971.
[8] S. Demey, A. Zisserman, and P. Beardsley, "Affine and projective structure from motion," inProc. British Mach. Vision Conf., Oct. 1992.
[9] R. Dutta and M. A. Synder, "Robustness of correspondence based structure from motion," inProc. Int. Conf. Comput. Vision, 1990, pp. 106-110.
[10] W. Faig, "Calibration of close-range photogrammetry systems: Mathematical formulation,"Photogrammetric Eng. Remote Sensing, vol. 41, pp. 1479-1486, 1975.
[11] O. D. Faugeras, "What can be seen in three dimensions with an uncalibrated stereo rig," inComputer Vision--ECCV'92, LNCS-Series, vol. 588. New York: Springer-Verlag, 1992, pp. 563-578.
[12] O. D. Faugeras, Q. T. Luong, and S. J. Maybank, "Camera self-calibration: Theory and experiments," inProc. European Conf. Comput. Vision, 1992, pp. 321-334.
[13] O. Faugeras and S. Maybank, "Motion from point matches: multiplicity of solutions,"Int. J. Comput. Vision, vol. 4, pp. 225-246, June 1990.
[14] W. E. L. Grimson, "Why stereo vision is not always about 3-D reconstruction," AI Memo 1435, Artificial Intell. Lab., Massachusetts Inst. of Technol., July 1993.
[15] R. Hartley, R. Gupta, and T. Chang, "Stereo from uncalibrated cameras," inProc. IEEE Conf. Comput. Vision and Pattern Recognit., 1992, pp. 761-764.
[16] R. I. Hartley, "Euclidean reconstruction from uncalibrated views," in2nd European Workshop Invariants, Azores Islands, Portugal, Oct. 1993.
[17] T. S. Huang and C. H. Lee, "Motion and structure from orthographic projections,"IEEE Trans. Patt. Anal. Mach. Intell., vol. 11, pp. 536-540, 1989.
[18] D. P. Huttenlocher and S. Ullman, "Recognizing solid objects by alignment with an image,"J. Comput. Vision, vol. 5, no. 2, pp. 195-212, 1990.
[19] J. J. Koenderink and A. J. Van Doorn, "Affine structure from motion,"J. Opt. Soc. Am., vol. 8, pp. 377-385, 1991.
[20] C. H. Lee, "Structure and motion from two perspective views via planar patch," presented at the 2nd IEEE Int. Conf. Computer Vision, Tampa, FL, Dec. 1988.
[21] R. K. Lenz and R. Y. Tsai, "Techniques for calibration of the scale factor and image center for high accuracy 3D machine vision metrology," inProc. IEEE Int. Conf. Robotics Automat.(Raleigh, NC), Mar. 1987, pp. 68-75.
[22] H. C. Longuet-Higgins, "A computer algorithm for reconstructing a scene from two projections,"Nature, vol. 293, pp. 133-135, 1981.
[23] B. D. Lucas and T. Kanade, "An iterative image registration technique with an application to stereo vision," inProc. IJCAI, 1981, pp. 674-679.
[24] Q. T. Luong, R. Deriche, O. D. Faugeras, and T. Papadopoulo, "On determining the fundamental matrix: Analysis of different methods and experimental results," Tech. Rep. INRIA, France, 1993.
[25] Q. T. Luong and T. Vieville, "Canonical representations for the geometries of multiple projective views," Tech. Rep. INRIA, France, 1993.
[26] R. Mohr, L. Quan, F. Veillon, and B. Boufama, "Relative 3-D reconstruction using multiple uncalibrated images," Tech. Rep. RT 84-IMAG, LIFIA--IRIMAG, France, June 1992.
[27] J. L. Mundy, R. P. Welty, M. H. Brill, P. M. Payton, and E. B. Barrett, "3-D model alignment without computing pose," inProc. Image Understanding Workshop. San Mateo, CA: Morgan Kaufmann, 1992, pp. 727-735.
[28] J. W. Roach and J. K. Aggarwal, "Computer tracking of objects moving in space,"IEEE Trans. Patt. Anal. Mach. Intell., vol. 1, pp. 127-135, 1979.
[29] L. Robert and O. D. Faugeras, "Relative 3-D positioning and 3-D convex hull computation from a weakly calibrated stereo pair," inProc. Int. Conf. Comput. Vision, 1993, pp. 540-544.
[30] A. Shashua, "Correspondence and affine shape from two orthographic views: Motion and Recognition," AI Memo 1327, Artificial Intell. Lab., Massachusetts Inst. of Technol., 1991.
[31] A. Shashua, "Geometry and photometry in 3-D visual recognition," Ph.D. dissertation, MIT Artificial Intell. Lab., AI-TR-1401, 1992.
[32] A. Shashua, "Projective structure from two uncalibrated images: Structure from motion and recognition," AI Memo 1363, Artificial Intell. Lab., Massachusetts Inst. of Technol., 1992.
[33] A. Shashua, "On geometric and algebraic aspects of 3-D affine and projective structures from perspective 2-D views," inProc. 2nd European Workshop Invariants, Azores Islands, Portugal, Oct. 1993 (also in MIT AI Memo 1405, July 1993).
[34] A. Shashua, "Projective depth: A geometric invariant for 3-D reconstruction from two perspective/orthographic views and for visual recognition," inProc. Int. Conf. Comput. Vision, 1993, pp. 583-590.
[35] A. Shashua, "Trilinearity in visual recognition by alignment," inProc. European Conf. Comput. Vision, Stockholm, Sweden, May 1994.
[36] A. Shashua and N. Navab, "Relative affine structure: Theory and application to 3-D reconstruction from perspective views," inProc. IEEE Conf. Comput. Vision Patt. Recognition, Seattle, WA, USA, 1994.
[37] A. Shashua and S. Toelg, "The quadric reference surface: Applications in registering views of complex 3-D objects," inProc. European Conf. Comput. Vision, Stockholm, Sweden, May 1994.
[38] C. Tomasi, "Shape and motion from image streams: A factorization method," Ph.D. dissertation, School of Comput. Sci., Carnegie Mellon Univ., Pittsburgh, PA, USA, 1991.
[39] C. Tomasi and T. Kanade, "Factoring image sequences into shape and motion," inIEEE Workshop on Visual Motion, 1991, pp. 21-29.
[40] R. Y. Tsai and T. S. Huang, "Uniqueness and estimation of 3-D motion parameters of rigid objects with curved surface,"IEEE Trans. Patt. Anal. Mach. Intell., vol. PAMI-6, pp. 13-26, 1984.
[41] S. Ullman,The Interpretation of Visual Motion. Cambridge, MA: MIT Press, 1979.
[42] S. Ullman, "Aligning pictorial descriptions: An approach to object recognition:Cognition, vol. 32, no. 3, pp. 193-254, 1989; A.I. Memo 931, Artificial Intell. Lab., Mass. Inst. Technol., 1986.
[43] S. Ullman and R. Basri, "Recognition by linear combination of models,"IEEE Trans. Patt. Anal. Mach. Intell., vol. 13, pp. 992-1006, 1991 (also in MIT AI Memo 1052, 1989).
[44] A. Verri and V. Torre, "Absolute depth estimate in stereopsis,"J. Opt. Soc. Am., vol. 3, pp. 297-299, 1986.
[45] D. Weinshall, "Model based invariants for 3-D vision,"Int. J. Comput. Vision, vol. 10, pp. 27-42, 1993.

Index Terms:
motion estimation; image reconstruction; projective structure; uncalibrated images; structure from motion; 3-D space; reprojection; projective depth; image point correspondences; reconstruction
A. Shashua, "Projective Structure from Uncalibrated Images: Structure From Motion and Recognition," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 8, pp. 778-790, Aug. 1994, doi:10.1109/34.308472
Usage of this product signifies your acceptance of the Terms of Use.