This Article 
 Bibliographic References 
 Add to: 
Reducing "Structure From Motion": A General Framework for Dynamic Vision Part 1: Modeling
September 1998 (vol. 20 no. 9)
pp. 933-942

Abstract—The literature on recursive estimation of structure and motion from monocular image sequences comprises a large number of apparently unrelated models and estimation techniques. We propose a framework that allows us to derive and compare all models by following the idea of dynamical system reduction. The "natural" dynamic model, derived from the rigidity constraint and the projection model, is first reduced by explicitly decoupling structure (depth) from motion. Then, implicit decoupling techniques are explored, which consist of imposing that some function of the unknown parameters is held constant. By appropriately choosing such a function, not only can we account for models seen so far in the literature, but we can also derive novel ones.

[1] G. Adiv, "Determining Three-Dimensional Motion and Structure From Optical Flow Generated by Several Moving Objects," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 4, pp. 384-401, July 1985.
[2] A. Azarbayejani and A. Pentland, "Recursive Estimation of Motion, Structure and Focal Length," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 562-575, June 1995.
[3] J. Barron, D. Fleet, and S. Beauchemin, "Performance of Optical Flow Techniques," Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-78, 1994.
[4] J. Bergen, R. Kumar, P. Anandan, and M. Irani, "Representation of Scenes From Collections of Images," Internal Report, Sarnoff Research Center, 1995.
[5] W. Boothby, Introduction to Differentiable Manifolds and Riemannian Geometry. Academic Press, 1986.
[6] T. Broida, S. Chandrashekhar, and R. Chellappa, "Recursive 3D Motion Estimation From a Monocular Image Sequence," IEEE Trans. Aerospace and Electronic Systems, vol. 26, no. 4, pp. 639-656, 1990
[7] T. Broida and R. Chellappa, "Estimating the Kinematics and Structure of a Rigid Object From a Sequence of Monocular Frames," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 6, pp. 497-513, June 1991.
[8] R.L. Bryant, S.S. Chern, R.B. Gardner, H.L. Goldshmidt, and P.A. Griffith, Exterior Differential Systems. Mathematical Research Institute, Springer Verlag, 1991.
[9] N Cui, J. Weng, and P. Cohen, "Recursive-Batch Estimation of Motion and Structure From Monocular Image Sequences," IEEE Trans. Aerospace and Electronic Systems, 1990.
[10] E. Dickmanns and W. Graefe, "Dynamic Monocular Machine Vision," Machine Vision and Applications, 1988.
[11] O.D. Faugeras, Three Dimensional Vision, A Geometric Viewpoint. MIT Press, 1993.
[12] C. Fermüller and Y. Aloimonos, "Tracking Facilitates 3D Motion Estimation," Biological Cybernetics, vol. 67, pp. 259-268, 1992.
[13] E. Gennery, "Tracking Known Three-Dimensional Objects," Prof. AAAI Second Nat'l Conf. Artificial Intelligence, 1982.
[14] V. Guillemin and A. Pollack, Differential Topology. Prentice-Hall, 1974.
[15] D. Heeger and A. Jepson, "Subspace Methods for Recovering Rigid Motion I: Algorithm and Implementationm," Int'l J. Computer Vision, vol. 7, no. 2, 1992.
[16] J. Heel, "Direct Estimation of Structure and Motion From Multiple Frames," AI Memo 1190, MIT AI Lab, Mar. 1990.
[17] B.K.P. Horn, "Relative Orientation," Int'l J. Computer Vision, vol. 4, pp. 59-78, 1990.
[18] A. Isidori, Nonlinear Control Systems. Springer Verlag, 1989.
[19] D. Jacobs, "Linear Fitting with Missing Data," Proc. IEEE CVPR, 1997.
[20] A.H. Jazwinski, Stochastic Processes and Filtering Theory. Academic Press, 1970.
[21] A. Jepson and D. Heeger, "Linear Subspace Methods for Recovering Rigid Motion," Spatial Vision in Humans and Robots, Cambridge University Press, 1992.
[22] T. Kailath, Linear Systems. Prentice Hall, 1980.
[23] H.C. Longuet-Higgins, "A Computer Algorithm for Reconstructing a Scene From Two Projections," Nature, vol. 293, pp. 133-135, 1981.
[24] Q. Luong and O. Faugeras In preparation, 1997.
[25] L. Matthies, R. Szeliski, and T. Kanade, "Kalman Filter-Based Algorithms for Estimating Depth From Image Sequences," Int'l J. Computer Vision, 1989.
[26] S.J. Maybank, Theory of Reconstruction From Image Motion. Springer Verlag, 1992.
[27] P. McLauchlan, I. Reid, and D. Murray, "Recursive Affine Structure and Motion From Image Sequences," Proc. Third ECCV, 1994.
[28] R.M. Murray, Z. Li, and S.S. Sastry, A Mathematical Introduction to Robotic Manipulation. CRC Press, 1994.
[29] J. Oliensis, "Rigorous Bounds for 2-Frame Structure From Motion. Proc. IEEE CVPR, 1996.
[30] J. Oliensis and J. Inigo-Thomas, "Recursive Multiframe Structure From Motion Incorporating Motion Error," Proc. DARPA Image Understanding Workshop, 1992.
[31] C. Poelman and T. Kanade, "A Paraperspective Factorization Method for Shape and Motion Recovery," Proc. Third ECCV, LNCS, vol. 810, Springer Verlag, 1994.
[32] P. Anandan, R. Kumar, and K. Hanna, "Shape Recovery From Multiple Views: A Parallax Based Approach," Proc. Image Understanding Workshop, 1994.
[33] D. Raviv and M. Herman, "A Unified Approach to Camera Fixation and Vision-Based Road Following," IEEE Trans. Systems, Man, and Cybernetics, vol. 24, no. 8, 1994.
[34] H.S. Sawhney, "Simplifying Motion and Structure Analysis Using Planar Parallax and Image Warping," Proc. Int'l Conf. Pattern Recognition, 1994.
[35] J.G. Semple and G.J. Kneebone, Algebraic Projective Geometry.Oxford, 1952.
[36] S. Soatto, "3D Structure From Visual Motion: Modeling, Representation and Observability," Automatica, vol. 33, no. 9, 1997.
[37] S. Soatto, R. Frezza, and P. Perona, "Motion Estimation Via Dynamic Vision," IEEE Trans. Automatic Control, vol. 41, no. 3, 1996.
[38] S. Soatto and P. Perona, "Recursive 3D Visual Motion Estimation Using Subspace Constraints," Int'l J. Computer Vision, vol. 22, no. 3, 1997.
[39] S. Soatto and P. Perona, "Reducing "Structure From Motion": General Framework for Dynamic Vision Part 2: Implementation and Experimental Assessment," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp. 943-961, Sept. 1998.
[40] M. Spetsakis and J. Aloimonos, "A Multiframe Approach to Visual Motion Perception," Int'l J. Computer Vision, vol. 6, no. 3, 1991.
[41] R. Szeliski, "Recovering 3D Shape and Motion From Image Streams Using Nonlinear Least Squares," J. Visual Communication and Image Representation, 1994.
[42] M.A. Taalebinezhaad, "Direct Recovery of Motion and Shape in the General Case by Fixation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 8, pp. 847-853, Aug. 1992.
[43] I. Thomas and E. Simoncelli, "Linear Structure From Motion," Technical Report IRCS 94-26, Univ. of Pennsylvania, 1994.
[44] T.Y. Tian, C. Tomasi, and D.J. Heeger, “Comparison of Approaches to Egomotion Computation,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 315–320, 1996.
[45] C. Tomasi and T. Kanade, "Shape and Motion From Image Streams Under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[46] J. Weng, N. Ahuja, and T. Huang, "Motion and Structure From Point Correspondences With Error Estimation: Planar Surfaces," IEEE Trans. Signal Processing, vol. 39, no. 12, pp. 2,691-2,716, 1991.
[47] J. Weng, N. Ahuja, and T. Huang, "Optimal Motion and Structure Estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 15, no. 9, pp. 864-884, 1993.

Index Terms:
Visual motion estimation, epipolar geometry, motion decoupling, compensation, fixation, parallax, output stabilization, model reduction.
Stefano Soatto, Pietro Perona, "Reducing "Structure From Motion": A General Framework for Dynamic Vision Part 1: Modeling," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp. 933-942, Sept. 1998, doi:10.1109/34.713360
Usage of this product signifies your acceptance of the Terms of Use.