This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Recursive Estimation of 3D Motion and Surface Structure from Local Affine Flow Parameters
April 2005 (vol. 27 no. 4)
pp. 562-574
A recursive structure from motion algorithm based on optical flow measurements taken from an image sequence is described. It provides estimates of surface normals in addition to 3D motion and depth. The measurements are affine motion parameters which approximate the local flow fields associated with near-planar surface patches in the scene. These are integrated over time to give estimates of the 3D parameters using an extended Kalman filter. This also estimates the camera focal length and, so, the 3D estimates are metric. The use of parametric measurements means that the algorithm is computationally less demanding than previous optical flow approaches and the recursive filter builds in a degree of noise robustness. Results of experiments on synthetic and real image sequences demonstrate that the algorithm performs well.

[1] J. Aggarwal and N. Nandhakumar, “On the Computation of Motion from Sequences of Images— A Review,” Proc. IEEE, vol. 76, no. 8, pp. 917-935, 1988.
[2] T. Huang and A. Netravali, “Motion and Structure from Feature Correspondences: A Review,” Proc IEEE, vol. 82, no. 2, pp. 252-268, 1994.
[3] S. Soatto and P. Perona, “Reducing `Structure from Motion': A General Framework for Dynamic Vision Part 1: Modelling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp. 933-942, Sept. 1998.
[4] A. Jebara, T. Azarbayejani, and A. Pentland, “3D Structure from 2D Motion,” IEEE Signal Processing Magazine, vol. 16, no. 3, pp. 66-84, 1999.
[5] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[6] A. Chiuso, P. Favaro, H. Jin, and S. Soatto, “Structure from Motion Causally Integrated over Time,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 523-535, Apr. 2002.
[7] D.W. Murray and L.S. Shapiro, “Dynamic Updating of Planar Structure and Motion: The Case of Constant Motion,” Computer Vision and Image Understanding, vol. 63, no. 1, pp. 169-181, 1996.
[8] S. Soatto and P. Perona, “Recursive 3D Visual Motion Estimation Using Subspace Contraints,” Int'l J. Computer Vision, vol. 22, no. 3, pp. 235-259, 1997.
[9] Y. Xiong and S. Schafer, “Dense Structure from a Dense Optical Flow Sequence,” Computer Vision and Image Understanding, vol. 69, no. 2, pp. 222-245, 1998.
[10] G. Adiv, “Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 4, pp. 384-401, July 1985.
[11] J.Y.A. Wang and E.H. Adelson, “Representing Moving Images with Layers,” IEEE Trans. Image Processing, vol. 3, no. 5, pp. 625-638, 1994.
[12] F.G. Meyer and P. Bouthemy, “Region-Based Tracking Using Affine Motion Models in Long Range Sequences,” CVGIP: Image Understanding, vol. 60, no. 2, pp. 119-140, 1994.
[13] M.J. Black and P. Anandan, “The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields,” Computer Vision and Image Understanding, no. 1, pp. 75-104, 1996.
[14] M.J. Black and A. Jepson, “Estimating Optical Flow in Segmented Images Using Variable-Order Parametric Models with Local Deformations,” IEEE Trans. Pattern Analysis and Machine Intelligence, no. 10, pp. 972-985, Oct. 1996.
[15] R. Wilson, P. Meulemans, A. Calway, and S. Kruger, “Image Sequence Analysis and Segmentation Using G-Blobs,” Proc IEEE Int'l Conf. Image Processing, 1998.
[16] H.C. Longuet-Higgins and K. Prazdny, “The Interpretation of a Moving Retinal Image,” Proc. Royal Soc. London, vol. B-208, pp. 385-397, 1980.
[17] M. Subbarao and A.M. Waxman, “Closed Form Solutions to Image Flow Equations for Planar Surfaces in Motion,” Computer Vision, Graphics and Image Processing, vol. 36, pp. 208-228, 1986.
[18] S. Negahdaripour and S. Lee, “Motion Recovery from Image Sequences Using Only 1st Order Optical-Flow Information,” Int'l J. Computer Vision, vol. 9, no. 3, pp. 163-184, 1992.
[19] H.C. Longuet-Higgins and K. Prazdny, “The Visual Ambiguity of a Moving Plane,” Proc. Royal Soc. London, vol. B-223, pp. 165-175, 1984.
[20] F. Dellaert, C. Thorpe, and S. Thrun, “Super-Resolved Texture Tracking of Planar Surface Patches,” Proc. IEEE/RSJ Int'l Conf. Intelligent Robotic Systems, 1998.
[21] A. Azarbayejani and A.P. Pentland, “Recursive Estimation of Motion, Structure and Focal Length,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 562-575, June 1995.
[22] R. Szeliski and S. Kang, “Recovering 3D Shape and Motion from Image Streams Using Nonlinear Least Squares,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 752-753, 1993.
[23] A. Calway, “Estimating the Structure of Textured Surfaces Using Local Affine Flow,” Proc. British Machine Vision Conf., pp. 92-101, 2000.
[24] J.V. Candy, Signal Processing: The Model Based Approach. Singapore: McGraw-Hill, 1986.
[25] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-77, 1994.
[26] D. Fleet and A. Jepson, “Computation of Component Image Velocity from Local Phase Information,” Int'l J. Computer Vision, vol. 5, no. 1, pp. 77-104, 1990.
[27] J. Bergen, P. Burt, R. Hingorani, and S. Peleg, “A Three-Frame Algorithm for Estimating Two-Component Image Motion,” IEEE Trans. Pattern Analysis and Machine Intelligence, no. 9, pp. 886-895, Sept. 1992.
[28] A. Calway, “Image Representation Based on the Affine Symmetry Group,” Proc. IEEE Int'l Conf. Image Processing, pp. 189-192, 1996.
[29] T. Tuytelaars and L. Van Gool, “Matching Widely Separated Views Based on Affine Invariant Regions,” Int'l J. Computer Vision, vol. 59, no. 1, pp. 61-85, 2004.

Index Terms:
Structure from motion, surface normals, affine motion models, Kalman filtering.
Citation:
Andrew Calway, "Recursive Estimation of 3D Motion and Surface Structure from Local Affine Flow Parameters," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 562-574, April 2005, doi:10.1109/TPAMI.2005.83
Usage of this product signifies your acceptance of the Terms of Use.