This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Multi-Frame Estimation of Planar Motion
October 2000 (vol. 22 no. 10)
pp. 1105-1116

Abstract—Traditional plane alignment techniques are typically performed between pairs of frames. In this paper, we present a method for extending existing two-frame planar motion estimation techniques into a simultaneous multi-frame estimation, by exploiting multi-frame subspace constraints of planar surfaces. The paper has three main contributions: 1) we show that when the camera calibration does not change, the collection of all parametric image motions of a planar surface in the scene across multiple frames is embedded in a low dimensional linear subspace; 2) we show that the relative image motion of multiple planar surfaces across multiple frames is embedded in a yet lower dimensional linear subspace, even with varying camera calibration; and 3) we show how these multi-frame constraints can be incorporated into simultaneous multi-frame estimation of planar motion, without explicitly recovering any 3D information, or camera calibration. The resulting multi-frame estimation process is more constrained than the individual two-frame estimations, leading to more accurate alignment, even when applied to small image regions.

[1] E.H. Adelson, C.H. Anderson, J.R. Bergen, P.J. Burt, and J.M. Ogden, “Pyramid Methods in Image Processing,” RCA Eng., vol. 29, no. 6, pp. 33-41, 1985.
[2] G. Adiv, “Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 4, pp. 384-401, 1985.
[3] S. Ayer and H. Sawhney, "Layered Representation of Motion Video Using Robust Maximum-Likelihood Estimation of Mixture Models and mdl Encoding," Int'l Conf. Computer Vision, pp. 777-784,Cambridge, Mass., June 1995.
[4] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, “Hiercharchical Model-Based Motion Estimation,” Proc. European Conf. Computer Vision, pp. 237-252, 1992.
[5] M.J. Black and P. Anandan, “Robust Dynamic Motion Estimation Over Time,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 296–302, June 1991.
[6] O.D. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint.Cambridge, Mass.: MIT Press, 1993.
[7] D.J. Fleet and K. Langley, “Recursive Filters for Optical Flow,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, pp. 61–67, 1995.
[8] G. Golub and C. Van Loan, Matrix Computations. Baltimore and London: The Johns Hopkins Univ. Press, 1989.
[9] R.I. Hartley, In Defense of the 8-Point Algorithm IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 6, pp. 580-593, June 1997.
[10] M. Irani, B. Rousso, and P. Peleg, “Recovery of Ego-Motion Using Region Alignment,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 3, pp. 268-272, Mar. 1997.
[11] M. Irani, B. Rousso, and S. Peleg, “Detecting and Tracking Multiple Moving Objects Using Temporal Integration,” Proc. European Conf. Computer Vision, pp. 282-287, May 1992.
[12] M. Irani, B. Rousso, and S. Peleg, “Computing Occluding and Transparent Motions,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 5-16, Jan. 1994.
[13] S.X. Ju, M.J. Black, and A.D. Jepson, Skin and Bones: Multi-Layer, Locally Affine, Optical Flow and Regularization with Transparency Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 307-314, 1996.
[14] R. Kumar, P. Anandan, and K. Hanna, “Direct Recovery of Shape from Multiple Views: A Parallax Based Approach,” Proc. Int'l Conf. Pattern Recognition, pp. 685-688, Oct. 1994.
[15] H.C. Longuet-Higgins, “Visual Ambiguity of a Moving Plane,” Proc. The Royal Soc. of London B, vol. 223, pp. 165-175, 1984.
[16] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C, second ed. Cambridge Univ. Press, 1992.
[17] H. Sawhney and R. Kumar, “True Multi-Image Alignment and Its Application to Mosaicing and Lens Distortion Correction,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 450-456, 1997.
[18] A. Shashua and S. Avidan, "The Rank4 Constraint in Multiple View Geometry," Proc. European Conf. Computer Vision,Cambridge, UK, Apr. 1996.
[19] R. Szeliski and P. Torr, “Geometrically Constrained Structure from Motion: Points on Planes,” Proc. European Workshop 3D Structure from Multiple Images of Large-Scale Environments (SMILE), pp. 171-186, June 1998.
[20] C. Tomasi and T. Kanade, "Shape and Motion From Image Streams Under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[21] J.Y.A. Wang and E. Adelson, "Layered Representation for Motion Analysis," Proc. Computer Vision and Pattern Recognition Conf., 1993.
[22] L. Zelnik-Manor and M. Irani, “Multi-Frame Alignment of Planes,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 1999.
[23] L. Zelnik-Manor and M. Irani, “Multiview Subspace Constraints on Homographies,” Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp. 710-715, Sept. 1999.

Index Terms:
Motion estimation, plane alignment, multi-frame analysis, gradient-based methods.
Citation:
Lihi Zelnik-Manor, Michal Irani, "Multi-Frame Estimation of Planar Motion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 10, pp. 1105-1116, Oct. 2000, doi:10.1109/34.879791
Usage of this product signifies your acceptance of the Terms of Use.