This Article 
 Bibliographic References 
 Add to: 
Estimation of Displacements from Two 3-D Frames Obtained From Stereo
December 1992 (vol. 14 no. 12)
pp. 1141-1156

A method for estimating 3D displacements from two stereo frames is presented. It is based on the hypothesize-and-verify paradigm used to match 3D line segments between the two frames. In order to reduce the complexity of the method, an assumption is made that objects are rigid. The formulate a set of complete rigidity constraints for 3D line segments and integrate the uncertainty of measurements in this formation. The hypothesize-and-verify stages of the method use an extended Kalman filter to produce estimates of the displacements and of their uncertainty. The algorithm is shown to work on indoor and natural scenes. It is also shown to be easily extended to the case in which several mobile objects are present. The method is quite robust, fast, and has been thoroughly tested on hundreds of real stereo frames.

[1] T. Huang, "Motion analysis," inAI Encyclopedia. New York: Wiley, 1986, pp. 620-632.
[2] H. Nagel, "Image sequences--Ten (octal) years from phenomenology toward a theoretical foundation," inProc. 8th Int'l Conf. Patt. Recog.(Paris, France), Oct. 1986, pp. 1174-1185.
[3] J. K. Aggarwal and N. Nandhakumar, "On the computation of motion from sequences of images--A review,"Proc. IEEE, vol. 76, no. 8, pp. 917-935, 1988.
[4] S. T. Barnard and M. A. Fischler, "Computational stereo,"Comput. Surveys, vol. 14, no. 4, pp. 553-572, 1982.
[5] W. Grimson, "Computational experiments with a feature based stereo algorithm,"IEEE Trans. Patt. Anal. Machine Intell., vol. PAMI-7, no. 1, pp. 17-34, 1985.
[6] S. Pollard, J. Mayhew, and J. Frisby, "PMF: A stereo correspondence algorithm using a disparity gradient limit,"Perception, vol. 14, pp. 449-470, 1985.
[7] N. Ayache and B. Favejon, "A fast stereo vision matcher based on prediction and recursive verification of hypotheses," inProc. IEEE Workshop Comput. Vision: Representation Contr.Oct. 1985, pp. 27-37; Shorter version inProc. CVPR(San Francisco, CA), 1985.
[8] N. Ayache and F. Lustman, "Fast and reliable passive trinocular stereovision," inProc. First Int. Conf. Comput. Vision(London), June 1987, pp. 422-427.
[9] O. Faugeraset al., "Toward a flexible vision system," inRobot Vision(A. Pugh, Ed.). London: IFS, 1983, pp. 129-142, ch. 3.
[10] B. Horn and B. Schunk, "Determining optical flow,"Artif. Intell., vol. 20, pp. 199-228, 1981.
[11] H. Nagel, "Displacement vectors derived from second-order intensity variations in image sequences,"Comput. vision Graphics Image Processing, vol. 21, pp. 85-117, 1983.
[12] E. Hildreth,The Measurement of Visual Motion, Cambridge, MA: MIT Press, 1983.
[13] A. Bruss and B. Horn, "Passive navigation,"Comput. vision Graphics Image Processing, vol. 21, pp. 3-20, Jan. 1983.
[14] G. Adiv, "Inherent ambiguities in recovering 3-D motion and structure from a noisy flow field," inProc. IEEE Conf. Comput. Vision Patt. Recog.(San Francisco, CA), June 1985, pp. 70-77.
[15] S. Maybank, "A theoretical study of optical flow," Ph.D. thesis, Birkbeck College, Univ. of London, 1987.
[16] S. Ullman,The Interpretation of Visual Motion. Cambridge, MA: MIT Press, 1979.
[17] R. Tsai and T. Huang, "Estimating 3-D motion parameters of a rigid planar patch, i,"IEEE Trans. Acous. Speech Signal Processing, vol. ASSP-29, pp. 1147-1152, Dec. 1981.
[18] T. Huang and R. Tsai, "Image sequence analysis: Motion estimation," inImage Sequence Analysis, (T. Huang, Ed.). Berlin, Heidelberg: Springer, 1981, pp. 1-18, ch. 3.
[19] H. Longuet-Higgins, "A computer algorithm for reconstructing a scene from two projections,"Nature, vol. 293, pp. 133-135, 1981.
[20] A. Mitiche, S. Seida, and J. Aggarwal, "Line based computation of structure and motion using angular invariance," inProc. Workshop Motion: Representation Anal.(Charleston, SC), May 1986, pp. 175-180.
[21] O. Faugeras, F. Lustman, and G. Toscani, "Motion and structure from motion from point and line matches, " inProc. First Int. Conf. Comput. Vision(London), 1987, pp. 25-34.
[22] O. Faugeras and S. Maybank, "Motion from point matches: multiplicity of solutions,"Int. J. Comput. Vision, vol. 4, pp. 225-246, June 1990.
[23] T. Broida and R. Chellappa, "Experiments and uniqueness results on object structure and kinematics from a sequences of monocular images," inProc. IEEE Workshop Visual Motion(Irvine, CA), Mar. 1989, pp. 21-30.
[24] J. Weng, T.S. Huang, and N. Ahuja, "3-D motion estimation, understanding, and prediction from noisy image sequences,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, no. 3, 1987.
[25] G. Young and R. Chellappa, "3-D motion estimation using a sequence of noisy stereo images: Models, estimation, and uniqueness results,"IEEE Trans. Patt. Anal. Macvhine Intell., vol. 12, pp. 735-759, Aug. 1990.
[26] Z. Zhang and O. Faugeras, "Tracking and grouping 3-D line segments," inProc. Third Int. Conf. Comput. Vision(Osaka, Japan), Dec. 1990, pp. 577-580.
[27] H. Shariat and K. Price, "Motion estimation with more than two frames,"IEEE Trans. Patt. Anal. Machine Intell., vol. 12, pp. 417-434, May 1990.
[28] T. J. Broida, S. Chandrashekhar, and R. Chellappa, "Recursive estimation of 3-D kinematics and structure from noisy monocular image sequences,"IEEE Trans. Aerosp. Electron. Syst., vol. AES-26, pp. 639-656, Aug. 1990.
[29] S. Pollard, J. Porrill, J. Mayhew, and J. Frisby, "Matching geometrical descriptions in three-space,"Image Vision Comput., vol. 5, pp. 73-78, May 1987.
[30] R. Bolles and R. Cain, "Recognizing and locating partially visible objects: The local-feature-focus method,"Int. J. Robotics Res., vol. 1, no. 3, pp. 57-82, 1982.
[31] R. Horaud and R.C. Bolles, "3DPO's strategy for matching three-dimensional objects in range data," inProc. Int. Conf. Robotics, (Atlanta, GA), Mar. 1984, pp. 78-85.
[32] W. Grimson and T. Lozano-Perez, "Model-based recognition and localization from sparse range or tactile data,"Int. J. Robotics Res., vol. 5, pp. 3-34, Fall 1984.
[33] H. H. Chen and T. S. Huang, "Maximal matching of 3-D points for multiple-object motion estimation,"Patt. Recognition, vol. 21, no. 2, pp. 75-90, 1988.
[34] H. Chen and T. Huang, "An algorithm for matching 3-D line segments with application to multiple-object motion estimation," inProc. IEEE Workshop Comput. Vision, Nov. 30-Dec. 2, 1987, pp. 151-156.
[35] Y.C. Kim and J.K. Aggarwal, "Determining object motion in a sequence of stereo images,"IEEE J. Robotics and Automat., vol. RA-3, pp. 599-614, Dec. 1987.
[36] S. Blostein and T. Huang, "Estimation 3-D motion from range data," inProc. First Conf. Artif. Intell. Applications(Denver, CO), Dec. 1984, pp. 246-250.
[37] T. Huang and S. Blostein, "Robust algorithms for motion estimation based on two sequential stereo image pairs," inProc. IEEE Conf. Comput. Vision Pattern Recog.(San Francisco, CA), 1985, pp. 518-523.
[38] O. D. Faugeras and M. Hebert, "The representation, recognition, and locating of 3-D objects,"Int. J. Robotics Res., vol. 5, no. 3, Fall 1986, pp. 27-52.
[39] O. D. Faugeras, N. Ayache, B. Faverjon, and F. Lutsman, "Building visual maps by combining noisy stereo measurements," inProc. IEEE Conf. Robotics Automat.pp. 1433-1438, 1987.
[40] N. Ayache and O. Faugeras, "Building, registrating and fusing noisy visual maps," inProc. First Int. Conf. Comput. Vision(London), June 1987, pp. 73-82.
[41] Z. Lin, H. Lee, and T. Huang, "Finding 3-D point correspondence in motion estimation," inProc. 8th Int. Conf. Patt. Recog.(Paris, France), 1986, pp. 303-305.
[42] W. E. L. Grimson and T. Lozano-Perez, "Localizing overlapping parts by searching the interpretation tree,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, no. 4, July 1987.
[43] N. Ayache and O. D. Faugeras, "HYPER: A new approach for the recognition and positioning of two-dimensional objects,"IEEE Trans. Patt. Anal. Machine Intell., vol. 8, no. 1, pp. 44-54, 1986.
[44] D. Lowe, Perceptual Organization And Visual Recognition. Boston: Kluwer, 1985.
[45] D. Lowe, "The viewpoint consistency constraint,"Int. J. Comput. Vision, no. 1, pp. 57-72, 1987.
[46] O. D. Faugeras, N. Ayache, and Z. Zhang, "A preliminary investigation of the problem of determining ego and object motions from stereo," inProc. 9th ICPR(Rome, Italy), 1988, pp. 242-246.
[47] Z. Zhang, O. D. Faugeras, and N. Ayache, "Analysis of a sequence of stereo scenes containing multiple moving objects using rigidity constraints," inProc. Second Int. Conf. Computer Vision, Tampa, FL, 1988, pp. 177-187.
[48] Z. Zhang, "Motion analysis from a sequence of stereo frames and its applications," Ph.D. thesis, Univ of Paris, Orsay, Paris, France, 1990, in English.
[49] H. Durrant-Whyte, "Uncertain geometry in robotics,"IEEE J. Robotics Automat., vol. 4, pp. 23-31, Feb. 1988.
[50] N. Ayache and O. D. Faugeras, "Maintaining representations of the environment of a mobile robot," inInt. Symp. Robotics Research, Santa Cruz, CA, Aug. 1987.
[51] N. Ayache,Artificial Vision for Mobile Robots: Stereo Vision and Multisensory Perception. Cambridge, MA: MIT Press, 1991.
[52] N. Ayache, O. D. Faugeras, "Maintaining representations of the environment of a mobile robot,"IEEE Trans. Robot. Automation, vol. 5, no. 6, pp. 804-819, 1989.
[53] K. S. Roberts, "A new representation for a line," inProc. Comput. Vision," Patt. Recogn., Ann Arbor, MI, 1988.
[54] Z. Zhang and O. Faugeras, "Building a 3-D world model with a mobile robot: 3-D line segment representation and integration," inProc. 10th Int. Conf. Patt. Recog.(Atlantic City, NJ), June 1990, pp. 38-42.
[55] D. Murray and D. Cook, "Using the orientation of fragmentary 3-D edge segments for polyhedral object recognition,"Int. J. Comput. Vision, no. 2, pp. 153-169, 1988.
[56] K. S. Arun, T. S. Huang, and S. D. Blostein, "Least-squares fitting of two 3-D point sets,"IEEE Trans. Pattern Anal. Machine Intell., vol. PAMI-9, no. 5, pp. 698-700, 1987.
[57] Z. Zhang and O. Faugeras, "Determining motion from 3-D line segments: A comparative study," inProc. Brit. Machine vision Conf.(London), Sept. 24-27, 1990, pp. 85-90.
[58] A. Jazwinsky,Stochastic Processes and Filtering Theory. New York: Academic, 1970.
[59] P. Maybeck,Stochastic Models, Estimation and Control. New York: Academic, 1979, vol. 1.
[60] P. Maybeck,Stochastic Models, Estimation and Control. New York: Academic, 1982, vol. 2.
[61] B. K. P. Horn,Robot Vision. Cambridge, MA: M.I.T. Press, 1986.
[62] O. D. Faugeras, "A few steps toward artificial 3-D vision," inRobotics Science, (M. Brady, Ed.). Cambridge, MA: MIT Press, 1989, pp. 39-137, ch. 2.
[63] O. Rodrigues, "Des lois géométriques qui régissent les déplacements d'un systéme solide dans l'espace, et de la variation des coordonnées provenant de ces déplacements considérés indépendamment des causes qui peuvent les produire,"J. Mathématiques Pures et Appliquées, vol. 5, pp. 380-440, 1840.
[64] Z. Zhang and O. Faugeras, "Calibration of a mobile robot with application to visual navigation," inProc. IEEE Workshop Visual Motion(Irvine, CA), Mar. 1989, pp. 306-313.

Index Terms:
3D displacement estimation; line segment matching; stereo; hypothesize-and-verify paradigm; complexity; rigidity constraints; extended Kalman filter; image segmentation; Kalman filters; motion estimation; stereo image processing
Z. Zhang, O.D. Faugeras, "Estimation of Displacements from Two 3-D Frames Obtained From Stereo," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 14, no. 12, pp. 1141-1156, Dec. 1992, doi:10.1109/34.177380
Usage of this product signifies your acceptance of the Terms of Use.