This Article 
 Bibliographic References 
 Add to: 
Structure and Motion from Line Segments in Multiple Images
November 1995 (vol. 17 no. 11)
pp. 1021-1032

Abstract—This paper presents a new method for recovering the three dimensional structure of a scene composed of straight line segments using the image data obtained from a moving camera. The recovery algorithm is formulated in terms of an objective function which measures the total squared distance in the image plane between the observed edge segments and the projections (perspective) of the reconstructed lines. This objective function is minimized with respect to the line parameters and the camera positions to obtain an estimate for the structure of the scene. The effectiveness of this approach is demonstrated quantitatively through extensive simulations and qualitatively with actual image sequences. The implementation is being made publicly available.

[1] S. Ullman,The Interpretation of Visual Motion.Cambridge, Mass.: MIT Press, 1979.
[2] H.C. Longuet-Higgins,“A computer algorithm for reconstructing a scene from two projections,” Nature, vol. 293, pp. 133-135, 1981.
[3] J. Weng,T. Huang,, and N. Ahuja,“Motion and structure from two perspective views:Algorithms, error analysis and error estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 5, pp. 451-476, May 1989.
[4] X. Hu and N. Ahuja,“Motion and structure estimation using long sequence motion models,” Image and Vision Computing, vol. 11, no. 9, pp. 549-570, Nov. 1993.
[5] B.K.P. Horn, “Relative Orientation,” Int'l J. Computer Vision, vol. 4, pp. 59-78, 1990.
[6] J. Weng,T. S. Huang,, and N. Ahuja,Motion and Structure from Image Sequences, Springer Series on Information Sciences. Berlin: Springer-Verlag, 1993.
[7] J.L. Crowley,P. Stelmaszyk,T. Skordas,, and P. Puget,“Measurement and integration of 3D structures by tracking edge lines,” Int’l J. Computer Vision, vol. 8, no. 1, pp. 29-52, July 1992.
[8] O.D. Faugeras,F. Lustaman,, and G. Toscani,“Motion and structure from point and line matches,” Proc. Int’l Conf. Computer Vision, pp. 25-33, June 1987.
[9] J.L. Jezouin and N. Ayache,“3D structure from a monocular sequence of images,” Proc. Int’l Conf. Computer Vision, p. 441, Dec. 1990.
[10] T. Vieville and O. Faugeras,“Feed-forward recovery of motion and structure from a sequence of 2D-lines matches,” Proc. Int’l Conf. Computer Vision, p. 517, Dec. 1990.
[11] C. Tomasi and T. Kanade, "Shape and Motion From Image Streams Under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[12] R. Szeliski and S.B. Kang,“Recovering 3D shape and motion from image streams using nonlinear least squares,” J. Visual Communication and Image Representation, vol. 5, no. 1, pp. 10-28, Mar. 1994.
[13] T. Vieville,“Estimation of 3D-motion and structure from tracking 2D-lines in a sequence of images,” Proc. European Conf. Computer Vision, p. 281, Apr. 1990.
[14] B. Giai-Checa and T. Vieville,“3D-vision for active visual loops using locally rectilinear edges,” Proc. 1992 IEEE Int’l Symp. Intelligent Control, p. 341, Aug. 1992.
[15] N. Navab,R. Deriche,, and O.D. Faugeras,“Recovering 3D motion and structure from stereo and 2D token tracking,” Proc. Int’l Conf. Computer Vision, p. 513, Dec. 1990.
[16] J. Weng,Y. Liu,T.S. Huang,, and N. Ahuja,“Estimating motion/structure from line correspondences: A robust linear algorithm and uniqueness theorems,” Proc. IEEE Conf. Comp. Vision and Pattern Recognition, pp. 387-392, 1988.
[17] M.E. Spetsakis and J. Aloimonos, “Structure from Motion Using Line Correspondences,” Int'l J. Computer Vision, vol. 4, pp. 171-183, 1990.
[18] M. Spetsakis,“A linear algorithm for point and line based structure from motion,” CVGIP:Image Understanding, vol. 56, no. 2, Sept. 1992.
[19] O.D. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint.Cambridge, Mass.: MIT Press, 1993.
[20] C. Taylor, D. Kriegman, and P. Anandan, "Structure and Motion from Multiple Images: A Least Squares Approach," IEEE Workshop on Visual Motion, pp. 242-248, Oct. 1991.
[21] J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679-698, June 1986.
[22] J. Craig, Introduction to Robotics: Mechanics and Control, Addison Wesley Longman, Reading, Mass., 1986.
[23] G.H. Golub and C.F. Van Loan,Matrix Computations.Baltimore, Md.: Johns Hopkins Univ. Press, 1983.
[24] Y. Liu,T.S. Huang,, and O.D. Faugeras,“Determination of camera location from 2D to 3D line and point correspondences,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 82-88, 1988.
[25] R. Kumar and A.R. Hanson,“Robust estimation of camera location and orientation from noisy data having outliers,” Proc. Workshop on the Interpretation of 3D Scenes, pp. 52-60, Nov. 1989.
[26] C.J. Taylor and D.J. Kriegman,“Minimization on the lie group SO(3) and related manifolds,” Technical Report 9405, Center for Systems Science, Dept. of Electrical Engineering, Yale Univ., New Haven, Conn., Apr. 1994.
[27] S Smith,“Geometric Optimization Methods for Adaptive Filtering,” PhD thesis, Harvard Univ., Division of Applied Sciences, Cambridge Mass., Sept. 1993.
[28] M.M. Thompson,Manual of Photogrammetry. American Society of Photogrammetry, 1966.
[29] Y. Liu and T. Huang, “A Linear Algorithm for Determining Motion and Structure from Line Correspondences,” Computer Vision, Graphics, and Image Processing, vol. 44, no. 1, pp. 35-57, 1988.
[30] K. Sugihara, “An Algebraic Approach to Shape-from-Image problem,” Artificial Intelligence, vol. 23, pp. 59-95, 1984.

Index Terms:
Structure from motion, straight lines, three-dimensional reconstruction, perspective projection, numerical minimization.
Camillo J. Taylor, David J. Kriegman, "Structure and Motion from Line Segments in Multiple Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 11, pp. 1021-1032, Nov. 1995, doi:10.1109/34.473228
Usage of this product signifies your acceptance of the Terms of Use.