This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Advances in Computational Stereo
August 2003 (vol. 25 no. 8)
pp. 993-1008

Abstract—Extraction of three-dimensional structure of a scene from stereo images is a problem that has been studied by the computer vision community for decades. Early work focused on the fundamentals of image correspondence and stereo geometry. Stereo research has matured significantly throughout the years and many advances in computational stereo continue to be made, allowing stereo to be applied to new and more demanding problems. In this paper, we review recent advances in computational stereo, focusing primarily on three important topics: correspondence methods, methods for occlusion, and real-time implementations. Throughout, we present tables that summarize and draw distinctions among key ideas and approaches. Where available, we provide comparative analyses and we make suggestions for analyses yet to be done.

[1] P. Aschwanden and W. Guggenbuhl, Experimental Results from a Comparative Study on Correlation-Type Registration Algorithms Robust Computer Vision, Forstner and Ruwiedel, eds., pp. 268-289, Wickmann, 1993.
[2] H.H. Baker, Depth from Edge and Intensity Based Stereo Technical Report AIM-347, Artificial Intelligence Laboratory, Stanford Univ., 1982.
[3] J. Banks and P. Corke, Quantitative Evaluation of Matching Methods and Validity Measures for Stereo Vision Int'l J. Robotics Research, vol. 20, no. 7, 2001.
[4] S. Barnard and M. Fischler, “Computational Stereo,” ACM Computing Surveys, vol. 14, no. 4, pp. 553–572, 1982.
[5] P.N. Belhumeur, “A Bayesian Approach to Binocular Stereopsis,” Int'l J. Computer Vision, vol. 19, no. 3, pp. 237-260, 1996.
[6] D. Beymer and K. Konolige, Real-Time Tracking of Multiple People Using Continuous Detection Proc. IEEE Frame Rate Workshop, 1999.
[7] D.N. Bhat and S.K. Nayar, “Ordinal Measures for Image Correspondence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 415–423, Apr. 1998.
[8] F. Bigone, O. Henricsson, P. Fua, and M. Stricker, Automatic Extraction of Generic House Roofs from High Resolution Aerial Imagery Proc. European Conf. Computer Vision, pp. 85-96, 1996.
[9] S. Birchfield and C. Tomasi, Depth Discontinuities by Pixel-to-Pixel Stereo Technical Report STAN-CS-TR-96-1573, Stanford Univ., 1996.
[10] S. Birchfield and C. Tomasi, Depth Discontinuities by Pixel-to-Pixel Stereo Proc. IEEE Int'l Conf. Computer Vision, pp. 1073-1080, 1998.
[11] S. Birchfield and C. Tomasi, Multiway Cut for Stereo and Motion with Slanted Surfaces Proc. Int'l Conf. Computer Vision, pp. 489-495, 1999.
[12] Y. Boykov, O. Veksler, and R. Zabih, Markov Random Fields with Efficient Approximations Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.
[13] Y. Boykov, O. Veksler, and R. Zabih, Fast Approximate Energy Minimization via Graph Cuts IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[14] Y. Boykov and V. Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision,” Proc. Int'l Workshop Energy Minimization Methods in Computer Vision and Pattern Recognition, pp. 359-374, Sept. 2001.
[15] C. Chang, S. Chatterjee, and P.R. Kube, “On an Analysis of Static Occlusion in Stereo Vision,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 722-723, 1991.
[16] W.-S. Ching, A New Method of Identifying Occlusion and Specular Highlights Using Active Vision Proc. Int'l Symp. Speech, Image Processing and Neural Networks, pp. 437-440, 1994.
[17] S.D. Cochran and G. Medioni, "3-D Surface Description From Binocular Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 10, pp. 981-994, Oct. 1992.
[18] P. Corke and P. Dunn, Real-Time Stereopsis Using FPGAs Proc. IEEE TENCON Speech and Image Technologies for Computing and Telecommunications, pp. 235-238, 1997.
[19] I. Cox, S. Hingorani, and S. Rao, “A Maximum Likelihood Stereo Algorithm,” Computer Vision and Image Understanding, vol. 63, no. 3, pp. 542–567, 1996.
[20] T.H. Cormen,C.E. Leiserson, and R.L. Rivest,Introduction to Algorithms.Cambridge, Mass.: MIT Press/McGraw-Hill, 1990.
[21] J.D. Crisman and J.A. Webb, "The Warp Machine on Navlab," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 5, pp. 451-465, May 1991.
[22] T. Darrell, G. Gordon, M. Harville, and J. Woodfill, “Integrated Person Tracking Using Stereo, Color, and Pattern Detection,” Proc. Computer Vision and Pattern Recognition (CVPR '98), pp. 601-608, 1998.
[23] U. Dhond and J.K. Aggarwal, "Structure From Stereo—A Review," IEEE Trans. Systems, Man, and Cybernetics, vol. 19, no. 6, pp. 1,489-1,510, Nov. 1989.
[24] U.R. Dhond and J.K. Aggarwal, Analysis of the Stereo Correspondence Process in Scenes with Narrow Occluding Objects Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 470-473, 1992.
[25] G. Egnal and R.P. Wildes, “Detecting Binocular Half-Occlusions: Empirical Comparisons of Four Approaches,” Proc. Conf. Computer Vision and Pattern Recognition, vol. II, pp. 466-473, 2000.
[26] O. Faugeras, B. Hotz, H. Matthieu, T. Vieville, Z. Zhang, P. Fua, E. Theron, L. Moll, G. Berry, J. Vuillemin, P. Bertin, and C. Proy, Real Time Correlation-Based Stereo: Algorithm, Implementations and Applications INRIA Technical Report 2013, 1993.
[27] O. Faugeras and R. Keriven, “Variational Principles, Surface Evolution, PDE's, Level Set Methods and the Stereo Problem,” IEEE Trans. Image Processing, vol. 7, no. 3, pp. 336-344, 1998.
[28] O. Faugeras and Q.-T. Luong, The Geometry of Multiple Images. Cambridge, Mass.: The MIT Press, 2001.
[29] P. Fua, A Parallel Stereo Algorithm that Produces Dense Depth Maps and Preserves Image Features Machine Vision and Applications, vol. 6, pp. 35-49, 1993.
[30] P. Fua and Y.G. Leclerc, “Object-Centered Surface Reconstruction: Combining Multi-Image Stereo and Shading,” Int'l J. Computer Vision, vol. 16, pp. 35-56, Sept. 1995.
[31] A. Fusiello and V. Roberto, “Efficient Stereo with Multiple Windowing,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 858-863, 1997.
[32] A. Gruen, Adaptive Least Squares Correlation: A Powerful Image Matching Technique South African J. Photogrammetry, Remote Sensing and Cartography, vol. 3, no. 14, pp. 175-187, 1985.
[33] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[34] W. Hoff and N. Ahuja, "Surfaces From Stereo: Integrating Feature Matching, Disparity Estimation, and Contour Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 2, pp. 121-136, Feb. 1989.
[35] B.K.P. Horn and B.G. Schunk, Determining Optical Flow Artificial Intelligence, vol. 17, pp. 185-204, 1981.
[36] S.S. Intille and A.F. Bobick, Incorporating Intensity Edges in the Recovery of Occlusion Regions Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 674-677, 1994.
[37] M. Irani, B. Rousso, and S. Peleg, “Computing Occluding and Transparent Motions,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 5-16, Jan. 1994.
[38] T. Kanade, M. Okutomi, and T. Nakahara, A Multiple-Baseline Stereo Method Proc. ARPA Image Understanding Workshop, pp. 409-426, 1992.
[39] T. Kanade and M. Okutomi, “A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, pp. 920-932, 1994.
[40] T. Kanade, A. Yoshida, K. Oda, H. Kano, and M. Tanaka, "A Video-Rate Stereo Machine and Its New Applications," Proc. IEEE Conf. Computer Vision and Pattern Recognition 96, pp.196-202, 1996.
[41] S. Kimura, T. Kanade, H. Kano, A. Yoshida, E. Kawamura, and K. Oda, CMU Video-Rate Stereo Machine Proc. Mobile Mapping Symp. 1995.
[42] S. Kimura, T. Shinbo, H. Yamaguchi, E. Kawamura, and K. Naka, A Convolver-Based Real-Time Stereo Machine (SAZAN) Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 457-463, 1999.
[43] R. Klette, K. Schluns, and A. Koschan, Computer Vision, Three Dimensional Data from Images, Springer-Verlag, Singapore, 1998.
[44] V.S. Kluth, G.W. Kunkel, and U.A. Rauhala, Global Least Squares Matching Proc. Int'l Geoscience and Remote Sensing Symp., vol. 2, pp. 1615-1618, 1992.
[45] V. Kolmogorov and R. Zabih, Computing Visual Correspondence with Occlusions Using Graph Cuts Proc. Int'l Conf. Computer Vision, 2001.
[46] K. Konolige, Small Vision Systems: Hardware and Implementation Proc. Eighth Int'l Symp. Robotics Research, 1997.
[47] A. Koschan, What is New in Computational Stereo Since 1989: A Survey of Current Stereo Papers Technical Report 93-22, Technical Univ. of Berlin, 1993.
[48] K.N. Kutulakos and S.M. Seitz, “A Theory of Shape by Space Carving,” Int'l J. Computer Vision, vol. 38, no. 3, pp. 199-218, 2000.
[49] Z.-N. Li and H.W. Chin, Depth and Occlusion Recovery in Motion Stereo Proc. IEEE Int'l Conf. Systems, Man and Cybernetics, vol. 5, pp. 3890-3895, 1995.
[50] J.J. Little and W.E. Gillett,“Direct evidence for occlusion in stereo and motion,” Img. Vision Computg., vol. 8, pp. 328–340, Nov. 1990.
[51] B.D. Lucas and T. Kanade, An Iterative Image Registration Technique with an Application to Stereo Vision Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981.
[52] A.-R. Mansouri, A. Mitiche, and J. Konrad, Selective Image Diffusion: Application to Disparity Estimation Proc. Int'l Conf. Image Processing, vol. 3, pp. 284-288, 1998.
[53] H. Mathieu, A Multi-DSP 96002 Board INRIA Technical Report 153, 1993.
[54] L.H. Matthies, "Stereo Vision for Planetary Rovers: Stochastic Modeling to Near Real-Time Implementation," Int'l J. Computer Vision, Vol. 8, No. 1, 1992, pp. 71-91.
[55] L. Matthies, A. Kelly, T. Litwin, and G. Tharp, Obstacle Detection for Unmanned Ground Vehicles: A Progress Report Proc. Intelligent Vehicles '95 Symp., pp. 66-71, 1995.
[56] M.J. McDonnell, Box-Filtering Techniques Computer Graphics and Image Processing, vol. 17, pp. 65-70, 1981.
[57] K. Muhlmann, D. Maier, J. Hesser, and R. Manner, Calculating Dense Disparity Maps from Color Stereo Images, an Efficient Implementation Proc. IEEE Workshop Stereo and Multi-Baseline Vision, pp. 30-36, 2001.
[58] Y. Nakamura, T. Matsuura, K. Satoh, and Y. Ohta, Occlusion Detectable Stereo Occlusion Patterns in Camera Matrix Proc. Computer Vision and Pattern Recognition, pp. 371-378, 1996.
[59] H.K. Nishihara, Real-Time Stereo- and Motion-Based Figure-Ground Discrimination and Tracking Using LOG Sign-Correlation Proc. 27th Asilomar Conf. Signals, Systems, and Computers, pp. 95-100, 1993.
[60] Y. Ohta and T. Kanade, Stereo by Intra- and Intra-Scanline Search Using Dynamic Programming IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7 pp. 139-154, 1985.
[61] M. Pilon and P. Cohen, Occlusion Detection and Interpretation Based on Image Reprojection Proc. S.W. Symp. Image Analysis and Interpretation, pp. 166-171, 1996.
[62] S. Randriamasy and A. Gagalowicz, Region Based Stereo Matching Oriented Image Processing Proc. Computer Vision and Pattern Recognition, pp. 736-737, 1991.
[63] U.A. Rauhala, Introduction to Array Algebra Photogrammetric Engineering and Remote Sensing, vol. 46, no. 2, pp. 177-192, 1980.
[64] M.K. Reed and P.K. Allen, Constraint-Based Sensor Planning for Scene Modeling IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp. 1460-1467, 2000.
[65] S. Roy and I. Cox, A Maximum-Flow Formulation of the N-Camera Stereo Correspondence Problem IEEE Proc. Int'l Conf. Computer Vision, pp. 492-499, 1998.
[66] R. Sara and R. Bajcsy, On Occluding Contour Artifacts in Stereo Vision Proc. Computer Vision and Pattern Recognition, pp. 852-857, 1997.
[67] K. Satoh and Y. Ohta, Occlusion Detectable Stereo Systematic Comparison of Detection Algorithms Proc. Int'l Conf. Pattern Recognition, pp. 280-286, 1996.
[68] D. Scharstein and R. Szeliski, Stereo Matching with Nonlinear Diffusion Int'l J. Computer Vision, vol. 28, no. 2, pp. 155-174, 1998.
[69] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” IJCV, vol. 47, no. 1-3, pp. 7-42, Apr. 2002.
[70] S. Scherer, P. Werth, and A. Pinz, The Discriminatory Power of Ordinal Measures Towards a New Coefficient Proc. Computer Vision and Pattern Recognition, vol. 1, pp. 76-81, 1999.
[71] J. Shah, A Nonlinear Diffusion Model for Discontinuous Disparity and Half-Occlusions in Stereo Proc. Computer Vision and Pattern Recognition, pp. 34-40, 1993.
[72] C. Schmid and A. Zisserman, The Geometry and Matching of Curves in Multiple Views Proc. European Conf. Computer Vision, pp. 104-118, 1998.
[73] J. Shi and C. Tomasi, Good Features to Track Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[74] C. Silva and J. Santos-Victor, Intrinsic Images for Dense Stereo Matching with Occlusions Proc. European Conf. Computer Vision, pp. 100-114, 2000.
[75] C.V. Stewart, “Bias in Robust Regression Caused by Discontinuities and Multiple Structures,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 8, pp. 816-833, Aug. 1997.
[76] C. Sun, A Fast Stereo Matching Method Proc. Digital Image Computing: Techniques and Applications, pp. 95-100, 1997.
[77] J. Sun, H.-Y. Shum, and N.-N. Zheng, Stereo Matching Using Belief Propagation Proc. European Conf. Computer Vision, pp. 510-524, 2002.
[78] K. Tarabanis, P. Allen, and R. Tsai, A Survey of Sensor Planning in Computer Vision IEEE Trans. Robotics and Automation, vol. 11, pp. 86-104, 1995.
[79] I. Thomos, S. Malasiotis, and M.G. Strintzis, Optimized Block Based Disparity Estimation in Stereo Systems Using a Maximum-Flow Approach Proc. SIBGRAPI '98 Conf., 1998.
[80] C. Tomasi and R. Manduchi, Stereo without Search Technical Report STAN-CS-TR-95-1543, Stanford Univ., 1995.
[81] C. Tomasi and R. Manduchi, Stereo Matching as a Nearest-Neighbor Problem IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, pp. 333-340, 1998.
[82] E. Trucco and A. Verri, Introductory Techniques for 3D Computer Vision. Upper Saddle River, N.J.: Prentice-Hall, 1998.
[83] C.-J. Tsai and A.K. Katsaggelos, Dense Disparity Estimation with a Divide-and-Conquer Disparity Space Image Technique IEEE Trans. Multimedia, vol. 1, pp. 18-28, 1999.
[84] V. Venkateswar and R. Chellappa, “Hierarchical Stereo and Motion Correspondence Using Feature Groupings,” Int'l J. Computer Vision, vol. 15, pp. 245-269, 1995.
[85] J.A. Webb, Implementation and Performance of Fast Parallel Multi-Baseline Stereo Vision Proc. DARPA Image Understanding Workshop, pp. 1005-1012, 1993.
[86] R.P. Wildes, “Direct Recovery of Three-Dimensional Scene Geometry from Binocular Stereo Disparity,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 8, pp. 761-774, 1991.
[87] J. Woodfill and B. Von Herzen, Real-Time Stereo Vision on the PARTS Reconfigurable Computer Proc. IEEE Workshop FPGAs for Custom Computing Machines, pp. 242-250, 1997.
[88] A.L. Yuille and T. Poggio, A Generalized Ordering Constraint for Stereo Correspondence A.I. Laboratory Memo 777, MIT, Cambridge, Mass., 1984.
[89] R. Zabih and J. Woodfill, “Non-Parametric Local Transforms for Computing Visual Correspondence,” Proc. European Conf. Computer Vision, May 1994.
[90] C. L. Zitnick and T. Kanade, A Cooperative Algorithm for Stereo Matching and Occlusion Detection IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 7, July 2000.
[91] Z. Zhang, Determining the Epipolar Geometry and Its Uncertainty: A Review INRIA Technical Report 2927, 1996.
[92] H. Zhao, Global Optimal Surface from Stereo Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 101-104, 2000.

Index Terms:
Computational stereo, stereo correspondence, occlusion, real-time stereo, review.
Citation:
Myron Z. Brown, Darius Burschka, Gregory D. Hager, "Advances in Computational Stereo," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003, doi:10.1109/TPAMI.2003.1217603
Usage of this product signifies your acceptance of the Terms of Use.