This Article 
 Bibliographic References 
 Add to: 
A Cooperative Algorithm for Stereo Matching and Occlusion Detection
July 2000 (vol. 22 no. 7)
pp. 675-684

Abstract—This paper presents a stereo algorithm for obtaining disparity maps with occlusion explicitly detected. To produce smooth and detailed disparity maps, two assumptions that were originally proposed by Marr and Poggio are adopted: uniqueness and continuity. That is, the disparity maps have a unique value per pixel and are continuous almost everywhere. These assumptions are enforced within a three-dimensional array of match values in disparity space. Each match value corresponds to a pixel in an image and a disparity relative to another image. An iterative algorithm updates the match values by diffusing support among neighboring values and inhibiting others along similar lines of sight. By applying the uniqueness assumption, occluded regions can be explicitly identified. To demonstrate the effectiveness of the algorithm, we present the processing results from synthetic and real image pairs, including ones with ground-truth values for quantitative comparison with other methods.

[1] H.H. Baker and T.O. Binford, “Depth from Edge and Intensity Based Stereo,” Proc. Seventh Int'l Joint Conf. Artificial Intelligence, pp. 631–636, 1981.
[2] S.T. Barnard and M.A. Fischler, “Stereo Vision,” Encyclopedia of Artificial Intelligence, pp. 1,083–1,090, New York: John Wiley, 1987.
[3] P.N. Belhumeur and D. Mumford, "A Bayesian Treatment of the Stereo Correspondence Problem Using Half-Occluded Regions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 506-512, June 1992.
[4] Y. Boykov, O. Veksler, and R. Zabih, Markov Random Fields with Efficient Approximations Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.
[5] R.T. Collins, “A Space-Sweep Approach to True Multi-Image Matching,” Proc. Computer Vision and Pattern Recognition, pp. 358-363, 1996.
[6] P. Fua, “A Parallel Stereo Algorithm that Produces Dense Depth Maps and Preserves Image Features,” Machine Vision and Applications, vol. 6, pp. 35–49, 1993.
[7] W. Forstner and A. Pertl, Photogrammetric Standard Methods and Digital Image Matching Techniques for High Precision Surface Measurements. pp. 57–72, New York: Elsevier Science B. V., 1986.
[8] D. Geiger, B. Ladendorf, and A. Yuille, “Occlusions and Binocular Stereo,” Int'l J. Computer Vision, vol. 14, pp. 211-226, 1995.
[9] W.E.L. Grimson, “Computational Experiments with a Feature Based Stereo Algorithm,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol 7, no. 1, pp. 17–34, 1985.
[10] S.S. Intille and A.F. Bobick, Incorporating Intensity Edges in the Recovery of Occlusion Regions Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 674-677, 1994.
[11] S.S. Intille and A.F. Bobick, “Disparity-Space Images and Large Occlusion Stereo,” Proc. European Conf. Computer Vision, pp. 179-186, 1994.
[12] T. Kanade and M. Okutomi, “A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, pp. 920-932, 1994.
[13] M. Levine, D. O'Handley, and G. Yagi, “Computer Determination of Depth Maps,” Computer Graphics and Image Processing, vol. 2, pp. 131–150, 1973.
[14] S.A. Lloyd, E.R. Haddow, and J.F. Boyce, “A Parallel Binocular Stereo Algorithm Utilizing Dynamic Programming and Relaxation Labelling,” Computer Vision, Graphics, and Image Processing, vol. 39, pp. 202–225, 1987.
[15] D. Marr and T. Poggio, “Cooperative Computation of Stereo Disparity,” Science vol. 194, pp. 209–236, 1976.
[16] D. Marr and T. Poggio, “A Computational Theory of Human Stereo Vision,” Proc. Royal Soc. London B, vol. 204, pp. 301–328, 1979.
[17] H. Moravec, “Robot Spatial Perception by Stereoscopic Vision and 3D Evidence Grids,” CMU Robotics Institute Technical Report CMU-RI-TR-96-34, 1996.
[18] K. Mori, M. Kidode, and H. Asada, “An Iterative Prediction and Correction Method for Automatic Stereo Comparison,” Computer Graphics and Image Processing, vol. 2, pp. 393–401, 1973.
[19] Y. Nakamura, T. Matsuura, K. Satoh, and Y. Ohta, Occlusion Detectable Stereo Occlusion Patterns in Camera Matrix Proc. Computer Vision and Pattern Recognition, pp. 371-378, 1996.
[20] H.K. Nishihara and T. Poggio, “Stereo Vision for Robotics,” Proc. First Int'l Symp. Robotics Research, Cambridge, Mass.: MIT Press, pp. 489–505, 1984.
[21] M. Okutomi and T. Kanade, “A Multiple-Baseline Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 353-363, Apr. 1993.
[22] Y. Ohta and T. Kanade, “Stereo by Intra- and Inter-Scanline Search using Dynamic Programming,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, pp. 139–154, 1985.
[23] D.J. Panton, “A Flexible Approach to Digital Stereo Mapping,” Photogrammetric Eng., Remote Sensing, vol. 44, no. 12, pp. 1,499–1,512, 1978.
[24] T. Poggio, “The Marr and Poggio algorithm for real scenes was not defined so any implementation will be a change; however the algorithm worked well on random-dot stereograms and there are several papers to support this,” Personal communication Aug. 1999.
[25] S.B. Pollard, J.E.W. Mayhew, and J.P. Frisby, “Pmf: A Stereo Correspondence Algorithm Using a Disparity Gradient Limit,” Perception, vol. 14, pp. 449–470, 1985.
[26] K. Prazdny, “Detection of Binocular Disparities,” Biological Cybernetics, vol. 52, no. 2, pp. 93–99, 1985.
[27] D. Scharstein and R. Szeliski, Stereo Matching with Nonlinear Diffusion Int'l J. Computer Vision, vol. 28, no. 2, pp. 155-174, 1998.
[28] R. Szeliski and R. Zabih, “An Experimental Comparison of Stereo Algorithms,” Proc. Int'l Workshop Vision Algorithms, pp. 1-19, 1999.
[29] G. Wood, “Realities of Automatic Correlation Problem,” Photogrammetric Eng., and Remote Sensing, vol. 49, pp. 537–538, 1983.
[30] C.L. Zitnick and T. Kanade, “A Volumetric Iterative Approach to Stereo Matching and Occlusion Detection,” CMU Technical Report CMU-RI-98-30, 1998.

Index Terms:
Stereo vision, occlusion detection, 3D vision.
C. Lawrence Zitnick, Takeo Kanade, "A Cooperative Algorithm for Stereo Matching and Occlusion Detection," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 7, pp. 675-684, July 2000, doi:10.1109/34.865184
Usage of this product signifies your acceptance of the Terms of Use.