This Article 
 Bibliographic References 
 Add to: 
A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment
September 1994 (vol. 16 no. 9)
pp. 920-932

A central problem in stereo matching by computing correlation or sum of squared differences (SSD) lies in selecting an appropriate window size. The window size must be large enough to include enough intensity variation for reliable matching, but small enough to avoid the effects of projective distortion. If the window is too small and does not cover enough intensity variation, it gives a poor disparity estimate, because the signal (intensity variation) to noise ratio is low. If, on the other hand, the window is too large and covers a region in which the depth of scene points (i.e., disparity) varies, then the position of maximum correlation or minimum SSD may not represent correct matching due to different projective distortions in the left and right images. For this reason, a window size must be selected adaptively depending on local variations of intensity and disparity. The authors present a method to select an appropriate window by evaluating the local variation of the intensity and the disparity. The authors employ a statistical model of the disparity distribution within the window. This modeling enables the authors to assess how disparity variation, as well as intensity variation, within a window affects the uncertainty of disparity estimate at the center point of the window. As a result, the authors devise a method which searches for a window that produces the estimate of disparity with the least uncertainty for each pixel of an image: the method controls not only the size but also the shape (rectangle) of the window. The authors have embedded this adaptive-window method in an iterative stereo matching algorithm: starting with an initial estimate of the disparity map, the algorithm iteratively updates the disparity estimate for each point by choosing the size and shape of a window till it converges. The stereo matching algorithm has been tested on both synthetic and real images, and the quality of the disparity maps obtained demonstrates the effectiveness of the adaptive window method.

[1] S. T. Barnard and M. A. Fischler, "Stereo vision," inEncyclopedia of Artificial Intelligence. New York: John Wiley, 1987, pp. 1083-1090.
[2] B. B. Mandelbrot and B. J. Van Ness, "Fractional brownian motion, fractional noises and applications,"SIAM, vol. 10, no. 4, pp. 422-438, 1968.
[3] J. C. Candy, M. A. Franke, B. G. Haskell, and F. W. Mounts, "Transmitting television as clusters of frame-to-frame differences,"Bell Syst. Tech. J., vol. 50, no. 6, pp. 1889-1919, 1971.
[4] F. deCoulon,Signal Theory and Processing. Norwood, MA: Artech House, Inc., 1986.
[5] M. Drumheller and T. Poggio, "On parallel stereo," in Proc. IEEE Int Conf. Robotics and Automation, Cambridge, MA, Apr. 1986, pp. 1439-1448.
[6] W. Forstner and A. Pertl,Photogrammetric Standard Methods and Digital Image Matching Techniques for High Precision Surface Measurements. New York: Elsevier Science Publishers B.V., 1986, pp. 57-72.
[7] W. E. L. Grimson, "Computational experiments with a feature based stereo algorithm,"IEEE Trans. Pattern Anal. Machine Intell., vol. 7, no. 1, pp. 17-34, Jan. 1985. (The shape of the support region is due to personal communication.)
[8] J. M. Hakkarainen, J. J. Little. H. Lee, Jr. and J. L. Wyatt, "Interaction of algorithm and implementation for analog VLSI stereo vision," inSPIE Visual Inform. Processing: From Neurons to Chips, pp. 173-184, 1991.
[9] M. J. Hannah, "A system for digital stereo image matching,"Photogrammetric Engineering and Remote Sensing, vol. 55, no. 12, pp. 1765-1770, Dec. 1989.
[10] W. Hoff and N. Ahuja, "Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection,"IEEE Trans. Pattern Anal. Machine Intell., vol. 11, no. 2, 1989.
[11] M. D. Levine, D. A. O'Handley, and G. M. Yagi, "Computer determination of depth maps,"Comput. Graphics and Image Processing, vol. 2, no. 4, pp. 131-150, 1973.
[12] D. Marr and T. Poggio, "Cooperative computation of stereo disparity,"Science, vol. 194, pp. 283-287, Oct. 1976.
[13] L. Matthies, R. Szeliski, and T. Kanade, "Kalman filter-based algorithms for estimating depth from image sequences,"Int. J. Comput. Vision, vol. 3, pp. 209-236, 1989.
[14] K. Mori, M. Kidode, and H. Asada, "An iterative prediction and correction method for automatic sterocomparison,"Comput. Graphics and Image Processing, vol. 2, pp. 393-401, 1973.
[15] M. Okutomi and T. Kanade, "A locally adaptive window for signal matching," inProc. Int. Conf. Comput. Vision, Dec. 1990; also inInt. J. Comput. Vision, vol. 7, no. 2, pp. 143-162, 1992.
[16] M. Okutomi and T. Kanade, "A multiple-baseline stereo," inProc. Comput. Vision Pattern Recognit., June 1991, pp. 63-69. Also inIEEE Trans. Pattern Anal. Machine Intell., vol. 15, no. 4, pp. 353-363, 1993.
[17] J. B. O'Neal, "Predictive quantizing systems for the transmission of television signals,"Bell Syst. Tech. J., vol. 45, no. 5, pp. 689-722, 1966.
[18] D. J. Panton, "A flexible approach to digital stereo mapping,"Photogram. Eng. Remote Sensing, vol. 44, no. 12, pp. 1499-1512, Dec. 1978.
[19] S. B. Pollard, J. E. W. Mayhew, and J. P. Frisby, "Pmf: A stereo correspondence algorithm using a disparity gradient limit,"Perception, vol. 14, pp. 449-470, 1985.
[20] S. B. Pollard, J. Porrill, J. E. W. Mayhew, and J. P. Frisby, "Disparity gradient, Lipschitz continuity, and computing binocular correspondences," inRobotics Research, The Third International Symposium, O. Faugeras and G. Giralt Eds. Cambridge MA: The MIT Press, 1986, pp. 19-26.
[21] K. Prazdny, "Detection of binocular disparities,"Biological Cybern., vol. 52, pp. 93-99, 1985.
[22] R. F. Voss, "Fractals in nature," inCourse note on FRACTALS: Introduction, Basics, and Perspectives, 1987.
[23] G. A. Wood, "Realities of automatic correlation problem,"Photogrammetric Engineering and Remote Sensing, vol. 49, pp. 537-538, Apr. 1983.

Index Terms:
stereo image processing; image sequences; iterative methods; correlation methods; statistical analysis; stereo matching algorithm; adaptive window; window size; intensity variation; reliable matching; projective distortion; disparity estimate; maximum correlation; sum of squared differences; statistical model; disparity distribution; disparity variation; uncertainty; iterative stereo matching algorithm; disparity map; synthetic images; real images
T. Kanade, M. Okutomi, "A Stereo Matching Algorithm with an Adaptive Window: Theory and Experiment," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 16, no. 9, pp. 920-932, Sept. 1994, doi:10.1109/34.310690
Usage of this product signifies your acceptance of the Terms of Use.