The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - September (2009 vol.31)
pp: 1582-1599
Heiko Hirschmüller , Institute of Robotics and Mechatronics, Wessling
Daniel Scharstein , Middlebury College, Middlebury
ABSTRACT
Stereo correspondence methods rely on matching costs for computing the similarity of image locations. We evaluate the insensitivity of different costs for passive binocular stereo methods with respect to radiometric variations of the input images. We consider both pixel-based and window-based variants like the absolute difference, the sampling-insensitive absolute difference, and normalized cross correlation, as well as their zero-mean versions. We also consider filters like LoG, mean, and bilateral background subtraction (BilSub) and nonparametric measures like Rank, SoftRank, Census, and Ordinal. Finally, hierarchical mutual information (HMI) is considered as pixelwise cost. Using stereo data sets with ground-truth disparities taken under controlled changes of exposure and lighting, we evaluate the costs with a local, a semiglobal, and a global stereo method. We measure the performance of all costs in the presence of simulated and real radiometric differences, including exposure differences, vignetting, varying lighting, and noise. Overall, the ranking of methods across all data sets and experiments appears to be consistent. Among the best costs are BilSub, which performs consistently very well for low radiometric differences; HMI, which is slightly better as pixelwise matching cost in some cases and for strong image noise; and Census, which showed the best and most robust overall performance.
INDEX TERMS
Stereo, matching cost, performance evaluation, radiometric differences.
CITATION
Heiko Hirschmüller, Daniel Scharstein, "Evaluation of Stereo Matching Costs on Images with Radiometric Differences", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 9, pp. 1582-1599, September 2009, doi:10.1109/TPAMI.2008.221
REFERENCES
[1] “Daimler AG Driving Car Stereo Sequences,” http://www.mi. auckland.ac.nz/EISATS, 2007.
[2] “Point Grey—Triclops Stereo Library,” http://www.ptgrey.com/products/triclopsSDK , 2009.
[3] “Videre Design—Small Vision System,” http://www.videre design.com/visionsvs.htm, 2009.
[4] N. Campbell, G. Vogiatzis, C. Hernandez, and R. Cipolla, “Multiple Hypotheses Depth-Maps for Multi-View Stereo,” Proc. European Conf. Computer Vision, 2008.
[5] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S.M. Seitz, “Multi-View Stereo for Community Photo Collections,” Proc. Int'l Conf. Computer Vision, Oct. 2007.
[6] J.-P. Pons, R. Keriven, and O. Faugeras, “Multi-View Stereo Reconstruction and Scene Flow Estimation with a Global Image-Based Matching Score,” Int'l J. Computer Vision, vol. 72, no. 2, pp. 179-193, Apr. 2007.
[7] S. Birchfield and C. Tomasi, “A Pixel Dissimilarity Measure ThatIs Insensitive to Image Sampling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.
[8] E.P. Baltsavias and D. Stallmann, “Spot Stereo Matching for Digital Terrain Model Generation,” Proc. Second Swiss Symp. Pattern Recognition and Computer Vision, pp. 61-72, 1993.
[9] K. Konolige, “Small Vision Systems: Hardware and Implementation,” Proc. Eighth Int'l Symp. Robotics Research, pp. 203-212, Oct. 1997.
[10] H. Hirschmüller, P.R. Innocent, and J.M. Garibaldi, “Real-Time Correlation-Based Stereo Vision with Reduced Border Errors,” Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 229-246, Apr.-June 2002.
[11] A. Ansar, A. Castano, and L. Matthies, “Enhanced Real-Time Stereo Using Bilateral Filtering,” Proc. Int'l Symp. 3D Data Processing, Visualization and Transmission, 2004.
[12] C. Tomasi and R. Manduchi, “Bilateral Filtering for Grey and Color Images,” Proc. IEEE Int'l Conf. Computer Vision, pp. 836-846, 1998.
[13] R. Zabih and J. Woodfill, “Non-Parametric Local Transforms for Computing Visual Correspondance,” Proc. European Conf. Computer Vision, pp. 151-158, May 1994.
[14] D. Bhat and S. Nayar, “Ordinal Measures for Image Correspondence,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 415-423, Apr. 1998.
[15] R. Sara and R. Bajcsy, “On Occluding Contour Artifacts in Stereo Vision,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[16] R.A. Gideon and R.A. Hollister, “A Rank Correlation Coefficient,” J. Am. Statistical Assoc., vol. 82, pp. 656-666, 1987.
[17] P. Viola and W.M. Wells, “Alignment by Maximization of Mutual Information,” Int'l J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997.
[18] R. Chrastek and J. Jan, “Mutual Information as a Matching Criterion for Stereo Pairs of Images,” Analysis of Biomedical Signals and Images, vol. 14, pp. 101-103, 1998.
[19] G. Egnal, “Mutual Information as a Stereo Correspondence Measure,” Technical Report MS-CIS-00-20, Computer and Information Science, Univ. of Pennsylvania, 2000.
[20] C. Fookes, M. Bennamoun, and A. Lamanna, “Improved Stereo Image Matching Using Mutual Information and Hierarchical Prior Probabilities,” Proc. IEEE Int'l Conf. Pattern Recognition, 2002.
[21] I. Sarkar and M. Bansal, “A Wavelet-Based Multiresolution Approach to Solve the Stereo Correspondence Problem UsingMutual Information,” IEEE Trans. Systems, Man, and Cybernetics—Part B: Cybernetics, vol. 37, no. 4, pp. 1009-1014, Aug. 2007.
[22] C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-Quality Video View Interpolation Using a Layered Representation,” Proc. ACM SIGGRAPH, 2004.
[23] J. Kim, V. Kolmogorov, and R. Zabih, “Visual Correspondence Using Energy Minimization and Mutual Information,” Proc. Int'l Conf. Computer Vision, Oct. 2003.
[24] H. Hirschmüller, “Stereo Processing by Semiglobal Matching and Mutual Information,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008.
[25] J. Zhang, L. McMillan, and J. Yu, “Robust Tracking and Stereo Matching under Variable Illumination,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 871-878, 2006.
[26] L. Wang, R. Yang, and J. Davis, “BRDF Invariant Stereo Using Light Transport Constancy,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9,pp. 1616-1626, Sept. 2007.
[27] A. Hertzmann and S.M. Seitz, “Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1254-1264, Aug. 2005.
[28] H. Jin, S. Soatto, and A.J. Yezzi, “Multi-View Stereo Beyond Lambert,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 171-178, June 2003.
[29] Y. Li, S. Lin, and H. Li, “Multibaseline Stereo in the Presence of Specular Reflections,” Proc. Int'l Conf. Pattern Recognition, vol. 3, 2002.
[30] M. Liao, L. Wang, R. Yang, and M. Gong, “Light Fall-Off Stereo,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2007.
[31] T.E. Zickler, P.N. Belhumeur, and D.J. Krigman, “Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction,” Int'l J. Computer Vision, vol. 49, nos. 2/3, pp. 215-227, 2002.
[32] M.Z. Brown, D. Burschka, and G.D. Hager, “Advances in Computational Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003.
[33] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 7-42, Apr.-June 2002.
[34] Middlebury Stereo Website, http://vision.middlebury.edu/stereo, 2009.
[35] L. Wang, M. Gong, M. Gong, and R. Yang, “How Far Can We Go with Local Optimization in Real-Time Stereo Matching,” Proc. Third Int'l Symp. 3D Data Processing, Visualization and Transmission, 2006.
[36] S. Gautama, S. Lacroix, and M. Devy, “Evaluation of Stereo Matching Algorithms for Occupant Detection,” Proc. Int'l Workshop Recognition, Analysis, and Tracking of Faces and Gestures in Real-Time Systems, pp. 177-184, 1999.
[37] J. Banks and P. Corke, “Quantitative Evaluation of Matching Methods and Validity Measures for Stereo Vision,” Int'l J. Robotics Research, vol. 20, no. 7, pp. 512-532, July 2001.
[38] C. Fookes, A. Maeder, S. Sridharan, and J. Cook, “Multi-Spectral Stereo Matching Using Mutual Information,” Proc. Int'l Symp. 3D Data Processing, Visualization and Transmission, pp. 961-968, 2004.
[39] K.-J. Yoon and I.-S. Kweon, “Adaptive Support-Weight Approach for Correspondence Search,” IEEE Trans. Pattern Matching and Machine Intelligence, vol. 28, no. 4, pp. 650-656, Apr. 2006.
[40] J. Cech and R. Sara, “Complex Correlation Statistic for Dense Stereoscopic Vision,” Proc. Scandinavian Conf. Image Analysis, pp.598-608, 2005.
[41] H. Moravec, “Toward Automatic Visual Obstacle Avoidance,” Proc. Fifth Int'l Joint Conf. Artificial Intelligence, pp. 584-590, Aug. 1977.
[42] C.L. Zitnick, personal communication.
[43] D.N. Bhat, S.K. Nayar, and A. Gupta, “Motion Estimation Using Ordinal Measures,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[44] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[45] H. Hirschmüller, “Stereo Vision Based Mapping and Immediate Virtual Walkthroughs,” PhD dissertation, School of Computing, De Montfort Univ., Leicester, U.K., June 2003.
[46] A.F. Bobick and S.S. Intille, “Large Occlusion Stereo,” Int'l J. Computer Vision, vol. 33, no. 3, pp. 181-200, Sept. 1999.
[47] I.J. Cox, S.L. Hingorani, S.B. Rao, and B.M. Maggs, “A Maximum Likelihood Stereo Algorithm,” Computer Vision and Image Understanding, vol. 63, no. 3, pp. 542-567, May 1996.
[48] Y. Boykov and V. Kolmogorov, “An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124-1137, Sept. 2004.
[49] V. Kolmogorov and R. Zabih, “What Energy Functions Can Be Minimized via Graph Cuts?” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.
[50] R. Szeliski, E. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agrawala, M. Tappen, and C. Rother, “A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June 2008.
[51] D. Scharstein and R. Szeliski, “High-Accuracy Stereo Depth Maps Using Structured Light,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 195-202, June 2003.
[52] J. Woodfill and B. von Herzen, “Real-Time Stereo Vision on the Parts Reconfigurable Computer,” Proc. Fifth IEEE Symp. Field-Programmable Gate Arrays (FPGAs) for Custom Computing Machines, pp. 201-210, Apr. 1997.
[53] P.I. Corke, P.A Dunn, and J.E. Banks, “Frame-Rate Stereopsis Using Non-Parametric Transforms and Programmable Logic,” Proc. IEEE Int'l Conf. Robotics and Automation, vol. 3, pp. 1928-1933, May 1999.
[54] O. Faugeras, B. Hotz, H. Mathieu, T. Vieville, Z. Zhang, P. Fua, E. Thron, L. Moll, G. Berry, J. Vuillemin, P. Bertin, and C. Proy, “Real Time Correlation-Based Stereo: Algorithm, Implementations and Application,” Technical Report 2013, INRIA, Aug. 1993.
5 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool