The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.11 - Nov. (2012 vol.34)
pp: 2121-2133
Xiaoyan Hu , Dept. of Comput. Sci., Stevens Inst. of Technol., Hoboken, NJ, USA
P. Mordohai , Dept. of Comput. Sci., Stevens Inst. of Technol., Hoboken, NJ, USA
ABSTRACT
We present an extensive evaluation of 17 confidence measures for stereo matching that compares the most widely used measures as well as several novel techniques proposed here. We begin by categorizing these methods according to which aspects of stereo cost estimation they take into account and then assess their strengths and weaknesses. The evaluation is conducted using a winner-take-all framework on binocular and multibaseline datasets with ground truth. It measures the capability of each confidence method to rank depth estimates according to their likelihood for being correct, to detect occluded pixels, and to generate low-error depth maps by selecting among multiple hypotheses for each pixel. Our work was motivated by the observation that such an evaluation is missing from the rapidly maturing stereo literature and that our findings would be helpful to researchers in binocular and multiview stereo.
INDEX TERMS
Cost function, Estimation, Reliability, Error analysis, Stereo image processing, Benchmark testing, Pattern matching, distinctiveness, Stereo vision, 3D reconstruction, confidence, correspondence
CITATION
Xiaoyan Hu, P. Mordohai, "A Quantitative Evaluation of Confidence Measures for Stereo Vision", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 11, pp. 2121-2133, Nov. 2012, doi:10.1109/TPAMI.2012.46
REFERENCES
[1] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 7-42, 2002.
[2] H. Hirschmüller and D. Scharstein, "Evaluation of Cost Functions for Stereo Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[3] C. Strecha, W. von Hansen, L. Van Gool, P. Fua, and U. Thoennessen, "On Benchmarking Camera Calibration and Multi-View Stereo for High Resolution Imagery," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[4] P. Kohli and P.H.S. Torr, "Measuring Uncertainty in Graph Cut Solutions," Computer Vision and Image Understanding, vol. 112, no. 1, pp. 30-38, 2008.
[5] M. Gong and Y. Yang, "Fast Unambiguous Stereo Matching Using Reliability-Based Dynamic Programming," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 998-1003, June 2005.
[6] X. Hu and P. Mordohai, "Evaluation of Stereo Confidence Indoors and Outdoors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[7] D. Scharstein and R. Szeliski, "High-Accuracy Stereo Depth Maps Using Structured Light," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. I-195-I-202, 2003.
[8] P. Mordohai, "The Self-Aware Matching Measure for Stereo," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[9] A. Bobick and S. Intille, "Large Occlusion Stereo," Int'l J. Computer Vision, vol. 33, no. 3, pp. 1-20, 1999.
[10] Q. Chen and G. Medioni, "A Volumetric Stereo Matching Method: Application to Image-Based Modeling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. I-29-I-34, 1999.
[11] Y. Wei and L. Quan, "Region-Based Progressive Stereo Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. I-106-I-113, 2004.
[12] M. Lhuillier and L. Quan, "A Quasi-Dense Approach to Surface Reconstruction from Uncalibrated Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 3, pp. 418-433, Mar. 2005.
[13] J. Cech and R. Sara, "Efficient Sampling of Disparity Space for Fast and Accurate Matching," Proc. CVPR Workshop Towards Benchmarking Automated Calibration, 2007.
[14] L. Matthies, "Stereo Vision for Planetary Rovers: Stochastic Modeling to Near Real-Time Implementation," Proc. SPIE, vol. 1570, pp. 187-200, 1991.
[15] D. Scharstein and R. Szeliski, "Stereo Matching with Nonlinear Diffusion," Int'l J. Computer Vision, vol. 28, no. 2, pp. 155-174, 1998.
[16] Z. Zhang and Y. Shan, "A Progressive Scheme for Stereo Matching," Proc. Second European Workshop 3D Structure from Multiple Images of Large-Scale Environments, pp. 68-85, 2001.
[17] H. Hirschmüller, P. Innocent, and J. Garibaldi, "Real-Time Correlation-Based Stereo Vision with Reduced Border Errors," Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 229-246, 2002.
[18] P. Merrell, A. Akbarzadeh, L. Wang, P. Mordohai, J.-M. Frahm, R. Yang, D. Nistér, and M. Pollefeys, "Real-Time Visibility-Based Fusion of Depth Maps," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[19] K. Yoon and I. Kweon, "Distinctive Similarity Measure for Stereo Matching under Point Ambiguity," Computer Vision and Image Understanding, vol. 112, no. 2, pp. 173-183, 2008.
[20] G. Egnal, M. Mintz, and R. Wildes, "A Stereo Confidence Metric Using Single View Imagery with Comparison to Five Alternative Approaches," Image and Vision Computing, vol. 22, no. 12, pp. 943-957, 2004.
[21] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S. Seitz, "Multi-View Stereo for Community Photo Collections," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[22] M. Pollefeys, D. Nistér, J.M. Frahm, A. Akbarzadeh, P. Mordohai, B. Clipp, C. Engels, D. Gallup, S.J. Kim, P. Merrell, C. Salmi, S. Sinha, B. Talton, L. Wang, Q. Yang, H. Stewénius, R. Yang, G. Welch, and H. Towles, "Detailed Real-Time Urban 3D Reconstruction from Video," Int'l J. Computer Vision, vol. 78, nos. 2/3, pp. 143-167, 2008.
[23] M. Jancosek, A. Shekhovtsov, and T. Pajdla, "Scalable Multi-View Stereo," Proc. IEEE Int'l Workshop 3-D Digital Imaging and Modeling, pp. 1526-1533, 2009.
[24] Y. Furukawa and J. Ponce, "Accurate, Dense, and Robust Multiview Stereopsis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 8, pp. 1362-1376, Aug. 2010.
[25] M. Brown, D. Burschka, and G. Hager, "Advances in Computational Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003.
[26] S.M. Seitz, B. Curless, J. Diebel, D. Scharstein, and R. Szeliski, "A Comparison and Evaluation of Multi-View Stereo Reconstruction Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 519-528, 2006.
[27] D. Neilson and Y. Yang, "Evaluation of Constructable Match Cost Measures for Stereo Correspondence Using Cluster Ranking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[28] H. Hirschmüller and D. Scharstein, "Evaluation of Stereo Matching Costs on Images with Radiometric Differences," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1582-1599, Sept. 2009.
[29] M. Gong, R. Yang, L. Wang, and M. Gong, "A Performance Study on Different Cost Aggregation Approaches Used in Real-Time Stereo Matching," Int'l J. Computer Vision, vol. 75, no. 2, pp. 283-296, 2007.
[30] F. Tombari, S. Mattoccia, L. di Stefano, and E. Addimanda, "Classification and Evaluation of Cost Aggregation Methods for Stereo Correspondence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[31] M. Bleyer, S. Chambon, U. Poppe, and M. Gelautz, "Evaluation of Different Methods for Using Colour Information in Global Stereo Matching Approaches," Proc. Int'l Soc. for Photogrammetry and Remote Sensing, pp. 63-68, 2008.
[32] M. Bleyer and S. Chambon, "Does Color Really Help in Dense Stereo Matching?" Proc. Int'l Symp. 3D Data Processing, Visualization, and Transmission, 2010.
[33] C. Dima and S. Lacroix, "Using Multiple Disparity Hypotheses for Improved Indoor Stereo," Proc. IEEE Int'l Conf. Robotics and Automation, vol. 4, pp. 3347-3353, 2002.
[34] S. Lefebvre, S. Ambellouis, and F. Cabestaing, "A Colour Correlation-Based Stereo Matching Using 1D Windows," Proc. IEEE Conf. Signal-Image Technologies and Internet-Based System, pp. 702-710, 2007.
[35] G. Egnal and R. Wildes, "Detecting Binocular Half-Occlusions: Empirical Comparisons of Five Approaches," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 8, pp. 1127-1133, Aug. 2002.
[36] P. Jodoin, C. Rosenberger, and M. Mignotte, "Detecting Half-Occlusion with a Fast Region-Based Fusion Procedure," Proc. British Machine Vision Conf., pp. I-417-I-426, 2006.
[37] D. Marr and T. Poggio, "Cooperative Computation of Stereo Disparity," Science, vol. 194, no. 4262, pp. 283-287, 1976.
[38] J. Sun, Y. Li, S. Kang, and H. Shum, "Symmetric Stereo Matching for Occlusion Handling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 399-406, 2005.
[39] Y. Deng, Q. Yang, X. Lin, and X. Tang, "Stereo Correspondence with Occlusion Handling in a Symmetric Patch-Based Graph-Cuts Model," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1068-1079, June 2007.
[40] L. Xu and J. Jia, "Stereo Matching: An Outlier Confidence Approach," Proc. European Conf. Computer Vision, pp. 775-787, 2008.
[41] D. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[42] R. Manduchi and C. Tomasi, "Distinctiveness Maps for Image Matching," Proc. Int'l Conf. Image Analysis and Processing, pp. 26-31, 1999.
[43] A. Bruhn and J. Weickert, "A Confidence Measure for Variational Optic Flow Methods," Geometric Properties from Incomplete Data, R. Klette, R. Kozera, L. Noakes, and J. Weickert, eds., pp. 283-297, Springer, 2006.
[44] J. Kostliva, J. Cech, and R. Sara, "Feasibility Boundary in Dense and Semi-Dense Stereo Matching," Proc. CVPR Workshop Towards Benchmarking Automated Calibration, 2007.
[45] D. Gallup, J.-M. Frahm, P. Mordohai, Q. Yang, and M. Pollefeys, "Real-Time Plane-Sweeping Stereo with Multiple Sweeping Directions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[46] O. Aodha, G. Brostow, and M. Pollefeys, "Segmenting Video into Classes of Algorithm-Suitability," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1054-1061, 2010.
[47] M. Reynolds, J. Doboš, L. Peel, T. Weyrich, and G.J. Brostow, "Capturing Time-of-Flight Data with Confidence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
32 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool