The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2011 vol.33)
pp: 807-822
Yong Seok Heo , Seoul National University, Seoul
Kyoung Mu Lee , Seoul National University, Seoul
Sang Uk Lee , Seoul National University, Seoul
ABSTRACT
A majority of the existing stereo matching algorithms assume that the corresponding color values are similar to each other. However, it is not so in practice as image color values are often affected by various radiometric factors such as illumination direction, illuminant color, and imaging device changes. For this reason, the raw color recorded by a camera should not be relied on completely, and the assumption of color consistency does not hold good between stereo images in real scenes. Therefore, the performance of most conventional stereo matching algorithms can be severely degraded under the radiometric variations. In this paper, we present a new stereo matching measure that is insensitive to radiometric variations between left and right images. Unlike most stereo matching measures, we use the color formation model explicitly in our framework and propose a new measure, called the Adaptive Normalized Cross-Correlation (ANCC), for a robust and accurate correspondence measure. The advantage of our method is that it is robust to lighting geometry, illuminant color, and camera parameter changes between left and right images, and does not suffer from the fattening effect unlike conventional Normalized Cross-Correlation (NCC). Experimental results show that our method outperforms other state-of-the-art stereo methods under severely different radiometric conditions between stereo images.
INDEX TERMS
Stereo matching, color, radiometric variation, illumination, camera exposure, gamma correction.
CITATION
Yong Seok Heo, Kyoung Mu Lee, Sang Uk Lee, "Robust Stereo Matching Using Adaptive Normalized Cross-Correlation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 4, pp. 807-822, April 2011, doi:10.1109/TPAMI.2010.136
REFERENCES
[1] http://vision.middlebury.edustereo/, 2010.
[2] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.
[3] D.J. Jobson, Z. Rahman, and G.A. Woodell, "A Multiscale Retinex for Bridging the Gap between Color Images and the Human Observation of Scenes," IEEE Trans. Image Processing, vol. 6, no. 7, pp. 965-976, July 1997.
[4] Y.S. Heo, K.M. Lee, and S.U. Lee, "Illumination and Camera Invariant Stereo Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[5] H. Hirschmüller and D. Scharstein, "Evaluation of Cost Functions for Stereo Matching," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[6] S. Birchfield and C. Tomasi, "A Pixel Dissimilarity Measure that Is Insensitive to Image Sampling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.
[7] H. Hirschmüller, P. Innocent, and J. Garibaldi, "Real-Time Correlation-Based Stereo Vision with Reduced Border Errors," Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 229-246, 2002.
[8] R. Zabih and J. Woodfill, "Non-Parametric Local Transforms for Computing Visual Correspondence," Proc. European Conf. Computer Vision, 1994.
[9] H. Hirschmüller, "Stereo Processing by Semiglobal Matching and Mutual Information," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008.
[10] O. Faugeras, B. Hotz, H. Mathieu, T. Viéville, Z. Zhang, P. Fua, E. Théron, L. Moll, G. Berry, J. Vuillemin, P. Bertin, and C. Proy, "Real Time Correlation-Based Stereo: Algorithm, Implementations and Applications," Technical Report RR-2013, INRIA, 1993.
[11] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S.M. Seitz, "Multi-View Stereo for Community Photo Collections," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[12] P. Viola and W.M. Wells, III, "Alignment by Maximization of Mutual Information," Int'l J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997.
[13] G. Egnal, "Mutual Information as a Stereo Correspondence Measure," Technical Report MS-CIS-00-20, Computer and Information Science, Univ. of Pennsylvania, 2000.
[14] J. Kim, V. Kolmogorov, and R. Zabih, "Visual Correspondence Using Energy Minimization and Mutual Information," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[15] L. Wang, R. Yang, and J.E. Davis, "BRDF Invariant Stereo Using Light Transport Constancy," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1616-1626, Sept. 2007.
[16] A.S. Ogale and Y. Aloimonos, "Robust Contrast Invariant Stereo Correspondence," Proc. IEEE Int'l Conf. Robotics and Automation, 2004.
[17] S. Negahdaripour, "Revised Definition of Optical Flow: Integration of Radiometric and Geometric Cues for Dynamic Scene Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 9, pp. 961-979, Sept. 1998.
[18] J. Zhang, L. McMillan, and J. Yu, "Robust Tracking and Stereo Matching under Variable Illumination," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[19] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[20] T. Meltzer, C. Yanover, and Y. Weiss, "Globally Optimal Solutions for Energy Minimization in Stereo Vision Using Reweighted Belief Propagation," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[21] V. Kolmogorov, "Convergent Tree-Reweighted Message Passing for Energy Minimization," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 10, pp. 1568-1583, Oct. 2006.
[22] E.H. Land and J.J. McCann, "Lightness and Retinex Theory," J. Optical Soc. of Am. A, vol. 61, no. 1, pp. 1-11, 1971.
[23] B. Funt, F. Ciurea, and J. McCann, "Retinex in Matlab," Proc. Eighth Color Imaging Conf. Color Science Systems and Applications, 2000.
[24] R. Kimmel, M. Elad, D. Shaked, R. Keshet, and I. Sobel, "A Variational Framework for Retinex," Int'l J. Computer Vision, vol. 52, no. 1, pp. 7-23, 2003.
[25] D.H. Brainard and B.A. Wandell, "Analysis of the Retinex Theory of Color Vision," J. Optical Soc. of Am. A, vol. 3, no. 10, pp. 1651-1661, 1986.
[26] D.A. Forsyth, "A Novel Algorithm for Color Constancy," Int'l J. Computer Vision, vol. 5, no. 1, pp. 5-24, 1990.
[27] G.D. Finlayson, S.D. Hordley, and P.M. Hubel, "Color by Correlation: A Simple, Unifying Framework for Color Constancy," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1209-1221, Nov. 2001.
[28] B. Funt, K. Barnard, and L. Martin, "Is Machine Colour Constancy Good Enough?" Proc. European Conf. Computer Vision, 1998.
[29] G.D. Finlayson, B. Schiele, and J.L. Crowley, "Comprehensive Colour Image Normalization," Proc. European Conf. Computer Vision, 1998.
[30] G.D. Finlayson and S.D. Hordley, "Color Constancy at a Pixel," J. Optical Soc. of Am. A, vol. 18, no. 2, pp. 253-264, 2001.
[31] G.D. Finlayson and R. Xu, "Illuminant and Gamma Comprehensive Normalisation in Log RGB Space," Pattern Recognition Letters, vol. 24, no. 11, pp. 1679-1690, 2003.
[32] J.M. Geusebroek, R. van den Boomgaard, A.W.M. Smeulders, and H. Geerts, "Color Invariance," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 12, pp. 1338-1350, Dec. 2001.
[33] T. Gevers and A.W.M. Smeulders, "Color Based Object Recognition," Pattern Recognition, vol. 32, no. 1, pp. 453-464, 1999.
[34] T. Gevers and H. Stokman, "Robust Histogram Construction from Color Invariants for Object Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 1, pp. 113-118, Jan. 2004.
[35] G.D. Finlayson, S.D. Hordley, and M.S. Drew, "Removing Shadows from Images," Proc. European Conf. Computer Vision, 2002.
[36] M. Ebner, "Color Constancy Using Local Color Shifts," Proc. European Conf. Computer Vision, 2004.
[37] D. Berwick and S.W. Lee, "A Chromaticity Space for Specularity, Illumination Color- and Illumination Pose-Invariant 3D Object Recognition," Proc. IEEE Int'l Conf. Computer Vision, 1998.
[38] R.T. Tan and K. Ikeuchi, "Separating Reflection Components of Textured Surfaces Using a Single Image," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[39] T. Zickler, S.P. Mallick, D.J. Kriegman, and P.N. Belhumeur, "Color Subspaces as Photometric Invariants," Int'l J. Computer Vision, vol. 79, no. 1, pp. 13-30, 2008.
[40] S.A. Shafer, "Using Color to Separate Reflection Components," COLOR Research and Applications, vol. 10, no. 4, pp. 210-218, 1985.
[41] G.J. Klinker, S.A. Shafer, and T. Kanade, "A Physical Approach to Color Image Understanding," Int'l J. Computer Vision, vol. 4, no. 1, pp. 7-38, 1990.
[42] R.W.G. Hunt, The Reproduction of Color, fifth ed. Fountain Press, 1995.
[43] G.D. Finlayson, M.S. Drew, and B.V. Funt, "Diagonal Transforms Suffice for Color Constancy," Proc. IEEE Int'l Conf. Computer Vision, 1993.
[44] G.D. Finlayson, M.S. Drew, and B.V. Funt, "Spectral Sharpening: Sensor Transformations for Improved Color Constancy," J. Optical Soc. of Am. A, vol. 11, no. 5, pp. 1553-1563, 1994.
[45] H.Y. Chong, S.J. Gortler, and T. Zickler, "The von Kries Hypothesis and a Basis for Color Constancy," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[46] P.E. Debevec and J. Malik, "Recovering High Dynamic Range Radiance Maps from Photographs," Proc. ACM SIGGRAPH, 1997.
[47] T. Mitsunaga and S.K. Nayar, "Radiometric Self Calibration," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[48] M.D. Grossberg and S.K. Nayar, "Modeling the Space of Camera Response Functions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1272-1282, Oct. 2004.
[49] C. Tomasi and R. Manduchi, "Bilateral Filtering for Gray and Color Images," Proc. IEEE Int'l Conf. Computer Vision, 1998.
[50] K. Yoon and I.S. Kweon, "Adaptive Support-Weight Approach for Correspondence Search," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 4, pp. 650-656, Apr. 2006.
[51] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[52] R. Szeliski, R. Zabih, D. Scharstein, O. Veksler, V. Kolmogorov, A. Agarwala, M. Tappen, and C. Rother, "A Comparative Study of Energy Minimization Methods for Markov Random Fields with Smoothness-Based Priors," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 6, pp. 1068-1080, June 2008.
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool