The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1094-1106
Yong Seok Heo , Dept. of Electr. Eng. & Comput. Sci., Seoul Nat. Univ., Seoul, South Korea
Kyoung Mu Lee , Dept. of Electr. Eng. & Comput. Sci., Seoul Nat. Univ., Seoul, South Korea
Sang Uk Lee , Dept. of Electr. Eng. & Comput. Sci., Seoul Nat. Univ., Seoul, South Korea
ABSTRACT
In this paper, we propose a method that infers both accurate depth maps and color-consistent stereo images for radiometrically varying stereo images. In general, stereo matching and performing color consistency between stereo images are a chicken-and-egg problem since it is not a trivial task to simultaneously achieve both goals. Hence, we have developed an iterative framework in which these two processes can boost each other. First, we transform the input color images to log-chromaticity color space, from which a linear relationship can be established during constructing a joint pdf of transformed left and right color images. From this joint pdf, we can estimate a linear function that relates the corresponding pixels in stereo images. Based on this linear property, we present a new stereo matching cost by combining Mutual Information (MI), SIFT descriptor, and segment-based plane-fitting to robustly find correspondence for stereo image pairs which undergo radiometric variations. Meanwhile, we devise a Stereo Color Histogram Equalization (SCHE) method to produce color-consistent stereo image pairs, which conversely boost the disparity map estimation. Experimental results show that our method produces both accurate depth maps and color-consistent stereo images, even for stereo images with severe radiometric differences.
INDEX TERMS
Image color analysis, Joints, Radiometry, Stereo vision, Robustness, Cameras, Probability density function, color consistency, Stereo matching, radiometric variation, mutual information, SIFT
CITATION
Yong Seok Heo, Kyoung Mu Lee, Sang Uk Lee, "Joint Depth Map and Color Consistency Estimation for Stereo Images with Different Illuminations and Cameras", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1094-1106, May 2013, doi:10.1109/TPAMI.2012.167
REFERENCES
[1] H. Hirschmüller and D. Scharstein, "Evaluation of Stereo Matching Costs on Images with Radiometric Differences," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 9, pp. 1582-1599, Sept. 2009.
[2] D. Scharstein and R. Szeliski, "A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms," Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, 2002.
[3] S. Birchfield and C. Tomasi, "A Pixel Dissimilarity Measure that is Insensitive to Image Sampling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.
[4] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S. Seitz, "Multi-View Stereo for Community Photo Collections," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[5] N. Snavely, S.M. Seitz, and R. Szeliski, "Modeling the World from Internet Photo Collections," Int'l J. Computer Vision, vol. 80, no. 2, pp. 189-210, 2008.
[6] http:/photosynth.net/, 2012.
[7] http:/www.photomodeler.com/, 2012.
[8] A. Chakrabarti, D. Scharstein, and T. Zickler, "An Empirical Camera Model for Internet Color Vision," Proc. British Machine Vision Conf., 2009.
[9] J. Yang and S.K. Shevell, "Stereo Disparity Improves Color Constancy," Vision Research, vol. 42, no. 1, pp. 1979-1989, 2002.
[10] W. Xiong and B. Funt, "Stereo Retinex," Proc. Canadian Conf. Computer and Robot Vision, 2006.
[11] Y.S. Heo, K.M. Lee, and S.U. Lee, "Mutual Information-Based Stereo Matching Combined with Sift Descriptor in Log-Chromaticity Color Space," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[12] Y.S. Heo, K.M. Lee, and S.U. Lee, "Simultaneous Color Consistency and Depth Map Estimation for Radiometrically Varying Stereo Images," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[13] P. Montesinos, V. Gouet, R. Deriche, and D. Pelé, "Matching Color Uncalibrated Images Using Differential Invariants," Image and Vision Computing, vol. 18, no. 1, pp. 659-671, 2000.
[14] A.S. Ogale and Y. Aloimonos, "Robust Contrast Invariant Stereo Correspondence," Proc. IEEE Int'l Conf. Robotics and Automation, 2004.
[15] Y.S. Heo, K.M. Lee, and S.U. Lee, "Robust Stereo Matching Using Adaptive Normalized Cross-Correlation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 807-822, Apr. 2011.
[16] P. Viola and W.M. WellsIII, "Alignment by Maximization of Mutual Information," Int'l J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997.
[17] J.P.W. Pluim, J.A. Maintz, and M.A. Viergever, "Image Registration by Maximization of Combined Mutual Information and Gradient Information," IEEE Trans. Medical Imaging, vol. 19, no. 8, pp. 809-814, Aug. 2000.
[18] J.P.W. Pluim, J.A. Maintz, and M.A. Viergever, "Mutual-Information-Based Registration of Medical Images: A Survey," IEEE Trans. Medical Imaging, vol. 22, no. 8, pp. 986-1004, Aug. 2003.
[19] D.B. Russakoff, C. Tomasi, T. Rohlfing, and C.R. Maurer, "Image Similarity Using Mutual Information of Regions," Proc. European Conf. Computer Vision, 2004.
[20] G. Egnal, "Mutual Information as a Stereo Correspondence Measure," Technical Report MS-CIS-00-20, Computer and Information Science, Univ. of Pennsylvania, 2000.
[21] J. Kim, V. Kolmogorov, and R. Zabih, "Visual Correspondence Using Energy Minimization and Mutual Information," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[22] H. Hirschmüller, "Stereo Processing by Semiglobal Matching and Mutual Information," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 328-341, Feb. 2008.
[23] C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, "High-Quality Video View Interpolation Using a Layered Representation," Proc. ACM Siggraph, 2004.
[24] H.-H. Jeon, A. Basso, and P.F. Driessen, "A Global Correspondence for Scale Invariant Matching Using Mutual Information and the Graph Search," Proc. IEEE Int'l Conf. Multimedia and Expo, 2006.
[25] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[26] C. Liu, J. Yuen, A. Torralba, J. Sivic, and W. Freeman, "Sift Flow: Dense Correspondence across Different Scenes," Proc. European Conf. Computer Vision, 2008.
[27] E. Tola, V. Lepetit, and P. Fua, "Daisy: An Efficient Dense Descriptor Applied to Wide-Baseline Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 815-830, May 2010.
[28] S.M. Seitz and S. Baker, "Filter Flow," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[29] M. Ebner, Color Constancy. Wiley-IS&T Series in Imaging Science and Technology, John Wiley & Sons, 2007.
[30] Y. Weiss, "Deriving Intrinsic Images from Image Sequences," Proc. IEEE Int'l Conf. Computer Vision, 2001.
[31] S. Lin, Y. Li, S. Kang, X. Tong, and H. Shum, "Diffuse-Specular Separation and Depth Recovery from Image Sequences," Proc. European Conf. Computer Vision, 2002.
[32] S. Kim and M. Pollefeys, "Robust Radiometric Calibration and Vignetting Correction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 562-576, Apr. 2008.
[33] Y.M.S. Kagarlitsky and Y. Hel-Or, "Piecewise-Consistent Color Mappings of Images Acquired under Various Conditions," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[34] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[35] X. Hu and P. Mordohai, "Evaluation of Stereo Confidence Indoors and Outdoors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[36] L. Hong and G. Chen, "Segment-Based Stereo Matching Using Graph Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[37] J. Sun, Y. Li, S.B. Kang, and H.-Y. Shum, "Symmetric Stereo Matching for Occlusion Handling," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[38] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach Toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[39] G. Finlayson, S. Hordley, G. Schaefer, and G.Y. Tian, "Illuminant and Device Invariant Colour Using Histogram Equalisation," Pattern Recognition, vol. 38, no. 2, pp. 179-190, 2005.
[40] http://vision.middlebury.edustereo/, 2012.
[41] R. Zabih and J. Woodfill, "Non-Parametric Local Transforms for Computing Visual Correspondence," Proc. European Conf. Computer Vision, 1994.
[42] L. Shen, P. Tan, and S. Lin, "Intrinsic Image Decomposition with Non-Local Texture Cues," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
303 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool