The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2009 vol.31)
pp: 428-443
Wei Xiong , The Chinese University of Hong Kong, Hong Kong
Jiaya Jia , The Chinese University of Hong Kong, Shatin
ABSTRACT
In our fractional stereo matching problem, a foreground object with a fractional boundary is blended with a background scene using unknown transparencies. Due to the spatially varying disparities in different layers, one foreground pixel may be blended with different background pixels in stereo images, making the color constancy commonly assumed in traditional stereo matching not hold any more. To tackle this problem, in this paper, we introduce a probabilistic framework constraining the matching of pixel colors, disparities, and alpha values in different layers, and propose an automatic optimization method to solve a Maximizing a Posterior (MAP) problem using Expectation-Maximization (EM), given only a short-baseline stereo input image pair. Our method encodes the effect of background occlusion by layer blending without requiring a special detection process. The alpha computation process in our unified framework can be regarded as a new approach by natural image matting, which handles appropriately the situation when the background color is similar to that of the foreground object. We demonstrate the efficacy of our method by experimenting with challenging stereo images and making comparisons with state-of-the-art methods.
INDEX TERMS
Stereo, Applications, Image matting
CITATION
Wei Xiong, Jiaya Jia, "Fractional Stereo Matching Using Expectation-Maximization", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 3, pp. 428-443, March 2009, doi:10.1109/TPAMI.2008.98
REFERENCES
[1] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[2] Y.-Y. Chuang, B. Curless, D.H. Salesin, and R. Szeliski, “A Bayesian Approach to Digital Matting,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '01), vol. 2, pp. 264-271, 2001.
[3] A. Criminisi and A. Blake, “The SPS Algorithm: Patching Figural Continuity and Transparency by Split-Patch Search,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '04), vol. 1, pp. 342-349, 2004.
[4] Y. Deng, Q. Yang, X. Lin, and X. Tang, “A Symmetric Patch-Based Correspondence Model for Occlusion Handling,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '05), pp. 1316-1322, 2005.
[5] P.F. Felzenszwalb and D.P. Huttenlocher, “Efficient Belief Propagation for Early Vision,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '04), vol. 70, no. 1, pp. 261-268, 2004.
[6] S.W. Hasinoff, S.B. Kang, and R. Szeliski, “Boundary Matting for View Synthesis,” Computer Vision and Image Understanding, vol. 103, no. 1, pp. 22-32, 2006.
[7] L. Hong and G. Chen, “Segment-Based Stereo Matching Using Graph Cuts,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '04), vol. 1, pp. 74-81, 2004.
[8] N. Joshi, W. Matusik, and S. Avidan, “Natural Video Matting Using Camera Arrays,” Proc. ACM SIGGRAPH '06, pp. 779-786, 2006.
[9] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions Using Graph Cuts,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '01), vol. 2, p. 508, 2001.
[10] A. Levin, D. Lischinski, and Y. Weiss, “A Closed Form Solution to Natural Image Matting,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '06), pp. 61-68, 2006.
[11] A. Levin, A. Rav-Acha, and D. Lischinski, “Spectral Matting,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2007.
[12] T. Minka, Expectation-Maximization as Lower Bound Maximization, http://research.microsoft.com/~minka/papers em.html, 1998.
[13] D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 7-42, 2002.
[14] A.R. Smith and J.F. Blinn, “Blue Screen Matting,” Proc. ACM SIGGRAPH '96, pp. 259-268, 1996.
[15] J. Sun, J. Jia, C.-K. Tang, and H.-Y. Shum, “Poisson Matting,” Proc. ACM SIGGRAPH '04, pp. 315-321, 2004.
[16] J. Sun, Y. Li, S.B. Kang, and H.-Y. Shum, “Symmetric Stereo Matching for Occlusion Handling,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '05), vol. 2, pp. 399-406, 2005.
[17] J. Sun, Y. Li, S.B. Kang, and H.-Y. Shum, “Flash Matting,” ACM Trans. Graphics, vol. 25, no. 3, pp. 772-778, 2006.
[18] J. Sun, N.-N. Zheng, and H.-Y. Shum, “Stereo Matching Using Belief Propagation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 7, pp. 787-800, July 2003.
[19] R. Szeliski and P. Golland, “Stereo Matching with Transparency and Matting,” Int'l J. Computer Vision, vol. 32, no. 1, pp. 45-61, 1999.
[20] E. Trucco, A. Fusiello, and A. Verri, “Rectification with Unconstrained Stereo Geometry,” Proc. British Machine Vision Conf. (BMVC), 1997.
[21] Y. Tsin, S.B. Kang, and R. Szeliski, “Stereo Matching with Reflections and Translucency,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '03), vol. 1, p. 702, 2003.
[22] J. Wang and M.F. Cohen, “An Iterative Optimization Approach for Unified Image Segmentation and Matting,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '05), pp. 936-943, 2005.
[23] Y. Wexler, A.W. Fitzgibbon, and A. Zisserman, “Bayesian Estimation of Layers from Multiple Images,” Proc. European Conf. Computer Vision (ECCV '02), pp. 487-501, 2002.
[24] W. Xiong and J. Jia, “Stereo Matching on Objects with Fractional Boundary,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR), 2007.
[25] Q. Yang, L. Wang, R. Yang, H. Stewenius, and D. Nister, “Stereo Matching with Color-Weighted Correlation, Hierarchical Belief Propagation and Occlusion Handling,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '06), pp. 2347-2354, 2006.
[26] L. Zhang and S.M. Seitz, “Parameter Estimation for MRF Stereo,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '05), vol. 2, pp. 288-295, 2005.
[27] C.L. Zitnick, N. Jojic, and S.B. Kang, “Consistent Segmentation for Optical Flow Estimation,” Proc. IEEE Int'l Conf. Computer Vision (ICCV '05), pp. 1308-1315, 2005.
[28] C.L. Zitnick, S.B. Kang, M. Uyttendaele, S. Winder, and R. Szeliski, “High-Quality Video View Interpolation Using a Layered Representation,” ACM Trans. Graphics, vol. 23, no. 3, pp. 600-608, 2004.
25 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool