The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - Sept. (2013 vol.35)
pp: 2175-2188
Qifeng Chen , Dept. of Comput. Sci., Stanford Univ., Stanford, CA, USA
Dingzeyu Li , Dept. of Comput. Sci., Columbia Univ., New York, NY, USA
Chi-Keung Tang , Dept. of Comput. Sci. & Eng., Hong Kong Univ. of Sci. & Technol., Hong Kong, China
ABSTRACT
This paper proposes to apply the nonlocal principle to general alpha matting for the simultaneous extraction of multiple image layers; each layer may have disjoint as well as coherent segments typical of foreground mattes in natural image matting. The estimated alphas also satisfy the summation constraint. As in nonlocal matting, our approach does not assume the local color-line model and does not require sophisticated sampling or learning strategies. On the other hand, our matting method generalizes well to any color or feature space in any dimension, any number of alphas and layers at a pixel beyond two, and comes with an arguably simpler implementation, which we have made publicly available. Our matting technique, aptly called KNN matting, capitalizes on the nonlocal principle by using K nearest neighbors (KNN) in matching nonlocal neighborhoods, and contributes a simple and fast algorithm that produces competitive results with sparse user markups. KNN matting has a closed-form solution that can leverage the preconditioned conjugate gradient method to produce an efficient implementation. Experimental evaluation on benchmark datasets indicates that our matting results are comparable to or of higher quality than state-of-the-art methods requiring more involved implementation. In this paper, we take the nonlocal principle beyond alpha estimation and extract overlapping image layers using the same Laplacian framework. Given the alpha value, our closed form solution can be elegantly generalized to solve the multilayer extraction problem. We perform qualitative and quantitative comparisons to demonstrate the accuracy of the extracted image layers.
INDEX TERMS
Kernel, Laplace equations, Image color analysis, Materials, Vectors, Mathematical model, Image segmentation,layer extraction, Natural image matting
CITATION
Qifeng Chen, Dingzeyu Li, Chi-Keung Tang, "KNN Matting", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 9, pp. 2175-2188, Sept. 2013, doi:10.1109/TPAMI.2013.18
REFERENCES
[1] R. Barrett, M. Berry, T.F. Chan, J. Demmel, J. Donato, J. Dongarra, V. Eijkhout, R. Pozo, C. Romine, and H.V. der Vorst, Templates for the Solution of Linear Systems: Building Blocks for Iterative Methods, second ed. SIAM, 1994.
[2] A. Buades, B. Coll, and J.-M. Morel, "Nonlocal Image and Movie Denoising," Int'l J. Computer Vision, vol. 76, no. 2, pp. 123-139, 2008.
[3] Q. Chen, D. Li, and C.-K. Tang, "KNN Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 869-876, 2012.
[4] Y. Chuang, B. Curless, D.H. Salesin, and R. Szeliski, "A Bayesian Approach to Digital Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 264-271, 2001.
[5] F.H. Cole, "Automatic BRDF Factorization," bachelor honors thesis, Harvard College, 2002.
[6] E.S.L. Gastal and M.M. Oliveira, "Shared Sampling for Real-Time Alpha Matting," Computer Graphics Forum, vol. 29, no. 2, pp. 575-584, May 2010.
[7] D.B. Goldman, B. Curless, A. Hertzmann, and S.M. Seitz, "Shape and Spatially-Varying BRDFS from Photometric Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 6, pp. 1060-1071, June 2010.
[8] Y. Guan, W. Chen, X. Liang, Z. Ding, and Q. Peng, "Easy Matting —A Stroke Based Approach for Continuous Image Matting," Computer Graphics Forum, vol. 25, no. 3, pp. 567-576, Sept. 2006.
[9] K. He, C. Rhemann, C. Rother, X. Tang, and J. Sun, "A Global Sampling Method for Alpha Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2049-2056, 2011.
[10] K. He, J. Sun, and X. Tang, "Fast Matting Using Large Kernel Matting Laplacian Matrices," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2165-2172, 2010.
[11] J. Lawrence, A. Ben-artzi, C. Decoro, W. Matusik, H. Pfister, R. Ramamoorthi, and S. Rusinkiewicz, "Inverse Shade Trees for Non-Parametric Material Representation and Editing," ACM Trans. Graphics, pp. 735-745, 2006.
[12] P. Lee and Y. Wu, "Nonlocal Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2193-2200, 2011.
[13] D. Lepage and J. Lawrence, "Material Matting," ACM Trans. Graphics, vol. 30, no. 6, article 144, 2011.
[14] A. Levin, D. Lischinski, and Y. Weiss, "A Closed-Form Solution to Natural Image Matting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 228-242, Feb. 2008.
[15] A. Levin, A. Rav-Acha, and D. Lischinski, "Spectral Matting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1699-1712, Oct. 2008.
[16] H. Lin, Y.-W. Tai, and M.S. Brown, "Motion Regularization for Matting Motion Blurred Objects," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 11, pp. 2329-2336, Nov. 2011.
[17] M.D. McCool, J. Ang, and A. Ahmad, "Homomorphic Factorization of BRDFS for High-Performance Rendering," Proc. ACM Siggraph, pp. 171-178, 2001.
[18] C. Rhemann, C. Rother, and M. Gelautz, "Improving Color Modeling for Alpha Matting," Proc. British Machine Vision Conf., pp. 1-10, 2008.
[19] C. Rhemann, C. Rother, P. Kohli, and M. Gelautz, "A Spatially Varying PSF-Based Prior for Alpha Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2149-2156, 2010.
[20] C. Rhemann, C. Rother, J. Wang, M. Gelautz, P. Kohli, and P. Rott, "A Perceptually Motivated Online Benchmark for Image Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1826-1833, 2009.
[21] E. Shahrian and D. Rajan, "Weighted Color and Texture Sample Selection for Image Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 718-725, 2012.
[22] D. Singaraju and R. Vidal, "Estimation of Alpha Mattes for Multiple Image Layers," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 7, pp. 1295-1309, July 2011.
[23] A.R. Smith and J.F. Blinn, "Blue Screen Matting," Proc. ACM Siggraph, pp. 259-268, 1996.
[24] J. Sun, J. Jia, C.-K. Tang, and H.-Y. Shum, "Poisson Matting," ACM Trans. Graphics, vol. 23, pp. 315-321, Aug. 2004.
[25] A. Vedaldi and B. Fulkerson, "VLFeat: An Open and Portable Library of Computer Vision Algorithms," http:/www.vlfeat. org/, 2008.
[26] J. Wang and M.F. Cohen, "An Iterative Optimization Approach for Unified Image Segmentation and Matting," Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 936-943, 2005.
[27] J. Wang and M.F. Cohen, "Optimized Color Sampling for Robust Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[28] J. Wang and M.F. Cohen, Image and Video Matting. Now Publishers Inc., 2008.
[29] Z. Zhang, Q. Zhu, and Y. Xie, "Learning Based Alpha Matting Using Support Vector Regression," Proc. IEEE Int'l Conf. Image Processing, pp. 2109-2112, 2012.
[30] Y. Zheng and C. Kambhamettu, "Learning Based Digital Matting," Proc. IEEE Int'l Conf. Computer Vision, pp. 889-896, 2009.
40 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool