The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2008 vol.30)
pp: 228-242
ABSTRACT
Interactive digital matting, the process of extracting a foreground object from an image based on limited user input, is an important task in image and video editing. From a computer vision perspective, this task is extremely challenging because it is massively ill-posed — at each pixel we must estimate the foreground and the background colors, as well as the foreground opacity (“alpha matte”) from a single color measurement. Current approaches either restrict the estimation to a small part of the image, estimating foreground and background colors based on nearby pixels where they are known, or perform iterative nonlinear estimation by alternating foreground and background color estimation with alpha estimation.In this paper we present a closed-form solution to natural image matting. We derive a cost function from local smoothness assumptions on foreground and background colors, and show that in the resulting expression it is possible to analytically eliminate the foreground and background colors to obtain a quadratic cost function in alpha. This allows us to find the globally optimal alpha matte by solving a sparse linear system of equations. Furthermore, the closed-form formula allows us to predict the properties of the solution by analyzing the eigenvectors of a sparse matrix, closely related to matrices used in spectral image segmentation algorithms. We show that high quality mattes for natural images may be obtained from a small amount of user input.
INDEX TERMS
Matting, Interactive Image Editing, Spectral Segmentation
CITATION
Anat Levin, Dani Lischinski, Yair Weiss, "A Closed-Form Solution to Natural Image Matting", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 2, pp. 228-242, February 2008, doi:10.1109/TPAMI.2007.1177
REFERENCES
[1] N.E. Apostoloff and A.W. Fitzgibbon, “Bayesian Video Matting Using Learnt Image Priors,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[2] A. Berman, P. Vlahos, and A. Dadourian, “Comprehensive Method for Removing from an Image the Background Surrounding a Selected Object,” US patent 6,135,345, 2000.
[3] Y. Boykov and M.P. Jolly, “Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images,” Proc. Eighth Int'l Conf. Computer Vision, 2001.
[4] C. Chennubhotla and A. Jepson, “Hierarchical Eigensolver for Transition Matrices in Spectral Methods,” Advances in Neural Information Processing Systems 17, L.K. Saul, Y. Weiss, and L.Bottou, eds. MIT Press, pp. 273-280, 2005.
[5] Y.Y. Chuang, A. Agarwala, B. Curless, D.H. Salesin, and R. Szeliski, “Video Matting of Complex Scenes,” ACM Trans. Graphics, vol. 21, no. 3, pp. 243-248, 2002.
[6] Y.Y. Chuang, B. Curless, D.H. Salesin, and R. Szeliski, “A Bayesian Approach to Digital Matting,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[7] Y.Y. Chuang, D.B. Goldman, B. Curless, D.H. Salesin, and R. Szeliski, “Shadow Matting and Compositing,” ACM Trans. Graphics, 2003.
[8] L. Grady, T. Schiwietz, S. Aharon, and R. Westermann, “Random Walks for Interactive Alpha-Matting,” Proc. Fifth IASTED Int'l Conf. Visualization, Imaging, and Image Processing, 2005.
[9] Y. Guan, W. Chen, X. Liang, Z. Ding, and Q. Peng, “Easy Matting,” Proc. Ann. Conf. European Assoc. for Computer Graphics, 2006.
[10] A. Levin, D. Lischinski, and Y. Weiss, “Colorization Using Optimization,” ACM Trans. Graphics, 2004.
[11] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, “Lazy Snapping,” ACM Trans. Graphics, vol. 23, no. 3, pp. 303-308, 2004.
[12] I. Omer and M. Werman, “Color Lines: Image Specific Color Representation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2004.
[13] C. Rother, V. Kolmogorov, and A. Blake, ““Grabcut”: Interactive Foreground Extraction Using Iterated Graph Cuts,” ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004.
[14] M.A. Ruzon and C. Tomasi, “Alpha Estimation in Natural Images,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[15] J. Shi and J. Malik, “Normalized Cuts and Image Segmentation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 731-737, 1997.
[16] R. Szeliski, “Locally Adapted Hierarchical Basis Preconditioning,” ACM Trans. Graphics, vol. 25, no. 3, pp. 1135-1143, 2006.
[17] J. Sun, J. Jia, C.-K. Tang, and H.-Y. Shum, “Poisson Matting,” ACM Trans. Graphics, vol. 23, no. 3, pp. 315-321, 2004.
[18] A. Torralba and W.T. Freeman, “Properties and Applications of Shape Recipes,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2003.
[19] J. Wang and M. Cohen, “An Iterative Optimization Approach for Unified Image Segmentation and Matting,” Proc. 10th IEEE Int'l Conf. Computer Vision, 2005.
[20] A. Zomet and S. Peleg, “Multi-Sensor Super Resolution,” Proc. IEEE Workshop Applications of Computer Vision, 2002.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool