Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1234-1247
C. Nieuwenhuis , Fac. of Comput. Sci., Tech. Univ. of Munich, Garching, Germany
D. Cremers , Fac. of Comput. Sci., Tech. Univ. of Munich, Garching, Germany
ABSTRACT
We propose a method for interactive multilabel segmentation which explicitly takes into account the spatial variation of color distributions. To this end, we estimate a joint distribution over color and spatial location using a generalized Parzen density estimator applied to each user scribble. In this way, we obtain a likelihood for observing certain color values at a spatial coordinate. This likelihood is then incorporated in a Bayesian MAP estimation approach to multiregion segmentation which in turn is optimized using recently developed convex relaxation techniques. These guarantee global optimality for the two-region case (foreground/background) and solutions of bounded optimality for the multiregion case. We show results on the GrabCut benchmark, the recently published Graz benchmark, and on the Berkeley segmentation database which exceed previous approaches such as GrabCut [32], the Random Walker [15], Santner's approach [35], TV-Seg [39], and interactive graph cuts [4] in accuracy. Our results demonstrate that taking into account the spatial variation of color models leads to drastic improvements for interactive image segmentation.
INDEX TERMS
Image color analysis, Image segmentation, Joints, Motion segmentation, Kernel, Probability distribution, Bayesian methods,convex optimization, Image segmentation, spatially varying, color distribution
CITATION
C. Nieuwenhuis, D. Cremers, "Spatially Varying Color Distributions for Interactive Multilabel Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1234-1247, May 2013, doi:10.1109/TPAMI.2012.183
REFERENCES
 [1] H. Akaike, "An Approximation to the Density Function," Ann. Inst. of Statistical Math., vol. 6, pp. 127-132, 1954. [2] X. Bai and G. Sapiro, "A Geodesic Framework for Fast Interactive Image and Video Segmentation and Matting," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007. [3] A. Blake, C. Rother, M. Brown, P. Perez, and P. Torr, "Interactive Image Segmentation Using an Adaptive GMMRF Model," Proc. European Conf. Computer Vision, 2004. [4] Y. Boykov and M. Jolly, "Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images," Proc. Eighth IEEE Int'l Conf. Computer Vision, 2001. [5] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001. [6] T. Brox and D. Cremers, "On Local Region Models and a Statistical Interpretation of the Piecewise Smooth Mumford-Shah Functional," Int'l J. Computer Vision, vol. 84, pp. 184-193, 2009. [7] A. Chambolle, D. Cremers, and T. Pock, "A Convex Approach for Computing Minimal Partitions," Technical Report TR-2008-05, Dept. of Computer Science, Univ. of Bonn, Germany, 2008. [8] T. Chan, S. Esedo$\bar{\rm g}$ lu, and M. Nikolova, "Algorithms for Finding Global Minimizers of Image Segmentation and Denoising Models," SIAM J. Applied Math., vol. 66, no. 5, pp. 1632-1648, 2006. [9] Y.-Y. Chuang, B. Curless, D.H. Salesin, and R. Szeliski, "A Bayesian Approach to Digital Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 264-271, 2001. [10] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach to Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002. [11] D. Cremers, M. Rousson, and R. Deriche, "A Review of Statistical Approaches to Level Set Segmentation: Integrating Color, Texture, Motion and Shape," Int'l J. Computer Vision, vol. 72, no. 2, pp. 195-215, Apr. 2007. [12] O. Duchenne and J.-Y. Audibert, "Fast Interactive Segmentation Using Color and Textural Information," technical report, Certis, Paris Tech., 2006. [13] H. Federer, Geometric Measure Theory. Springer, 1996. [14] E.S.L. Gastal and M.M. Oliveira, "Shared Sampling for Real-Time Alpha Matting," Eurographics Computer Graphics Forum, vol. 29, no. 2, pp. 575-584, 2010. [15] L. Grady, "Random Walks for Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1768-1783, Nov. 2006. [16] D.M. Greig, B.T. Porteous, and A.H. Seheult, "Exact Maximum A Posteriori Estimation for Binary Images," J. Royal Statistical Soc. Ser. B., vol. 51, no. 2, pp. 271-279, 1989. [17] K. He, C. Rhemann, C. Rother, X. Tang, and J. Sun, "A Global Sampling Method for Alpha Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2049-2056, 2011. [18] J. Lellmann, F. Becker, and C. Schnörr, "Convex Optimization for Multi-Class Image Labeling with a Novel Family of Total Variation Based Regularizers," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009. [19] J. Lellmann, J. Kappes, J. Yuan, F. Becker, and C. Schnörr, "Convex Multiclass Image Labeling by Simplex-Constrained Total Variation," technical report, HCI, IWR, Univ. of Heidelberg, 2008. [20] P. Kohli, L. Ladicky, C. Russell, and P. Torr, "Graph Cut Based Inference with Co-Occurrence Statistics," Proc. 11th European Conf. Computer Vision, 2010. [21] K. McGuinness and N. O'Connor, "A Comparative Evaluation of Interactive Segmentation Algorithms," Pattern Recognition, vol. 43, no. 1, pp. 434-444, 2010. [22] C. Michelot, "A Finite Algorithm for Finding the Projection of a Point Onto the Canonical Simplex of $r^n$ ," J. Optimization Theory and Applications, vol. 50, no. 1, pp. 195-200, 1986. [23] D. Mumford and J. Shah, "Optimal Approximation by Piecewise Smooth Functions and Associated Variational Problems," Comm. Pure Applied Math., vol. 42, pp. 577-685, 1989. [24] C. Nieuwenhuis, B. Berkels, M. Rumpf, and D. Cremers, "Interactive Motion Segmentation," Proc. 32nd DAGM Conf. Pattern Recognition, vol. 6376, pp. 483-492, 2010. [25] C. Nieuwenhuis, E. Töppe, and D. Cremers, "Space-Varying Color Distributions for Interactive Multiregion Segmentation: Discrete versus Continuous Approaches," Proc. Eighth Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2011. [26] T. Pock and A. Chambolle, "Diagonal Preconditioning for First Order Primal-Dual Algorithms in Convex Optimization," Proc. 13th IEEE Int'l Conf. Computer Vision, 2011. [27] T. Pock, A. Chambolle, H. Bischof, and D. Cremers, "A Convex Relaxation Approach for Computing Minimal Partitions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009. [28] T. Pock, D. Cremers, H. Bischof, and A. Chambolle, "An Algorithm for Minimizing the Piecewise Smooth Mumford-Shah Functional," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009. [29] R.B. Potts, "Some Generalized Order-Disorder Transformations," Proc. Cambridge Philosophical Soc., vo. 48, pp. 106-109, 1952. [30] C. Rhemann, C. Rother, and M. Gelautz, "Improving Color Modeling for Alpha Matting," Proc. British Machine Vision Conf., 2008. [31] F. Rosenblatt, "Remarks on Some Nonparametric Estimates of a Density Function," Annals of Math. Statistics, vol. 27, pp. 832-837, 1956. [32] C. Rother, V. Kolmogorov, and A. Blake, "Grab-Cut: Interactive Foreground Segmentation Using Iterated Graph Cuts," ACM Trans. Graphics, vol. 23, no. 3, pp. 309-314, 2004. [33] L. Rudin, S. Osher, and E. Fatemi, "Nonlinear Total Variation Based Noise Removal Algorithms," Physica D, vol. 60, pp. 259-268, 1992. [34] J. Santner, "Interactive Multi-Label Segmentation," PhD thesis, Univ. of Graz, 2010. [35] J. Santner, T. Pock, and H. Bischof, "Interactive Multi-Label Segmentation," Proc. Asian Conf. Computer Vision, 2010. [36] B.W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1992. [37] Y. Tai, J. Jia, and C. Tang, "Soft Color Segmentation and Its Applications," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1520-1537, Sept. 2007. [38] M. Taron, N. Paragios, and M.-P. Jolly, "Border Detection on Short Axis Echocardiographic Views Using an Ellipse Driven Region-Based Framework," Medical Image Computing and Computer-Assisted Intervention, vol. 3216, pp. 443-450, 2004. [39] M. Unger, T. Pock, W. Trobin, D. Cremers, and H. Bischof, "TVSeg—Interactive Total Variation Based Image Segmentation," Proc. British Machine Vision Conf., 2008. [40] J. Wang and M. Cohen, "Optimized Color Sampling for Robust Matting," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007. [41] J. Yuan and Y. Boykov, "TV-Based Image Segmentation with Label Cost Prior," Proc. British Machine Vision Conf., 2010. [42] C. Zach, D. Gallup, J.-M. Frahm, and M. Niethammer, "Fast Global Labeling for Real-Time Stereo Using Multiple Plane Sweeps," Proc. Vision, Modeling and Visualization Workshop, 2008.