This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Self-Validated Labeling of Markov Random Fields for Image Segmentation
October 2010 (vol. 32 no. 10)
pp. 1871-1887
Wei Feng, The Chinese University of Hong Kong and City University of Hong Kong, Hong Kong
Jiaya Jia, The Chinese University of Hong Kong, Hong Kong
Zhi-Qiang Liu, City University of Hong Kong, Hong Kong
This paper addresses the problem of self-validated labeling of Markov random fields (MRFs), namely to optimize an MRF with unknown number of labels. We present graduated graph cuts (GGC), a new technique that extends the binary s-t graph cut for self-validated labeling. Specifically, we use the split-and-merge strategy to decompose the complex problem to a series of tractable subproblems. In terms of Gibbs energy minimization, a suboptimal labeling is gradually obtained based upon a set of cluster-level operations. By using different optimization structures, we propose three practical algorithms: tree-structured graph cuts (TSGC), net-structured graph cuts (NSGC), and hierarchical graph cuts (HGC). In contrast to previous methods, the proposed algorithms can automatically determine the number of labels, properly balance the labeling accuracy, spatial coherence, and the labeling cost (i.e., the number of labels), and are computationally efficient, independent to initialization, and able to converge to good local minima of the objective energy function. We apply the proposed algorithms to natural image segmentation. Experimental results show that our algorithms produce generally feasible segmentations for benchmark data sets, and outperform alternative methods in terms of robustness to noise, speed, and preservation of soft boundaries.

[1] S.X. Yu, R. Gross, and J. Shi, "Concurrent Object Segmentation and Recognition with Graph Partitioning," Proc. Conf. Neural Information Processing Systems, 2002.
[2] Z. Tu, X. Chen, A. Yuille, and S.-C. Zhu, "Image Parsing: Unifying Segmentation, Detection, and Object Recognition," Int'l J. Computer Vision, vol. 63, no. 2, pp. 113-140, 2005.
[3] C. Liu, J. Yuen, and A. Torralba, "Nonparametric Scene Parsing: Label Transfer via Dense Scene Alignment," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[4] W. Feng and Z.-Q. Liu, "Region-Level Image Authentication Using Bayesian Structural Content Abstraction," IEEE Trans. Image Processing, vol. 17, no. 12, pp. 2413-2424, Dec. 2008.
[5] Z. Wu and L. Richard, "An Optimal Graph Theoretic Approach to Data Clustering: Theory and Its Application to Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 11, pp. 1101-1113, Nov. 1993.
[6] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[7] A.K. Sinop and L. Grady, "A Seeded Image Segmentation Framework Unifying Graph Cuts and Random Walker Which Yields a New Algorithm," Proc. IEEE 11th Int'l Conf. Computer Vision, 2007.
[8] D. Singaraju, L. Grady, and R. Vidal, "P-Brush: Continous Valued MRFs with Normed Pairwise Distributions for Image Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[9] L. Grady, "Random Walks for Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1768-1783, Nov. 2006.
[10] P. Kohli, A. Shekhovtsov, C. Rother, V. Kolmogorov, and P. Torr, "On Partial Optimality in Multi-Label MRFs," Proc. Ann. Int'l Conf. Machine Learning, 2008.
[11] A. Opelt, A. Pinz, M. Fussenegger, and P. Auer, "Generic Object Recognition with Boosting," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 416-431, Mar. 2006.
[12] M. Halkidi, Y. Batistakis, and M. Vazirgiannis, "On Clustering Validation Techniques," J. Intelligent Information Systems, vol. 17, nos. 2-3, pp. 107-145, Dec. 2001.
[13] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Graph-Based Image Segmentation," Int'l J. Computer Vision, vol. 59, no. 2, pp. 167-181, 2004.
[14] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach Toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 1-18, May 2002.
[15] C. D'Elia, G. Poggi, and G. Scarpa, "A Tree-Structured Markov Random Field Model for Bayesian Image Segmentation," IEEE Trans. Image Processing, vol. 12, no. 10, pp. 1250-1264, Oct. 2003.
[16] Z. Tu and S.-C. Zhu, "Image Segmentation by Data-Driven Markov Chain Monte Carlo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 657-673, May 2002.
[17] "Segmentation by Split-and-Merge Techniques," http://iris.usc. edu/Vision-Notes/bibliography segment347.html, 2010.
[18] L. Grady and E. Schwartz, "Isoperimetric Graph Partitioning for Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 3, pp. 469-475, Mar. 2006.
[19] E. Sharon, M. Galun, D. Sharon, R. Basri, and A. Brandt, "Hierarchy and Adaptivity in Segmenting Visual Scenes," Nature, vol. 442, no. 7104, pp. 719-846, 2006.
[20] D. Greig, B. Porteous, and A. Seheult, "Exact Maximus A Posteriori Estimation for Binary Images," J. Royal Statistical Soc., Series B, vol. 51, no. 2, pp. 271-279, 1989.
[21] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[22] Y. Boykov and V. Kolmogorov, "An Experimental Comparison of Min-Cut/Max-Flow Algorithms for Energy Minimization in Vision," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1124-1137, Sept. 2004.
[23] Y. Boykov and G. Funka-Lea, "Graph Cuts and Efficient N-D Image Segmentation," Int'l J. Computer Vision, vol. 70, no. 2, pp. 109-131, 2006.
[24] V. Kolmogorov and R. Zabih, "What Energy Functions Can Be Minimized via Graph Cuts?" IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 147-159, Feb. 2004.
[25] V. Kolmogorov and C. Rother, "Minimizing Nonsubmodular Functions with Graph Cuts—A Review," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 7, pp. 1274-1279, July 2007.
[26] S. Geman and D. Geman, "Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, Nov. 1984.
[27] D. Melas and S. Wilson, "Double Markov Random Fields and Bayesian Image Segmentation," IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 357-365, Feb. 2002.
[28] J. Marroquin, E. Santana, and S. Botello, "Hidden Markov Measure Field Models for Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 11, pp. 1380-1387, Nov. 2003.
[29] W. Feng and Z.-Q. Liu, "Self-Validated and Spatially Coherent Clustering with Net-Structured MRF and Graph Cuts," Proc. Int'l Conf' Pattern Recognition, vol. 4, pp. 37-40, 2006.
[30] J. Malik, S. Belongie, T. Leung, and J. Shi, "Contour and Texture Analysis for Image Segmentation," Intl'l J. Computer Vision, vol. 43, no. 1, pp. 7-27, 2001.
[31] S.X. Yu and J. Shi, "Segmentation Given Partial Grouping Constraints," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 173-183, Jan. 2004.
[32] S.X. Yu, "Segmentation Using Multiscale Cues," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[33] T. Cour, F. Bénézit, and J. Shi, "Spectral Segmentation with Multiscale Graph Decomposition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 1124-1131, 2005.
[34] J. Wang and M.F. Cohen, "An Iterative Optimization Approach for Unified Image Segmentation and Matting," Proc. IEEE Int'l Conf. Computer Vision, 2005.
[35] J. Wang, M. Agrawala, and M.F. Cohen, "Soft Scissors: An Interactive Tool for Realtime High Quality Matting," Proc. ACM SIGGRAPH, 2007.
[36] Y. Boykov and M.-P. Jolly, "Interactive Graph Cuts for Optimal Boundary & Region Segmentation of Objects in N-D Images," Proc. IEEE Int'l Conf. Computer Vision, pp. 105-112, 2001.
[37] Y. Li, J. Sun, C.-K. Tang, and H.-Y. Shum, "Lazy Snapping," Proc. ACM SIGGRAPH, pp. 303-308, 2004.
[38] C. Rother, V. Kolmogorov, and A. Blake, "Grabcut—Interactive Foreground Extraction Using Iterated Graph Cuts," Proc. ACM SIGGRAPH, pp. 309-314, 2004.
[39] C. Rother, V. Kolmogorov, V. Lempitsky, and M. Szummer, "Optimizing Binary MRFs Via Extended Roof Duality," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[40] "METIS Package," http://glaros.dtc.umn.edu/gkhome/viewsmetis /, 2010.
[41] P. Cheeseman and J. Stutz, "Bayesian Classification (AutoClass): Theory and Results," Advances in Knowledge Discovery and Data Mining, pp. 153-180, Am. Assoc. for Artificial Intelligence, 1996.
[42] Y. Li and D. Huttenlocher, "Sparse Long-Range Random Field and Its Application to Image Denoising," Proc. European Conf. on Computer Vision, 2008.
[43] A. Elmoataz, O. Lezoray, and S. Bougleux, "Nonlocal Discrete Regularization on Weighted Graphs: A Framework for Image and Manifold Processing," IEEE Trans. Image Processing, vol. 17, no. 7, pp. 1047-1060, July 2008.
[44] U. Braga-Neto and J. Goutsias, "Object-Based Image Analysis Using Multiscale Connectivity," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 892-907, June 2005.
[45] E. Dahlhaus, D. Johnson, C. Papadimitriou, P. Seymour, and M. Yannakakis, "The Complexity of Multiway Cuts," ACM Symp. Theory of Computing, pp. 241-251, 1992.
[46] J. Evans and E. Minieka, Optimization Algorithms for Networks and Graphs, second ed. Marcel Dekker, 1992.
[47] C. Elkan, "Using the Triangle Inequality to Accelerate K-Means," Proc. Ann. Int'l Conf. Machine Learning, 2003.
[48] R. Morris, X. Descombes, and J. Zerubia, "The Ising/Potts Model Is Not Well Suited to Segmentation Tasks," Proc. Digital Signal Processing Workshop, pp. 263-266, 1996.
[49] D. Martin, C. Fowlkes, and J. Malik, "A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 416-423, 2001.
[50] S. Alpert, M. Galun, R. Basri, and A. Brandt, "Image Segmentation by Probabilistic Bottom-Up Aggregation and Cue Integration," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[51] C. Rother, P. Kohli, W. Feng, and J. Jia, "Minimizing Sparse Higher Order Energy Functions of Discrete Variables," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[52] P. Kohli, L. Ladicky, and P. Torr, "Robust Higher Order Potentials for Enforcing Label Consistency," Int'l J. Computer Vision, vol. 82, pp. 302-324, 2009.

Index Terms:
Self-validated labeling, Markov random fields (MRFs), graduated graph cuts, image segmentation, split-and-merge.
Citation:
Wei Feng, Jiaya Jia, Zhi-Qiang Liu, "Self-Validated Labeling of Markov Random Fields for Image Segmentation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 32, no. 10, pp. 1871-1887, Oct. 2010, doi:10.1109/TPAMI.2010.24
Usage of this product signifies your acceptance of the Terms of Use.