The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.07 - July (2013 vol.35)
pp: 1690-1703
Tae Hoon Kim , Dept. of Electr. & Comput. Eng., Seoul Nat. Univ., Seoul, South Korea
Kyoung Mu Lee , Dept. of Electr. & Comput. Eng., Seoul Nat. Univ., Seoul, South Korea
Sang Uk Lee , Dept. of Electr. & Comput. Eng., Seoul Nat. Univ., Seoul, South Korea
ABSTRACT
Segmenting a single image into multiple coherent groups remains a challenging task in the field of computer vision. Particularly, spectral segmentation which uses the global information embedded in the spectrum of a given image's affinity matrix is a major trend in image segmentation. This paper focuses on the problem of efficiently learning a full range of pairwise affinities gained by integrating local grouping cues for spectral segmentation. We first construct a sparse multilayer graph whose nodes are both the pixels and the oversegmented regions obtained by an unsupervised segmentation algorithm. By applying the semi-supervised learning strategy to this graph, the intra and interlayer affinities between all pairs of nodes can be estimated without iteration. These pairwise affinities are then applied into the spectral segmentation algorithms. In this paper, two types of spectral segmentation algorithms are introduced: K-way segmentation and hierarchical segmentation. Our algorithms provide high-quality segmentations which preserve object details by directly incorporating the full-range connections. Moreover, since our full affinity matrix is defined by the inverse of a sparse matrix, its eigendecomposition can be efficiently computed. The experimental results on the BSDS and MSRC image databases demonstrate the superiority of our segmentation algorithms in terms of relevance and accuracy compared with existing popular methods.
INDEX TERMS
Image segmentation, Sparse matrices, Image color analysis, Image edge detection, Computational modeling, Nonhomogeneous media, Approximation methods,semi-supervised learning, Spectral segmentation, hierarchical segmentation, affinity estimation
CITATION
Tae Hoon Kim, Kyoung Mu Lee, Sang Uk Lee, "Learning Full Pairwise Affinities for Spectral Segmentation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 7, pp. 1690-1703, July 2013, doi:10.1109/TPAMI.2012.237
REFERENCES
[1] D. Comaniciu and P. Meer, "Mean Shift: A Robust Approach toward Feature Space Analysis," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[2] M.Á. Carreira-Perpiñán, "Fast Nonparametric Clustering with Gaussian Blurring Mean-Shift," Proc. Int'l Conf. Machine Learning, 2006.
[3] J. Puzicha, T. Hofmann, and J.M. Buhmann, "Non-Parametric Similarity Measures for Unsupervised Texture Segmentation and Image Retrieval," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[4] Y. Gdalyahu, D. Weinshall, and M. Werman, "Stochastic Image Segmentation by Typical Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[5] N. Shental, A. Zomet, T. Hertz, and Y. Weiss, "Learning and Inferring Image Segmentations with the GBP Typical Cut Algorithm," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[6] S. Sanjay-Gopal and T.J. Hebert, "Bayesian Pixel Classification Using Spatially Variant Finite Mixtures and the Generalized EM Algorithm," IEEE Trans. Image Processing, vol. 7, no. 7, pp. 1014-1028, July 1998.
[7] G. Sfikas, C. Nikou, and N. Galatsanos, "Edge Preserving Spatially Varying Mixtures for Image Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[8] H. Mobahi, S. Rao, A. Yang, S. Sastry, and Y. Ma, "Segmentation of Natural Images by Texture and Boundary Compression," Int'l J. Computer Vision, vol. 95, no. 1, pp. 86-98, 2011.
[9] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[10] P.F. Felzenszwalb and D.P. Huttenlocher, "Efficient Graph-Based Image Segmentation," Int'l J. Computer Vision, vol. 59, no. 2, pp. 167-181, 2004.
[11] R. Zabih and V. Kolmogorov, "Spatially Coherent Clustering Using Graph Cuts," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[12] S. Wang, T. Kubota, J.M. Siskind, and J. Wang, "Salient Closed Boundary Extraction with Ratio Contour," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 546-561, Apr. 2005.
[13] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "From Contours to Regions: An Empirical Evaluation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[14] T. Pock, A. Chambolle, D. Cremers, and H. Bischof, "A Convex Relaxation Approach for Computing Minimal Partitions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[15] M. Donoser, M. Urschler, M. Hirzer, and H. Bischof, "Saliency Driven Total Variation Segmentation," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[16] P. Kohli, L. Ladický, and P. Torr, "Robust Higher Order Potentials for Enforcing Label Consistency," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[17] T.H. Kim, K.M. Lee, and S.U. Lee, "Nonparametric Higher-Order Learning for Interactive Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[18] T. Cour, F. Bénézit, and J. Shi, "Spectral Segmentation with Multiscale Graph Decomposition," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[19] C. Fowlkes, D. Martin, and J. Malik, "Learning Affinity Functions for Image Segmentation: Combining Patch-Based and Gradient-Based Approaches," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[20] C. Fowlkes, S. Belongie, F. Chung, and J. Malik, "Spectral Grouping Using the Nyström Method," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 214-225, Feb. 2004.
[21] J. Wang, Y. Jia, X.-S. Hua, C. Zhang, and L. Quan, "Normalized Tree Partitioning for Image Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[22] P. Arbeláez and L. Cohen, "Constrained Image Segmentation from Hierarchical Boundaries," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[23] E. Sharon, A. Brandt, and R. Basri, "Fast Multiscale Image Segmentation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2000.
[24] S.X. Yu, "Segmenation Using Multiscale Cues," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[25] M. Maire, P. Arbelaez, C. Fowlkes, and J. Malik, "Using Contours to Detect and Localize Junctions in Natural Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[26] T. Leung and J. Malik, "Contour Continuity in Region-Based Image Segmentation," Proc. European Conf. Computer Vision, 1998.
[27] S. Brin and L. Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine," Proc. Int'l Conf. World Wide Web, 1998.
[28] D. Zhou, O. Bousquet, T.N. Lal, J. Weston, and B. Schölkopf, "Learning with Local and Global Consistency," Proc. Neural Information Processing Systems, 2003.
[29] J.-Y. Pan, H.-J. Yang, C. Faloutsos, and P. Duygulu, "Automatic Multimedia Cross-Modal Correlation Discovery," Proc. ACM SIGKDD 10th Int'l Conf. Knowledge Discovery and Data Mining, 2004.
[30] G.H. Golub and C.F.V. Loan, Matrix Computations. Johns Hopkins Univ. Press, 1996.
[31] Y. Weiss, "Segmentation Using Eigenvectors: A Unifying View," Proc. IEEE Int'l Conf. Computer Vision, 1999.
[32] S.X. Yu and J. Shi, "Multiclass Spectral Clustering," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[33] P. Arbelaez, M. Maire, C. Fowlkes, and J. Malik, "Contour Detection and Hierarchical Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 5, pp. 898-916, May 2011.
[34] L. Najman and M. Schmitt, "Geodesic Saliency of Watershed Contours and Hierarchical Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 12, pp. 1163-1173, Dec. 1996.
[35] P. Arbelaez, "Boundary Extraction in Natural Images Using Ultrametric Contour Maps," Proc. IEEE Workshop Perceptual Organization in Computer Vision, 2006.
[36] J. Canny, "A Computational Approach to Edge Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp. 679-698, Nov. 1986.
[37] Y. Boykov and G. Funka-Lea, "Graph Cuts and Efficient N-D Image Segmentation," Int'l J. Computer Vision , vol. 70, no. 2, pp. 109-131, 2006.
[38] S. Belongie and J. Malik, "Finding Boundaries in Natural Images: A New Method Using Point Descriptors and Area Completion," Proc. European Conf. Computer Vision, 1998.
[39] T. Malisiewicz and A.A. Efros, "Improving Spatial Support for Objects via Multiple Segmentations," Proc. British Machine Vision Conf., 2007.
[40] W.M. Rand, "Objective Criteria for the Evaluation of Clustering Methods," J. Am. Statistical Assoc., vol. 66, no. 336, pp. 846-850, 1971.
[41] R. Unnikrishnan, C. Pantofaru, and M. Hebert, "Toward Objective Evaluation of Image Segmentation Algorithms," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 929-944, June 2007.
[42] M. Meilă, "Comparing Clusterings: An Axiomatic View," Proc. Int'l Conf. Machine Learning, 2005.
[43] E. Sharon, M. Galun, D. Sharon, A. Brandt, and R. Basri, "Hierarchy and Adaptivity in Segmenting Visual Scenes," Nature, vol. 442, no. 7104, pp. 810-813, 2006.
[44] D. Martin, C. Fowlkes, and J. Malik, "Learning to Detect Natural Image Boundaries Using Local Brightness, Color and Texture Cues," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530-549, May 2004.
36 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool