CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2010 vol.32 Issue No.06 - June

Subscribe

Issue No.06 - June (2010 vol.32)

pp: 988-995

Anil Raj , Columbia University, New York

Chris H. Wiggins , Columbia University, New York

ABSTRACT

Min-cut clustering, based on minimizing one of two heuristic cost functions proposed by Shi and Malik nearly a decade ago, has spawned tremendous research, both analytic and algorithmic, in the graph partitioning and image segmentation communities over the last decade. It is, however, unclear if these heuristics can be derived from a more general principle, facilitating generalization to new problem settings. Motivated by an existing graph partitioning framework, we derive relationships between optimizing relevance information, as defined in the Information Bottleneck method, and the regularized cut in a K-partitioned graph. For fast-mixing graphs, we show that the cost functions introduced by Shi and Malik can be well approximated as the rate of loss of predictive information about the location of random walkers on the graph. For graphs drawn from a generative model designed to describe community structure, the optimal information-theoretic partition and the optimal min-cut partition are shown to be the same with high probability.

INDEX TERMS

Graphs, clustering, information theory, min-cut, Information Bottleneck, graph diffusion.

CITATION

Anil Raj, Chris H. Wiggins, "An Information-Theoretic Derivation of Min-Cut-Based Clustering",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.32, no. 6, pp. 988-995, June 2010, doi:10.1109/TPAMI.2009.124REFERENCES

- [1] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.- [2] M. Meila and J. Shi, "A Random Walks View of Spectral Segmentation,"
Proc. Int'l Conf. AI and Statistics, 2001.- [3] S. Lafon and A.B. Lee, "Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction, Graph Partitioning, and Data Set Parameterization,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1393-1403, Sept. 2006.- [4] B. Nadler, S. Lafon, R.R. Coifman, and I.G. Kevrekidis, "Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators,"
Advances in Neural Information Processing Systems 18, pp. 955-962, MIT Press, 2005.- [5] N. Tishby, F.C. Pereira, and W. Bialek, "The Information Bottleneck Method,"
arXiv preprint physics, http://arxiv.org/abs/physics0004057, Jan. 2000.- [6] N. Tishby and N. Slonim, "Data Clustering by Markovian Relaxation and the Information Bottleneck Method,"
Advances in Neural Information Processing Systems, MIT Press, Jan. 2001.- [7] C. Shannon, "A Mathematical Theory of Communication,"
ACM SIGMOBILE Mobile Computing and Comm. Rev., vol. 5, no. 1, pp. 3-55, Jan. 2001.- [8] N. Slonim, "The Information Bottleneck: Theory and Applications," PhD dissertation, The Hebrew Univ. of Jerusalem, 2002.
- [9] E. Ziv, M. Middendorf, and C.H. Wiggins, "Information-Theoretic Approach to Network Modularity,"
Physical Rev. E, vol. 71, p. 046117, Jan. 2005.- [10] U. von Luxburg, "A Tutorial on Spectral Clustering," arXiv, vol. cs.DS, http://arxiv.org/abs0711.0189v1, Nov. 2007.
- [11] M. Fiedler, "Algebraic Connectivity of Graphs,"
Czechoslovak Math. J., vol. 23, no. 98, pp. 298-305, 1973.- [12] D. Wagner and F. Wagner, "Between Min Cut and Graph Bisection,"
Proc. 18th Int'l Symp. Math. Foundations of Computer Science, pp. 744-750. 1993.- [13] F.R. Chung,
Spectral Graph Theory. Am. Math. Soc., 1997.- [14] P.W. Holland and S. Leinhardt, "Local Structure in Social Networks,"
Sociological Methodology, pp. 1-45, Jan. 1976, http://www.cs.cmu.edu/Web/Groups/NIPX/00papers-pu-on-web TishbySlonim.ps.gz. - [15] L. Danon, A. Diaz-Guilera, J. Duch, and A. Arenas, "Comparing Community Structure Identification,"
J. Statistical Mechanics: Theory and Experiment, Jan. 2005. |