CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2010 vol.32 Issue No.06 - June

Subscribe

Issue No.06 - June (2010 vol.32)

pp: 1112-1126

Amin Mantrach , IRIDIA—CoDE—Université Libre de Bruxelles, Brussels

Luh Yen , ISYS/LSM, Université Catholique de Louvain, Louvain-la-Neuve

Jerome Callut , ISYS/LSM, Université Catholique de Louvain, Louvain-la-Neuve

Kevin Francoisse , ISYS/LSM, Université Catholique de Louvain, Louvain-la-Neuve

Masashi Shimbo , Nara Institute of Technology and Science, Takayama

Marco Saerens , ISYS/LSM, Université Catholique de Louvain, Louvain-la-Neuve

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2009.78

ABSTRACT

This work introduces a link-based covariance measure between the nodes of a weighted directed graph, where a cost is associated with each arc. To this end, a probability distribution on the (usually infinite) countable set of paths through the graph is defined by minimizing the total expected cost between all pairs of nodes while fixing the total relative entropy spread in the graph. This results in a Boltzmann distribution on the set of paths such that long (high-cost) paths occur with a low probability while short (low-cost) paths occur with a high probability. The sum-over-paths (SoP) covariance measure between nodes is then defined according to this probability distribution: two nodes are considered as highly correlated if they often co-occur together on the same—preferably short—paths. The resulting covariance matrix between nodes (say n nodes in total) is a Gram matrix and therefore defines a valid kernel on the graph. It is obtained by inverting an n\times n matrix depending on the costs assigned to the arcs. In the same spirit, a betweenness score is also defined, measuring the expected number of times a node occurs on a path. The proposed measures could be used for various graph mining tasks such as computing betweenness centrality, semi-supervised classification of nodes, visualization, etc., as shown in Section 7.

INDEX TERMS

Graph mining, kernel on a graph, shortest path, correlation measure, betweenness measure, resistance distance, commute time distance, biased random walk, semi-supervised classification.

CITATION

Amin Mantrach, Luh Yen, Jerome Callut, Kevin Francoisse, Masashi Shimbo, Marco Saerens, "The Sum-over-Paths Covariance Kernel: A Novel Covariance Measure between Nodes of a Directed Graph",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.32, no. 6, pp. 1112-1126, June 2010, doi:10.1109/TPAMI.2009.78REFERENCES

- [1] M. Saerens, Y. Achbany, F. Fouss, and L. Yen, "Randomized Shortest-Path Problems: Two Related Models,"
Neural Computation, vol. 21, no. 8, pp. 2363-2404, 2009.- [2] L. Yen, A. Mantrach, M. Shimbo, and M. Saerens, "A Family of Dissimilarity Measures between Nodes Generalizing Both the Shortest-Path and the Commute-Time Distances,"
Proc. ACM SIGKDD, pp. 785-793, 2008.- [3] T. Akamatsu, "Cyclic Flows, Markov Process and Stochastic Traffic Assignment,"
Transportation Research B, vol. 30, no. 5, pp. 369-386, 1996.- [4] H. Small, "Co-Citation in the Scientific Literature: A New Measure of the Relationship between Two Documents,"
J. Am. Soc. Information Science, vol. 24, no. 4, pp. 265-269, 1973.- [5] M.M. Kessler, "Bibliographic Coupling between Scientific Papers,"
Am. Documentation, vol. 14, no. 1, pp. 10-25, 1963.- [6] S. Wasserman and K. Faust,
Social Network Analysis: Methods and Applications. Cambridge Univ. Press, 1994.- [7] J.-C. Delvenne, S. Yaliraki, and M. Barahona, "Stability of Graph Communities across Time Scales," arXiv:0812.1770, 2008.
- [8] M. Newman and M. Girvan, "Finding and Evaluating Community Structure in Networks,"
Physical Rev. E, vol. 69, p. 026113, 2004.- [9] D.J. Klein and M. Randic, "Resistance Distance,"
J. Math. Chemistry, vol. 12, pp. 81-95, 1993.- [10] R.B. Bapat, "Resistance Distance in Graphs,"
The Math. Student, vol. 68, pp. 87-98, 1999.- [11] F. Gobel and A.A. Jagers, "Random Walks on Graphs,"
Stochastic Processes and Their Applications, vol. 2, pp. 311-336, 1974.- [12] L. Lovasz, "Random Walks on Graphs: A Survey,"
Combinatorics: Paul Erdos Is Eighty, vol. 2, pp. 353-397, 1996.- [13] A.K. Chandra, P. Raghavan, W.L. Ruzzo, R. Smolensky, and P. Tiwari, "The Electrical Resistance of a Graph Captures Its Commute and Cover Times,"
Proc. Ann. ACM Symp. Theory of Computing, pp. 574-586, 1989.- [14] M. Saerens, F. Fouss, L. Yen, and P. Dupont, "The Principal Components Analysis of a Graph, and Its Relationships to Spectral Clustering,"
Proc. 15th European Conf. Machine Learning, pp. 371-383, 2004.- [15] P. Chebotarev and E. Shamis, "The Matrix-Forest Theorem and Measuring Relations in Small Social Groups,"
Automation and Remote Control, vol. 58, no. 9, pp. 1505-1514, 1997.- [16] P. Chebotarev and E. Shamis, "On Proximity Measures for Graph Vertices,"
Automation and Remote Control, vol. 59, no. 10, pp. 1443-1459, 1998.- [17] F. Fouss, A. Pirotte, J.-M. Renders, and M. Saerens, "Random-Walk Computation of Similarities between Nodes of a Graph, with Application to Collaborative Recommendation,"
IEEE Trans. Knowledge and Data Eng., vol. 19, no. 3, pp. 355-369, Mar. 2007.- [18] T. Ito, M. Shimbo, T. Kudo, and Y. Matsumoto, "Application of Kernels to Link Analysis,"
Proc. ACM SIGKDD, pp. 586-592, 2005.- [19] M. Shimbo and T. Ito, "Kernels as Link Analysis Measures,"
Mining Graph Data, D. Cook and L. Holder, eds., pp. 283-310, John Wiley & Sons, 2006.- [20] J. Kandola, N. Cristianini, and J. Shawe-Taylor, "Learning Semantic Similarity,"
Proc. Advances in Neural Information Processing Systems 15, pp. 657-664, 2002.- [21] D. Zhou and B. Schölkopf, "Learning from Labeled and Unlabeled Data Using Random Walks,"
Proc. 26th DAGM Symp., C.E. Rasmussen, H.H. Bülthoff, M.A. Giese, and B. Schölkopf, eds., pp. 237-244, 2004.- [22] D. Zhou, J. Huang, and B. Schölkopf, "Learning from Labeled and Unlabeled Data on a Directed Graph,"
Proc. 22nd Int'l Conf. Machine Learning, pp. 1041-1048, 2005.- [23] F.R. Chung, "Laplacians and the Cheeger Inequality for Directed Graphs,"
Annals of Combinatorics, vol. 9, pp. 1-19, 2005.- [24] T. Chen, Q. Yang, and X. Tang, "Directed Graph Embedding,"
Proc. Int'l Joint Conf. Artificial Intelligence, pp. 2707-2712, 2007.- [25] J. Shawe-Taylor and N. Cristianini,
Kernel Methods for Pattern Analysis. Cambridge Univ. Press, 2004.- [26] R.I. Kondor and J. Lafferty, "Diffusion Kernels on Graphs and Other Discrete Structures,"
Proc. 19th Int'l Conf. Machine Learning, pp. 315-322, 2002.- [27] P.G. Doyle and J.L. Snell,
Random Walks and Electric Networks. The Math. Assoc. of Am., 1984.- [28] D. Harel and Y. Koren, "On Clustering Using Random Walks,"
Proc. Conf. Foundations of Software Technology and Theoretical Computer Science, pp. 18-41, 2001.- [29] S. White and P. Smyth, "Algorithms for Estimating Relative Importance in Networks,"
Proc. ACM SIGKDD, pp. 266-275, 2003.- [30] S. Brin and L. Page, "The Anatomy of a Large-Scale Hypertextual Web Search Engine,"
Computer Networks and ISDN Systems, vol. 30, nos. 1-7, pp. 107-117, 1998.- [31] L. Page, S. Brin, R. Motwani, and T. Winograd, "The Pagerank Citation Ranking: Bringing Order to the Web," Technical Report 1999-0120, Computer Science Dept., Stanford Univ., 1999.
- [32] D. Liben-Nowell and J. Kleinberg, "The Link-Prediction Problem for Social Networks,"
J. Am. Soc. Information Science and Technology, vol. 58, no. 7, pp. 1019-1031, 2007.- [33] A.J. Smola and R. Kondor, "Kernels and Regularization on Graphs,"
Proc. Conf. Learning Theory, M. Warmuth and B. Schölkopf, eds., pp. 144-158, 2003.- [34] X. Zhu, J. Kandola, J. Lafferty, and Z. Ghahramani, "Graph Kernels by Spectral Transforms,"
Semi-Supervised Learning, O. Chapelle, B. Scholkopf, and A. Zien, eds., pp. 277-291, MIT Press, 2006.- [35] C. Palmer and C. Faloutsos, "Electricity Based External Similarity of Categorical Attributes,"
Proc. Seventh Pacific-Asia Conf. Knowledge Discovery and Data Mining, pp. 486-500, 2003.- [36] P. Sarkar and A. Moore, "A Tractable Approach to Finding Closest Truncated-Commute-Time Neighbors in Large Graphs,"
Proc. 23rd Conf. Uncertainty in Artificial Intelligence, 2007.- [37] H. Qiu and E.R. Hancock, "Image Segmentation Using Commute Times,"
Proc. 16th British Machine Vision Conf., pp. 929-938, 2005.- [38] H. Qiu and E.R. Hancock, "Clustering and Embedding Using Commute Times,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1873-1890, Nov. 2007.- [39] J. Ham, D. Lee, S. Mika, and B. Scholkopf, "A Kernel View of the Dimensionality Reduction of Manifolds,"
Proc. 21st Int'l Conf. Machine Learning, pp. 369-376, 2004.- [40] L. Yen, D. Vanvyve, F. Wouters, F. Fouss, M. Verleysen, and M. Saerens, "Clustering Using a Random Walk-Based Distance Measure,"
Proc. 13th European Symp. Artificial Neural Networks, pp. 317-324, 2005.- [41] M. Brand, "A Random Walks Perspective on Maximizing Satisfaction and Profit,"
Proc. 2005 SIAM Int'l Conf. Data Mining, 2005.- [42] M. Fiedler, "A Property of Eigenvectors of Nonnegative Symmetric Matrices and Its Applications to Graph Theory,"
Czechoslovak Math. J., vol. 25, no. 100, pp. 619-633, 1975.- [43] B. Mohar, "Laplace Eigenvalues of Graphs—A Survey,"
Discrete Math., vol. 109, pp. 171-183, 1992.- [44] T. Chan, P. Ciarlet, and W. Szeto, "On the Optimality of the Median Cut Spectral Bisection Graph Partitioning Method,"
SIAM J. Scientific Computing, vol. 18, no. 3, pp. 943-948, 1997.- [45] A. Pothen, H.D. Simon, and K.-P. Liou, "Partitioning Sparse Matrices with Eigenvectors of Graphs,"
SIAM J. Matrix Analysis and Applications, vol. 11, no. 3, pp. 430-452, 1990.- [46] L. Donetti and M. Munoz, "Detecting Network Communities: A New Systematic and Efficient Algorithm,"
J. Statistical Mechanics: Theory and Experiment, vol. P10012, 2004.- [47] L. Yen, F. Fouss, C. Decaestecker, P. Francq, and M. Saerens, "Graph Nodes Clustering Based on the Commute-Time Kernel,"
Proc. 11th Pacific-Asia Conf. Knowledge Discovery and Data Mining, pp. 1037-1045, 2007.- [48] L. Yen, F. Fouss, C. Decaestecker, P. Francq, and M. Saerens, "Graph Nodes Clustering with the Sigmoid Commute-Time Kernel: A Comparative Study,"
Data & Knowledge Eng., vol. 68, pp. 338-361, 2009.- [49] P. Chebotarev, "A New Family of Graph Distances," arxiv.org as manuscript arXiv:0810.2717v2, 2008.
- [50] H. Zhou, "Distance, Dissimilarity Index, and Network Community Structure,"
Physical Rev. E, vol. 67, p. 061901, 2003.- [51] H. Zhou, "Network Landscape from a Brownian Particle Perspective,"
Physical Rev. E, vol. 67, p. 041908, 2003.- [52] H. Tong, Y. Koren, and C. Faloutsos, "Fast Direction-Aware Proximity for Graph Mining,"
Proc. ACM SIGKDD, pp. 747-756, 2007.- [53] Y. Koren, S. North, and C. Volinsky, "Measuring and Extracting Proximity in Networks,"
Proc. ACM SIGKDD, pp. 245-255, 2006.- [54] Y. Koren, S. North, and C. Volinsky, "Measuring and Extracting Proximity Graphs in Networks,"
ACM Trans. Knowledge Discovery in Data, vol. 1, no. 3, pp. 12:1-12:30, 2007.- [55] B. Nadler, S. Lafon, R. Coifman, and I. Kevrekidis, "Diffusion Maps, Spectral Clustering and Eigenfunctions of Fokker-Planck Operators,"
Proc. Advances in Neural Information Processing Systems 18, pp. 955-962, 2005.- [56] B. Nadler, S. Lafon, R. Coifman, and I. Kevrekidis, "Diffusion Maps, Spectral Clustering and Reaction Coordinate of Dynamical Systems,"
Applied and Computational Harmonic Analysis, vol. 21, pp. 113-127, 2006.- [57] P. Pons and M. Latapy, "Computing Communities in Large Networks Using Random Walks,"
Proc. Int'l Symp. Computer and Information Sciences, pp. 284-293, 2005.- [58] P. Pons and M. Latapy, "Computing Communities in Large Networks Using Random Walks,"
J. Graph Algorithms and Applications, vol. 10, no. 2, pp. 191-218, 2006.- [59] F. Fouss, L. Yen, A. Pirotte, and M. Saerens, "An Experimental Investigation of Graph Kernels on a Collaborative Recommendation Task,"
Proc. Sixth Int'l Conf. Data Mining, pp. 863-868, 2006.- [60] S. Lafon and A.B. Lee, "Diffusion Maps and Coarse-Graining: A Unified Framework for Dimensionality Reduction, Graph Partitioning, and Data Set Parameterization,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1393-1403, Sept. 2006.- [61] M. Gori and A. Pucci, "A Random-Walk Based Scoring Algorithm with Application to Recommender Systems for Large-Scale E-Commerce,"
Proc. ACM SIGKDD, 2006.- [62] J.-Y. Pan, H.-J. Yang, C. Faloutsos, and P. Duygulu, "Automatic Multimedia Cross-Modal Correlation Discovery,"
Proc. ACM SIGKDD, pp. 653-658, 2004.- [63] H. Tong, C. Faloutsos, and J.-Y. Pan, "Random Walk with Restart: Fast Solutions and Applications,"
Knowledge and Information Systems, 2007.- [64] V.D. Blondel and P.V. Dooren, "A Measure of Similarity between Graph Vertices, with Application to Synonym Extraction and Web Searching,"
SIAM Rev., vol. 46, no. 4, pp. 647-666, 2004.- [65] W. Lu, J. Janssen, E. Milos, N. Japkowicz, and Y. Zhang, "Node Similarity in the Citation Graph,"
Knowledge and Information Systems, vol. 11, no. 1, pp. 105-129, 2006.- [66] A. Tahbaz and A. Jadbabaie, "A One-Parameter Family of Distributed Consensus Algorithms with Boundary: From Shortest Paths to Mean Hitting Times,"
Proc. IEEE Conf. Decision and Control, pp. 4664-4669, 2006.- [67] R. Agaev and P. Chebotarev, "The Matrix of Maximum Out Forests of a Digraph and Its Applications,"
Automation and Remote Control, vol. 61, no. 9, pp. 1424-1450, 2000.- [68] R. Agaev and P. Chebotarev, "Spanning Forests of a Digraph and Their Applications,"
Automation and Remote Control, vol. 62, no. 3, pp. 443-466, 2001.- [69] D. Zhao, Z. Lin, and X. Tang, "Contextual Distance for Data Perception,"
Proc. 11th IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.- [70] E.T. Jaynes, "Information Theory and Statistical Mechanics,"
Physical Rev., vol. 106, pp. 620-630, 1957.- [71] J.N. Kapur and H.K. Kesavan,
Entropy Optimization Principles with Applications. Academic Press, 1992.- [72] L. Reichl,
A Modern Course in Statistical Physics, second ed. Wiley, 1998.- [73] E. Schrodinger,
Statistical Thermodynamics, second ed. Cambridge Univ. Press, 1952.- [74] R. Bronson,
Matrix Operations. McGraw-Hill, 1989.- [75] J.G. Kemeny and J.L. Snell,
Finite Markov Chains. Springer-Verlag, 1976.- [76] T.A. Davis,
Direct Methods for Sparse Linear Systems. SIAM, 2006.- [77] G.H. Golub and C.F.V. Loan,
Matrix Computations, third ed. Johns Hopkins Univ. Press, 1996.- [78] C.D. Meyer,
Matrix Analysis and Applied Linear Algebra. SIAM, 2001.- [79] S. Fine and K. Scheinberg, "Efficient SVM Training Using Low-Rank Kernel Representations,"
J. Machine Learning Research, vol. 2, pp. 243-264, 2001.- [80] J. Callut, K. Francoisse, M. Saerens, and P. Dupont, "Semi-Supervised Classification from Discriminative Random Walks,"
Proc. European Conf. Machine Learning, pp. 162-177, 2008.- [81] M. Newman, "A Measure of Betweenness Centrality Based on Random Walks,"
Social Networks, vol. 27, no. 1, pp. 39-54, 2005.- [82] W.W. Zachary, "An Information Flow Model for Conflict and Fission in Small Groups,"
J. Anthropological Research, pp. 452-473, 1977.- [83] S.A. Macskassy and F. Provost, "Classification in Networked Data: A Toolkit and a Univariate Case Study,"
J. Machine Learning Research, vol. 8, pp. 935-983, 2007.- [84] I. Borg and P. Groenen,
Modern Multidimensional Scaling: Theory and Applications. Springer, 1997.- [85] J. Weston, B. Scholkopf, E. Ekin, C. Leslie, and W. Noble, "Dealing with Large Diagonals in Kernel Matrices,"
Annals Inst. of Statistical Math., vol. 55, no. 2, pp. 391-408, 2003.- [86] D. Greene and P. Cunningham, "Practical Solutions to the Problem of Diagonal Dominance in Kernel Document Clustering,"
Proc. 23rd Int'l Conf. Machine Learning, pp. 377-384, 2006.- [87] J. Rudnick and G. Gaspari,
Elements of the Random Walk. Cambridge Univ. Press, 2004.- [88] K. Borgwardt, C.S. Ong, S. Schonauer, S. Vishwanathan, A. Smola, and H.-P. Kriegel, "Protein Function Prediction via Graph Kernels,"
Bioinformatics, vol. 12, no. 1, pp. 337-357, 2005.- [89] S. Vishwanathan, K.M. Borgwardt, I.R. Kondor, and N.N. Schraudolph, "Graph Kernels," http://arxiv. org/pdf0807.0093, 2009.
- [90] O. Schabenberger and C.A. Gotway,
Statistical Methods for Spatial Data Analysis. Chapman & Hall, 2004.- [91] J.L. Sage and R.K. Pace,
Introduction to Spatial Econometrics. Chapman & Hall, 2009.- [92] D.A. Harville,
Matrix Algebra from a Statistician's Perspective. Springer-Verlag, 1997. |