The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - Aug. (2013 vol.24)
pp: 1651-1660
Enrico Gregori , Italian National Research Council, Pisa
Luciano Lenzini , University of Pisa, Pisa
Simone Mainardi , University of Pisa and Italian National Research Council, Pisa
ABSTRACT
The analysis of real-world complex networks has been the focus of recent research. Detecting communities helps in uncovering their structural and functional organization. Valuable insight can be obtained by analyzing the dense, overlapping, and highly interwoven $(k)$-clique communities. However, their detection is challenging due to extensive memory requirements and execution time. In this paper, we present a novel, parallel $(k)$-clique community detection method, based on an innovative technique which enables connected components of a network to be obtained from those of its subnetworks. The novel method has an unbounded, user-configurable, and input-independent maximum degree of parallelism, and hence is able to make full use of computational resources. Theoretical tight upper bounds on its worst case time and space complexities are given as well. Experiments on real-world networks such as the Internet and the World Wide Web confirmed the almost optimal use of parallelism (i.e., a linear speedup). Comparisons with other state-of-the-art $(k)$-clique community detection methods show dramatic reductions in execution time and memory footprint. An open-source implementation of the method is also made publicly available.
INDEX TERMS
Communities, Internet, Complexity theory, Program processors, Parallel processing, Sparse matrices, Optimization, k-clique communities, Communities, Internet, Complexity theory, Program processors, Parallel processing, Sparse matrices, Optimization, parallel community detection method
CITATION
Enrico Gregori, Luciano Lenzini, Simone Mainardi, "Parallel $(k)$-Clique Community Detection on Large-Scale Networks", IEEE Transactions on Parallel & Distributed Systems, vol.24, no. 8, pp. 1651-1660, Aug. 2013, doi:10.1109/TPDS.2012.229
REFERENCES
[1] S. Fortunato, "Community Detection in Graphs," Physics Reports, vol. 486, nos. 3-5, pp. 75-174, 2010.
[2] G. Palla, I. Derenyi, I. Farkas, and T. Vicsek, "Uncovering the Overlapping Community Structure of Complex Networks in Nature and Society," Nature, vol. 435, no. 7043, pp. 814-818, 2005.
[3] E. Gregori, L. Lenzini, and C. Orsini, "k-Dense Communities in the Internet AS-Level Topology," Proc. Third Int'l Conf. Comm. Systems and Networks (COMSNETS), 2010.
[4] E. Gregori, L. Lenzini, and C. Orsini, "K-Clique Communities in the Internet AS-Level Topology Graph," Proc. 31st Int'l Conf. Distributed Computing Systems Workshops (ICDCSW '11), 2011.
[5] R. Kumar, P. Raghavan, S. Rajagopalan, and A. Tomkins, "Trawling the Web for Emerging Cyber-Communities," Computer Networks, vol. 31, nos. 11-16, pp. 1481-1493, 1999.
[6] J.P. Onnela, J. Saramäki, J. Hyvönen, G. Szabó, D. Lazer, K. Kaski, J. Kertész, and A.L. Barabási, "Structure and Tie Strengths in Mobile Communication Networks," Proc. Nat'l Academy of Sciences of USA, vol. 104, no. 18, pp. 7332-7336, 2007.
[7] M. Girvan and M. Newman, "Community Structure in Social and Biological Networks," Proc. Nat'l Academy of Sciences of USA, vol. 99, no. 12, pp. 7821-7826, 2002.
[8] P. Chen and S. Redner, "Community Structure of the Physical Review Citation Network," J. Informetrics, vol. 4, no. 3, pp. 278-290, 2009.
[9] J.-L. Guillaume and M. Latapy, "Bipartite Graphs as Models of Complex Networks," Physica A: Statistical Mechanics and Its Applications, vol. 371, no. 2, pp. 795-813, 2006.
[10] P. Hui and J. Crowcroft, "Human Mobility Models and Opportunistic Communications System Design," Philosophical Trans. Royal Soc. A: Math., Physical and Eng. Sciences, vol. 366, no. 1872, pp. 2005-2016, 2008.
[11] P. Hui, E. Yoneki, S.Y. Chan, and J. Crowcroft, "Distributed Community Detection in Delay Tolerant Networks," Proc. ACM/IEEE Second Int'l Workshop Mobility in the Evolving Internet Architecture, pp. 7:1-7:8, 2007.
[12] G. Palla, A. Barabasi, and T. Vicsek, "Quantifying Social Group Evolution," Nature, vol. 446, pp. 664-667, Apr. 2007.
[13] G. Palla, I. Derenyi, I. Farkas, and T. Vicsek, "Uncovering the Overlapping Community Structure of Complex Networks in Nature and Society - Supplementary Material," Nature, vol. 435, no. 7043, pp. 814-818, 2005.
[14] E. Gregori, L. Lenzini, and S. Mainardi, "Parallel K-Clique Community Detection on Large-Scale Networks," http:/ cosparallel.sourceforge.net/, 2013.
[15] K. Saito, T. Yamada, and K. Kazama, "Extracting Communities from Complex Networks by the k-Dense Method," IEICE Trans. Fundamentals of Electronics Comm. Computer Sciences, vol. E91-A, no. 11, pp. 3304-3311, 2008.
[16] V. Batagelj and M. Zaversnik, "An o(m) Algorithm for Cores Decomposition of Networks," J. Computing Research Repository, vol. 0310049, 2003.
[17] Y. Zhang, J. Wang, Y. Wang, and L. Zhou, "Parallel Community Detection on Large Networks with Propinquity Dynamics," Proc. ACM SIGKDD 15th Int'l Conf. Knowledge Discovery and Data Mining, pp. 997-1006, 2009.
[18] S. Sadi, S.G. Ögüdücü, and A.S. Etaner-Uyar, "An Efficient Community Detection Method Using Parallel Clique-Finding Ants," Proc. IEEE Congress on Evolutionary Computation, pp. 1-7, 2010.
[19] V.D. Blondel, J.-L. Guillaume, R. Lambiotte, and E. Lefebvre, "Fast Unfolding of Communities in Large Networks," J. Statistical Mechanics: Theory and Experiment, vol. 2008, no. 10, p. P10008, 2008.
[20] M. Rosvall and C.T. Bergstrom, "Maps of Random Walks on Complex Networks Reveal Community Structure," Proc. Nat'l Academy of Sciences of USA, vol. 105, no. 4, pp. 1118-1123, 2008.
[21] A. Lancichinetti and S. Fortunato, "Community Detection Algorithms: A Comparative Analysis," Physical Rev. E, vol. 80, p. 056117, 2009.
[22] J. Moon and L. Moser, "On Cliques in Graphs," Israel J. Math., vol. 3, pp. 23-28, 1965.
[23] C. Bron and J. Kerbosch, "Algorithm 457: Finding All Cliques of an Undirected Graph," ACM Comm., vol. 16, no. 9, pp. 575-577, 1973.
[24] I.M. Bomze, M. Budinich, P.M. Pardalos, and M. Pelillo, "The Maximum Clique Problem," Handbook of Combinatorial Optimization, pp. 1-74, Kluwer Academic Publishers, 1999.
[25] N. Du, B. Wu, L. Xu, B. Wang, and X. Pei, "A Parallel Algorithm for Enumerating all Maximal Cliques in Complex Network," Proc. IEEE Sixth Int'l Conf. Data Mining Workshops, pp. 320-324, 2006.
[26] M.C. Schmidt, N.F. Samatova, K. Thomas, and B.-H. Park, "A Scalable, Parallel Algorithm for Maximal Clique Enumeration," J. Parallel Distributed Computing, vol. 69, pp. 417-428, 2009.
[27] Y. Zhang, F.N. Abu-Khzam, N.E. Baldwin, E.J. Chesler, M.A. Langston, and N.F. Samatova, "Genome-Scale Computational Approaches to Memory-Intensive Applications in Systems Biology," Proc. ACM/IEEE Conf. Supercomputing, 2005.
[28] J.M. Kumpula, M. Kivelä, K. Kaski, and J. Saramäki, "Sequential Algorithm for Fast Clique Percolation," Physical Rev. E, vol. 78, no. 2, p. 026109, 2008.
[29] E. Gregori, L. Lenzini, S. Mainardi, and C. Orsini, "Flip-cpm: A Parallel Community Detection Method," Proc. 26th Int'l Symp. Computer and Information Sciences (ISCIS), pp. 249-255, 2011.
[30] M.G. Everett and S.P. Borgatti, "Analyzing Clique Overlap," Connections, vol. 21, no. 1, pp. 49-61, 1998.
[31] J. Demmel, J. Dongarra, A. Ruhe, and H. van der Vorst, Templates for the Solution of Algebraic Eigenvalue Problems: A Practical Guide, Z. Bai ed. Soc. for Industrial and Applied Math., 2000.
[32] R.E. Tarjan and J. van Leeuwen, "Worst-Case Analysis of Set Union Algorithms," J. ACM, vol. 31, pp. 245-281, 1984.
[33] D. Eppstein, Z. Galil, and G.F. Italiano, "Dynamic Graph Algorithms," Algorithms and Theory of Computation Handbook, first ed., M.J. Atallah and S. Fox, eds., CRC Press, Inc., 1998.
[34] R.E. Tarjan, "Efficiency of a Good but Not Linear Set Union Algorithm," J. ACM, vol. 22, pp. 215-225, 1975.
[35] E. Gregori, L. Lenzini, and C. Orsini, "k-Clique Communities in the Internet AS-Level Topology Graph," technical report, http:/puma.isti.cnr.it/, 2010.
[36] G. Csardi and T. Nepusz, "The Igraph Software Package for Complex Network Research," http:/igraph.sf.net, 2013.
[37] E. Gregori, A. Improta, L. Lenzini, and C. Orsini, "The Impact of Ixps on the As-Level Topology Structure of the Internet," Computer Comm., vol. 34, no. 1, pp. 68-82, 2011.
[38] V. Batagelj and A. Mrvar, "Pajek Datasets," http://vlado.fmf. uni-lj.si/pub/networks data/, 2013.
[39] S. Kamvar, T. Haveliwala, C. Manning, and G. Golub, "Exploiting the Block Structure of the Web for Computing PageRank," technical Report, Stanford Univ., 2003.
[40] J.M. Kleinberg, "Authoritative Sources in a Hyperlinked Environment," J. ACM, vol. 46, pp. 604-632, 1999.
[41] D. Bu, Y. Zhao, L. Cai, H. Xue, X. Zhu, H. Lu, J. Zhang, S. Sun, L. Ling, N. Zhang, G. Li, and R. Chen, "Topological Structure Analysis of the Protein-Protein Interaction Network in Budding Yeast," Nucleic Acids Research, vol. 31, no. 9, pp. 2443-2450, 2003.
[42] M.E.J. Newman, "Finding Community Structure in Networks Using the Eigenvectors of Matrices," Phys. Rev. E, vol. 74, 2006.
[43] B. Jones, "Computational Geometry Database," ftp://ftp.cs. usask.ca/pubgeometry/, 2012.
[44] J. Leskovec, L.A. Adamic, and B.A. Huberman, "The Dynamics of Viral Marketing," ACM Trans. Web, vol. 1, article 5, 2007.
[45] J. Leskovec, J. Kleinberg, and C. Faloutsos, "Graph Evolution: Densification and Shrinking Diameters," ACM Trans. Knowledge Discovery from Data, vol. 1, article 2, 2007.
[46] J. Leskovec, J. Kleinberg, and C. Faloutsos, "Graphs over Time: Densification Laws, Shrinking Diameters and Possible Explanations," Proc. 11th ACM SIGKDD Int'l Conf. Knowledge Discovery in Data Mining, pp. 177-187, 2005.
[47] J. Leskovec, "Stanford Large Network Dataset Collection," http://snap.stanford.edudata/, 2013.
70 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool