The Community for Technology Leaders
RSS Icon
Subscribe
Miami, Florida
Dec. 6, 2009 to Dec. 9, 2009
ISBN: 978-0-7695-3895-2
pp: 229-238
ABSTRACT
In this paper, we describe PEGASUS, an open source Peta Graph Mining library which performs typical graph mining tasks such as computing the diameter of the graph, computing the radius of each node and finding the connected components. As the size of graphs reaches several Giga-, Tera- or Peta-bytes, the necessity for such a library grows too. To the best of our knowledge, PEGASUS is the first such library, implemented on the top of the Hadoop platform, the open source version of MapReduce. Many graph mining operations (PageRank, spectral clustering, diameter estimation, connected components etc.) are essentially a repeated matrix-vector multiplication. In this paper we describe a very important primitive for PEGASUS, called GIM-V (Generalized Iterated Matrix-Vector multiplication). GIM-V is highly optimized, achieving (a) good scale-up on the number of available machines (b) linear running time on the number of edges, and (c) more than 5 times faster performance over the non-optimized version of GIM-V. Our experiments ran on M45, one of the top 50 supercomputers in the world. We report our findings on several real graphs, including one of the largest publicly available Web Graphs, thanks to Yahoo!, with 6,7 billion edges.
INDEX TERMS
PEGASUS, graph mining, hadoop
CITATION
U. Kang, Charalampos E. Tsourakakis, Christos Faloutsos, "PEGASUS: A Peta-Scale Graph Mining System", ICDM, 2009, 2013 IEEE 13th International Conference on Data Mining, 2013 IEEE 13th International Conference on Data Mining 2009, pp. 229-238, doi:10.1109/ICDM.2009.14
31 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool