Comparison of the Efficiency of MapReduce and Bulk Synchronous Parallel Approaches to Large Network Processing
Brussels, Belgium Belgium
Dec. 10, 2012 to Dec. 10, 2012
Network structures, especially social networks, grow rapidly and provide huge datasets intractable to analyse. In this paper, two parallel approaches to process large graph structures within the Hadoop environment were compared: Bulk Synchronous Parallel (BSP) and MapReduce (MR). The experimental studies were carried out for two different graph problems: collective classification by means of Relational Influence Propagation (RIP) and Single Source Shortest Path (SSSP) calculation. The appropriate BSP and MapReduce algorithms for these problems were applied to various network datasets differing in size and structural profile, originating from three domains: telecommunication, multimedia and microblog. The collected results revealed that iterative graph processing with BSP implementation significantly outperform popular MapReduce, especially for algorithms with many iterations and sparse communication. However, MapReduce still remains the only alternative for enormous networks.
Networked Data, Bulk Synchronous Parallel, MapReduce, Large Graph Processing, Big Data, Cloud Computing, Parallel Processing, Collective Classification, Shortest Path
Tomasz Kajdanowicz, Wojciech Indyk, Przemyslaw Kazienko, Jakub Kukul, "Comparison of the Efficiency of MapReduce and Bulk Synchronous Parallel Approaches to Large Network Processing", ICDMW, 2012, 2013 IEEE 13th International Conference on Data Mining Workshops, 2013 IEEE 13th International Conference on Data Mining Workshops 2012, pp. 218-225, doi:10.1109/ICDMW.2012.135