This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Collective Receiver-Initiated Multicast for Grid Applications
February 2011 (vol. 22 no. 2)
pp. 231-244
ASCII Text
x 
 
Mathijs den Burger, Thilo Kielmann, "Collective Receiver-Initiated Multicast for Grid Applications," IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 2, pp. 231-244, February, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TPDS.2010.76,
author = {Mathijs den Burger and Thilo Kielmann},
title = {Collective Receiver-Initiated Multicast for Grid Applications},
journal ={IEEE Transactions on Parallel and Distributed Systems},
volume = {22},
number = {2},
issn = {1045-9219},
year = {2011},
pages = {231-244},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPDS.2010.76},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Parallel and Distributed Systems
TI - Collective Receiver-Initiated Multicast for Grid Applications
IS - 2
SN - 1045-9219
SP231
EP244
EPD - 231-244
A1 - Mathijs den Burger,
A1 - Thilo Kielmann,
PY - 2011
KW - High-throughput multicast
KW - load balancing
KW - cluster computing.
VL - 22
JA - IEEE Transactions on Parallel and Distributed Systems
ER -
 
Mathijs den Burger, Vrije Universiteit, Amsterdam
Thilo Kielmann, Vrije Universiteit, Amsterdam
Grid applications often need to distribute large amounts of data efficiently from one cluster to multiple others (multicast). Existing sender-initiated methods arrange nodes in optimized tree structures, based on external network monitoring data. This dependence on monitoring data severely impacts both ease of deployment and adaptivity to dynamically changing network conditions. In this paper, we present Robber, a collective, receiver-initiated, high-throughput multicast approach inspired by the BitTorrent protocol. Unlike BitTorrent, Robber is specifically designed to maximize the throughput between multiple cluster computers. Nodes in the same cluster work together as a collective that tries to steal data from peer clusters. Instead of using potentially outdated monitoring data, Robber automatically adapts to the currently achievable bandwidth ratios. Within a collective, nodes automatically tune the amount of data they steal remotely to their relative performance. Our experimental evaluation compares Robber to BitTorrent, to Balanced Multicasting, and to its predecessor MOB. Balanced Multicasting optimizes multicast trees based on external monitoring data, while MOB uses collective, receiver-initiated multicast with static load balancing. We show that both Robber and MOB outperform BitTorrent. They are competitive with Balanced Multicasting as long as the network bandwidth remains stable, and outperform it by wide margins when bandwidth changes dynamically. In large environments and heterogeneous clusters, Robber outperforms MOB.

[1] H. Rangwala, E. Lantz, R. Musselman, K. Pinnow, B. Smith, and B. Wallenfelt, "Massive Parallel BLAST for the Blue Gene/L," Proc. High Availability and Performance Computing Workshop (HAPCW '05), Oct. 2005.
[2] F. Seinstra, J. Geusebroek, D. Koelma, C. Snoek, M. Worring, and A. Smeulders, "High-Performance Distributed Image and Video Content Analysis with Parallel-Horus," IEEE Multimedia, vol. 14, no. 4, pp. 64-75, Oct.-Dec., 2007.
[3] O. Beaumont, L. Marchal, and Y. Robert, "Broadcast Trees for Heterogeneous Platforms," Proc. 19th Int'l Parallel and Distributed Processing Symp. (IPDPS '05), Apr. 2005.
[4] R. Izmailov, S. Ganguly, and N. Tu, "Fast Parallel File Replication in Data Grid," Proc. Future of Grid Data Environments Workshop (GGF-10), Mar. 2004.
[5] T. Kielmann, R.F. Hofman, H.E. Bal, A. Plaat, and R.A. Bhoedjang, "MagPIe: MPI's Collective Communication Operations for Clustered Wide Area Systems," Proc. ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP), pp. 131-140, May 1999.
[6] B. Lowekamp, B. Tierney, L. Cottrell, R. Hughes-Jones, T. Kielmann, and M. Swany, "A Hierarchy of Network Performance Characteristics for Grid Applications and Services," Proposed Recommendation GFD-R-P.023, Global Grid Forum, 2004.
[7] M. den Burger, T. Kielmann, and H.E. Bal, "Balanced Multicasting: High-Throughput Communication for Grid Applications," Proc. Conf. Supercomputing (SC '05), Nov. 2005.
[8] B. Cohen, "Incentives Build Robustness in BitTorrent," Proc. First Workshop Economics of Peer-to-Peer Systems, June 2003.
[9] V. Pai, K. Kumar, K. Tamilmani, V. Sambamurthy, and A. Mohr, "Chainsaw: Eliminating Trees from Overlay Multicast," Proc. Fourth Int'l Workshop Peer-to-Peer Systems (IPTPS '05), Feb. 2005.
[10] M. den Burger and T. Kielmann, "MOB: Zero-Configuration High-Throughput Multicasting for Grid Applications," Proc. 16th IEEE Int'l Symp. High Performance Distributed Computing (HPDC '07), June 2007.
[11] R. van Nieuwpoort, J. Maassen, G. Wrzesinska, R. Hofman, C. Jacobs, T. Kielmann, and H. Bal, "Ibis: A Flexible and Efficient Java-Based Grid Programming Environment," Concurrency and Computation: Practice and Experience, vol. 17, nos. 7/8, pp. 1079-1107, June/July 2005.
[12] The Distributed ASCI Supercomputer 3, http://www.cs.vu.nldas3/, 2010.
[13] R. Cohen and G. Kaempfer, "A Unicast-Based Approach for Streaming Multicast," Proc. IEEE INFOCOM, pp. 440-448, Apr. 2001.
[14] M. Kim, S. Lam, and D. Lee, "Optimal Distribution Tree for Internet Streaming Media," Proc. 23rd Int'l Conf. Distributed Computing Systems (ICDCS '03), May 2003.
[15] Y. Cui, Y. Xue, and K. Nahrstedt, "Max-Min Overlay Multicast: Rate Allocation and Tree Construction," Proc. 12th IEEE Int'l Workshop Quality of Service (IwQoS '04), June 2004.
[16] M. Castro, P. Druschel, A. Kermarrec, A. Nandi, A. Rowstron, and A. Singh, "SplitStream: High-Bandwidth Multicast in Cooperative Environments," Proc. 19th ACM Symp. Operating System Principles (SOSP-19), Oct. 2003.
[17] R. Wolski, "Experiences with Predicting Resource Performance On-Line in Computational Grid Settings," ACM SIGMETRICS Performance Evaluation Rev., vol. 30, no. 4, pp. 41-49, Mar. 2003.
[18] T. Gross, B. Lowekamp, R. Karrer, N. Miller, and P. Steenkiste, "Design, Implementation and Evaluation of the Remos Network," J. Grid Computing, vol. 1, no. 1, pp. 75-93, May 2003.
[19] J. Maassen, R.V. van Nieuwpoort, T. Kielmann, K. Verstoep, and M. den Burger, "Middleware Adaptation with the Delphoi Service," Concurrency and Computation: Practice and Experience, vol. 18, no. 13, pp. 1659-1679, Nov. 2006.
[20] N.T. Karonis, B.R. de Supinski, I. Foster, W. Gropp, E. Lusk, and J. Bresnahan, "Exploiting Hierarchy in Parallel Computer Networks to Optimize Collective Operation Performance," Proc. 14th Int'l Parallel and Distributed Processing Symp. (IPDPS '00), pp. 377-384, May 2000.
[21] T. Kielmann, H. Bal, S. Gorlatch, K. Verstoep, and R. Hofman, "Network Performance-Aware Collective Communication for Clustered Wide Area Systems," Parallel Computing, vol. 27, no. 11, pp. 1431-1456, 2001.
[22] K. Verstoep, K. Langendoen, and H. Bal, "Efficient Reliable Multicast on Myrinet," Proc. Int'l Conf. Parallel Processing (ICPP '96), vol. 3, pp. 156-165, Aug. 1996.
[23] Y. Cui, B. Li, and K. Nahrstedt, "On Achieving Optimized Capacity Utilization in Application Overlay Networks with Multiple Competing Sessions," Proc. 16th Ann. ACM Symp. Parallelism in Algorithms and Architectures (SPAA '04), pp. 160-169, June 2004.
[24] D. Kostić, A. Rodriguez, J. Albrecht, and A. Vahdat, "Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh," Proc. 19th ACM Symp. Operating System Principles (SOSP-19), Oct. 2003.
[25] R. Bindal, P. Cao, W. Chan, J. Medval, G. Suwala, T. Bates, and A. Zhang, "Improving Traffic Locality in BitTorrent via Biased Neighbor Selection," Proc. 26th Int'l Conf. Distributed Computing Systems (ICDCS '06), July 2006.
[26] J. Pouwelse, P. Garbacki, J.W.A. Bakker, J. Yang, A. Iosup, D. Epema, M. Reinders, M. van Steen, and H. Sips, "Tribler: A Social-Based Peer-to-Peer System," Proc. Fifth Int'l Workshop Peer-to-Peer Systems (IPTPS '06), Feb. 2006.
[27] C. Gkantsidis and P. Rodriguez, "Network Coding for Large Scale Content Distribution," Proc. IEEE INFOCOM, Mar. 2005.
[28] T. Fenner and A. Frieze, "On the Connectivity of Random m-Orientable Graphs and Digraphs," Combinatorica, vol. 2, no. 4, pp. 347-359, 1982.
[29] W. Si and M. Li, "On the Connectedness of Peer-to-Peer Overlay Networks," Proc. 11th Int'l Conf. Parallel and Distributed Systems (ICPADS '05), vol. 1, pp. 474-480, July 2005.
[30] K. Chung, Markov Chains with Stationary Transition Probabilities, chapter 3. Springer-Verlag, 1967.
[31] J. Maassen and H.E. Bal, "Solving the Connectivity Problems in Grid Computing," Proc. 16th IEEE Int'l Symp. High-Performance Distributed Computing (HPDC '07), June 2007.
[32] Linux Advanced Routing and Traffic Control, http:/lartc.org/, 2010.
[33] M. Devera, HTB Linux Queuing Discipline Manual—User Guide, http://luxik.cdi.cz/devik/qoshtb/, 2008.
[34] S. Hemminger, "Network Emulation with NetEm," Proc. Linux Conf. Australia, Apr. 2005.
[35] The Distributed ASCI Supercomputer 2, http://www.cs.vu.nldas2/, 2010.

Index Terms:
High-throughput multicast, load balancing, cluster computing.
Citation:
Mathijs den Burger, Thilo Kielmann, "Collective Receiver-Initiated Multicast for Grid Applications," IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 2, pp. 231-244, Feb. 2011, doi:10.1109/TPDS.2010.76
Usage of this product signifies your acceptance of the Terms of Use.