This Article 
 Bibliographic References 
 Add to: 
Pipelining Broadcasts on Heterogeneous Platforms
April 2005 (vol. 16 no. 4)
pp. 300-313

Abstract—In this paper, we consider the communications involved by the execution of a complex application, deployed on a heterogeneous platform. Such applications extensively use macrocommunication schemes, for example, to broadcast data items. Rather than aiming at minimizing the execution time of a single broadcast, we focus on the steady-state operation. We assume that there is a large number of messages to be broadcast in pipeline fashion, and we aim at maximizing the throughput, i.e., the (rational) number of messages which can be broadcast every time-step. We target heterogeneous platforms, modeled by a graph where resources have different communication and computation speeds. Achieving the best throughput may well require that the target platform is used in totality: We show that neither spanning trees nor DAGs are as powerful as general graphs. We show how to compute the best throughput using linear programming, and how to exhibit a periodic schedule, first when restricting to a DAG, and then when using a general graph. The polynomial compactness of the description comes from the decomposition of the schedule into several broadcast trees that are used concurrently to reach the best throughput. It is important to point out that a concrete scheduling algorithm based upon the steady-state operation is asymptotically optimal, in the class of all possible schedules (not only periodic solutions).

[1] V. Kumar, A. Grama, A. Gupta, and G. Karypis, Introduction to Parallel Computing. The Benjamin/Cummings Publishing Company, Inc., 1994.
[2] S.L. Johnsson and C.-T. Ho, “Optimum Broadcasting and Personalized Communication in Hypercubes,” IEEE Trans. Computers, vol. 38, no. 9, pp. 1249-1268, Sept. 1989.
[3] J. Watts and R. Van De Geijn, “A Pipelined Broadcast for Multidimensional Meshes,” Parallel Processing Letters, vol. 5, no. 2, pp. 281-292, 1995.
[4] Y.-C. Tseng, S.-Y. Wang, and C.-W. Ho, “Efficient Broadcasting in Wormhole-Routed Multicomputers: A Network-Partitioning Approach,” IEEE Trans. Parallel and Distributed Systems, vol. 10, no. 1, pp. 44-61, Jan. 1999.
[5] H. Ko, S. Latifi, and P. Srimani, “Near-Optimal Broadcast in All-Port Wormhole-Routed Hypercubes Using Error-Correcting Codes,” IEEE Trans. Parallel and Distributed Systems, vol. 11, no. 3, pp. 247-260, Mar. 2000.
[6] S.-Y. Wang and Y.-C. Tseng, “Algebraic Foundations and Broadcasting Algorithms for Wormhole-Routed All-Port Tori,” IEEE Trans. Computers, vol. 49, no. 3, pp. 246-258, Mar. 2000.
[7] M. Snir, S.W. Otto, S. Huss-Lederman, D.W. Walker, and J. Dongarra, MPI the Complete Reference. The MIT Press, 1996.
[8] K. Hwang and Z. Xu, Scalable Parallel Computing. McGraw-Hill, 1998.
[9] J. Moore and M. Quinn, “Generating an Efficient Broadcast Sequence Using Reflected Gray Codes,” IEEE Trans. Parallel and Distributed Systems, vol. 8, no. 11, pp. 1117-1122, Nov. 1997.
[10] F. Desprez, P. Fraigniaud, and B. Tourancheau, “Successive Broadcast on Hypercube,” Technical Report CS-93-210,The Univ. of Tennessee— K noxville, 1993.
[11] O. Beaumont, A. Legrand, L. Marchal, and Y. Robert, “Optimizing the Steady-State Throughput of Broadcasts on Heterogeneous Platforms Heterogeneous Platforms,” Technical Report RR-2003-34LIP, ENS Lyon, France, June 2003.
[12] A. Schrijver, Combinatorial Optimization: Polyhedra and Efficiency, series on algorithms and combinatorics, Springer-Verlag, vol. 24, 2003.
[13] D.B. West, Introduction to Graph Theory. Prentice Hall, 1996.
[14] T.H. Cormen, C.E. Leiserson, and R.L. Rivest, Introduction to Algorithms. The MIT Press, 1990.
[15] D. Bertsimas and D. Gamarnik, “Asymptotically Optimal Algorithm for Job Shop Scheduling and Packet Routing,” J. Algorithms, vol. 33, no. 2, pp. 296-318, 1999.
[16] K. Calvert, M. Doar, and E. Zegura, “Modeling Internet Topology,” IEEE Comm. Magazine, vol. 35, no. 6, pp. 160-163, June 1997.
[17] M. Banikazemi, V. Moorthy, and D.K. Panda, “Efficient Collective Communication on Heterogeneous Networks Of Workstations,” Proc. 27th Int'l Conf. Parallel Processing (ICPP '98), 1998.
[18] N. Hall, W.-P. Liu, and J. Sidney, “Scheduling in Broadcast Networks,” Networks, vol. 32, no. 14, pp. 233-253, 1998.
[19] P. Liu and T.-H. Sheng, “Broadcast Scheduling Optimization for Heterogeneous Cluster Systems,” Proc. SPAA 2000, 12th Ann. ACM Symp. Parallel Algorithms and Architectures, pp. 129-136, 2000.
[20] P. Liu, “Broadcast Scheduling Optimization for Heterogeneous Cluster Systems,” J. Algorithms, vol. 42, no. 1, pp. 135-152, 2002.
[21] M. Banikazemi, J. Sampathkumar, S. Prabhu, D. Panda, and P. Sadayappan, “Communication Modeling of Heterogeneous Networks of Workstations for Performance Characterization of Collective Operations,” Proc. HCW '99, Eighth Heterogeneous Computing Workshop, pp. 125-133, 1999.
[22] P. Bhat, C. Raghavendra, and V. Prasanna, “Efficient Collective Communication in Distributed Heterogeneous Systems,” Proc. ICDCS '99 19th Int'l Conf. Distributed Computing Systems, pp. 15-24, 1999.
[23] “Adaptive Communication Algorithms for Distributed Heterogeneous Systems,” J. Parallel and Distributed Computing, vol. 59, no. 2, pp. 252-279, 1999.
[24] P. Liu and D.-W. Wang, “Reduction Optimization in Heterogeneous Cluster Environments,” Proc. 14th Int'l Parallel and Distributed Processing Symp. (IPDPS 2000), 2000.
[25] R. Libeskind-Hadas, J.R.K. Hartline, P. Boothe, G. Rae, and J. Swisher, “On Multicast Algorithms for Heterogeneous Networks of Workstations,” J. Parallel and Distributed Computing, vol. 61, no. 11, pp. 1665-1679, 2001.

Index Terms:
Scheduling, collective communications, NP-completeness, broadcast, heuristics, heterogeneous clusters, grids.
Olivier Beaumont, Arnaud Legrand, Loris Marchal, Yves Robert, "Pipelining Broadcasts on Heterogeneous Platforms," IEEE Transactions on Parallel and Distributed Systems, vol. 16, no. 4, pp. 300-313, April 2005, doi:10.1109/TPDS.2005.48
Usage of this product signifies your acceptance of the Terms of Use.