
This Article  
 
Share  
Bibliographic References  
Add to:  
Digg Furl Spurl Blink Simpy Del.icio.us Y!MyWeb  
Search  
 
ASCII Text  x  
Arnaud Legrand, H?l?ne Renard, Yves Robert, Fr?d?ric Vivien, "Mapping and LoadBalancing Iterative Computations," IEEE Transactions on Parallel and Distributed Systems, vol. 15, no. 6, pp. 546558, June, 2004.  
BibTex  x  
@article{ 10.1109/TPDS.2004.10, author = {Arnaud Legrand and H?l?ne Renard and Yves Robert and Fr?d?ric Vivien}, title = {Mapping and LoadBalancing Iterative Computations}, journal ={IEEE Transactions on Parallel and Distributed Systems}, volume = {15}, number = {6}, issn = {10459219}, year = {2004}, pages = {546558}, doi = {http://doi.ieeecomputersociety.org/10.1109/TPDS.2004.10}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, }  
RefWorks Procite/RefMan/Endnote  x  
TY  JOUR JO  IEEE Transactions on Parallel and Distributed Systems TI  Mapping and LoadBalancing Iterative Computations IS  6 SN  10459219 SP546 EP558 EPD  546558 A1  Arnaud Legrand, A1  H?l?ne Renard, A1  Yves Robert, A1  Fr?d?ric Vivien, PY  2004 KW  Scheduling KW  loadbalancing KW  iterative computations KW  heterogeneous clusters. VL  15 JA  IEEE Transactions on Parallel and Distributed Systems ER   
Abstract—This paper is devoted to mapping iterative algorithms onto heterogeneous clusters. The application data is partitioned over the processors, which are arranged along a virtual ring. At each iteration, independent calculations are carried out in parallel, and some communications take place between consecutive processors in the ring. The question is to determine how to slice the application data into chunks, and to assign these chunks to the processors, so that the total execution time is minimized. One major difficulty is to embed a processor ring into a network that typically is not fully connected, so that some communication links have to be shared by several processor pairs. We establish a complexity result that assesses the difficulty of this problem, and we design a practical heuristic that provides efficient mapping, routing, linksharing, and data distribution schemes.
[1] J. Barbosa, J. Tavares, and A.J. Padilha, Linear Algebra Algorithms in a Heterogeneous Cluster of Personal Computers Proc. Ninth Heterogeneous Computing Workshop, pp. 147159, 2000.
[2] O. Beaumont, V. Boudet, A. Petitet, F. Rastello, and Y. Robert, A Proposal for a Heterogeneous Cluster ScaLAPACK (Dense Linear Solvers) IEEE Trans. Computers, vol. 50, no. 10, pp. 10521070, 2001.
[3] O. Beaumont, V. Boudet, F. Rastello, and Y. Robert, Matrix Multiplication on Heterogeneous Platforms IEEE Trans. Parallel and Distributed Systems, vol. 12, no. 10, pp. 10331051, Oct. 2001.
[4] F. Berman, HighPerformance Schedulers The Grid: Blueprint for a New Computing Infrastructure, I. Foster and C. Kesselman, eds., pp. 279309, MorganKaufmann, 1999.
[5] D. Bertsekas and R. Gallager, Data Networks. Prentice Hall, 1987.
[6] V. Bharadwaj, D. Ghose, V. Mani, and T.G. Robertazzi, Scheduling Divisible Loads in Parallel and Distributed Systems, IEEE CS Press, 1996.
[7] V. Bharadwaj, D. Ghose, and T.G. Robertazzi, A New Paradigm for Load Scheduling in Distributed Systems Cluster Computing, vol. 6, no. 1, pp. 718, Jan. 2003.
[8] R.P. Brent, The LINPACK Benchmark on the AP1000: Preliminary Report Proc. CAP Workshop, 1991.
[9] R. Buyya, High Performance Cluster Computing. Volume 1: Architecture and Systems. Upper Saddle River, N.J.: Prentice Hall PTR, 1999.
[10] K.L. Calvert, M.B. Doar, and E.W. Zegura, “Modeling Internet Topology,” IEEE Comm. Magazine, vol. 35, no. 6, pp. 160163, June 1997.
[11] M. Cierniak, M.J. Zaki, and W. Li, CompileTime Scheduling Algorithms for Heterogeneous Network of Workstations The Computer J., vol. 40, no. 6, pp. 356372, 1997.
[12] M. Cierniak, M.J. Zaki, and W. Li, Customized Dynamic Load Balancing for a Network of Workstations J. Parallel and Distributed Computing, vol. 43, pp. 156162, 1997.
[13] T.H. Cormen, C.E. Leiserson, and R.L. Rivest, Introduction to Algorithms. MIT Press, 1990.
[14] P.E. Crandall and M.J. Quinn, “Block Data Decomposition for DataParallel Programming on a Heterogeneous Workstation Network,” Proc. Second Int'l Symp. High Performance Distributed Computing, pp. 4249, 1993.
[15] E. Deelman and B.K. Szymanski, Dynamic Load Balancing in Parallel Discrete Event Simulation for Spatially Explicit Problems Proc. PADS'98 12th Workshop Parallel and Distributed Simulation, pp. 4653, 1998.
[16] M. Doar, A Better Model for Generating Test Networks Proc. Globecom '96, Nov. 1996.
[17] A.B. Downey, Using Pathchar to Estimate Internet Link Characteristics Measurement and Modeling of Computer Systems, pp. 222223, 1999.
[18] J.E. Flaherty, R.M. Loy, C. Özturan, M.S. Shephard, B.K. Szymanski, J.D. Teresco, and L.H. Ziantz, Parallel Structures and Dynamic Load Balancing for Adaptive Finite Element Computation Applied Numerical Math., vol. 26, nos. 12, pp. 241263, 1997.
[19] J.E. Flaherty, R.M. Loy, M.S. Shephard, B.K. Szymanski, J.D. Teresco, and L.H. Ziantz, Adaptive Local Refinement with Octree Load Balancing for the Parallel Solution of ThreeDimensional Conservation Laws J. Parallel and Distributed Computing, vol. 47, no. 2, pp. 139152, 1997.
[20] The Grid: Blueprint for a New Computing Infrastructure. I. Foster and C. Kesselman, eds., MorganKaufmann, 1999.
[21] M.R. Garey and D.S. Johnson, Computers and Intractability, a Guide to the Theory of NPCompleteness. W.H. Freeman and Company, 1991.
[22] Y.F. Hu and R.J. Blake, Load Balancing for Unstructured Mesh Applications Parallel and Distributed Computing Practices, vol. 2, no. 3, 1999.
[23] S. Ichikawa and S. Yamashita, Static Load Balancing of Parallel PDE Solver for Distributed Computing Environment Proc. 13th Int'l Conf. Parallel and Distributed Computing Systems, pp. 399405, 2000.
[24] M. Kaddoura, S. Ranka, and A. Wang, Array Decomposition for Nonuniform Computational Environments J. Parallel and Distributed Computing, vol. 36, pp. 91105, 1996.
[25] A. Kalinov and A. Lastovetsky, Heterogeneous Distribution of Computations while Solving Linear Algebra Problems on Networks of Heterogeneous Computers Proc. Conf. HighPerformance Computing and Networking (HPCN Europe), pp. 191200, 1999.
[26] D. Katabi, M. Handley, and C. Rohrs, Congestion Control for High BandwidthDelay Product Networks Proc. ACM 2002 Conf. Applications, Technologies, Architectures, and Protocols for Computer Comm. (SIGCOMM), pp. 89102, 2002.
[27] A. Legrand, H. Renard, Y. Robert, and F. Vivien, LoadBalancing Iterative Computations in Heterogeneous Clusters with Shared Communication Links Technical Report RR200323, LIP, ENS Lyon, France, also available as INRIA Research Report 4800, Apr. 2003.
[28] M. Nibhanupudi and B. Szymanski, BSPBased Adaptive Parallel Processing High Performance Cluster Computing. Volume 1: Architecture and Systems, R. Buyya, ed., pp. 702721, PrenticeHall, 1999.
[29] D. Nicol and P. Reynolds, “Optimal Dynamic Remapping of Data Parallel Computations,” IEEE Trans. Computers, vol. 39, no. 2, pp. 206219, Feb. 1990.
[30] D.M. Nicol and J.H. Saltz, "Dynamic Remapping of Parallel Computations with Varying Resource Demands," IEEE Trans. Computers., vol. 37, no. 9, pp. 1,0731,087, Sept. 1988.
[31] H. Renard, Y. Robert, and F. Vivien, Static LoadBalancing Techniques for Iterative Computations on Heterogeneous Clusters Proc. EuroPar'03: Parallel Processing, pp. 148159, 2003.
[32] B.A. Shirazi, A.R. Hurson, and K.M. Kavi, Scheduling and Load Balancing in Parallel and Distributed Systems. IEEE Computer Science Press, 1995.
[33] A.S. Tanenbaum, Computer Networks. Prentice Hall, 2003.
[34] A.G. Taylor and A.C. Hindmarsh, User Documentation for KINSOL, a Nonlinear Solver for Sequential and Parallel Computers Technical Report UCRLID131185, Lawrence Livermore Nat'l Laboratory, July 1998.
[35] J. Watts and S. Taylor, “A Practical Approach to Dynamic Load Balancing,” IEEE Trans. Parallel and Distributed Systems, vol. 9, no. 3, pp. 235–248, Mar. 1998.
[36] M.Y. Wu, On Runtime Parallel Scheduling for Processor Load Balancing IEEE Trans. Parallel and Distributed Systems, vol. 8, no. 2, pp. 173186, 1997.