This Article 
 Bibliographic References 
 Add to: 
A Practical Approach to Dynamic Load Balancing
March 1998 (vol. 9 no. 3)
pp. 235-248

Abstract—This paper presents a cohesive, practical load balancing framework that improves upon existing strategies. These techniques are portable to a broad range of prevalent architectures, including massively parallel machines, such as the Cray T3D/E and Intel Paragon, shared memory systems, such as the Silicon Graphics PowerChallenge, and networks of workstations. As part of the work, an adaptive heat diffusion scheme is presented, as well as a task selection mechanism that can preserve or improve communication locality. Unlike many previous efforts in this arena, the techniques have been applied to two large-scale industrial applications on a variety of multicomputers. In the process, this work exposes a serious deficiency in current load balancing strategies, motivating further work in this area.

[1] S. Barnard and H. Simon, "A Fast Multilevel Implementation of Recursive Spectral Bisection for Partitioning Unstructured Problems," Concurrency: Practice and Experience, vol. 6, pp. 101-117, 1994.
[2] G. Cybenko, "Dynamic Load Balancing for Distributed Memory Multiprocessors," J. Parallel and Distributed Computing, vol. 7, pp. 279-301, 1989.
[3] D. Evans and W. Butt, "Dynamic Load Balancing Using Task-Transfer Probabilities," Parallel Computing, vol. 19, pp. 897-916, 1993.
[4] R. Ferraro, P. Liewer, and V. Decyk, "Dynamic Load Balancing for a 2D Concurrent Plasma PIC Code," Technical Report CRPC-91-6, Center for Research on Parallel Computing, 1991.
[5] A. Heirich and S. Taylor, "A Parabolic Load Balancing Algorithm," Proc. 24th Int'l Conf. Parallel Programming, vol. 3, pp. 192-202. CRC Press, 1995.
[6] H. Hofstee, J. Lukkien, and J. van de Snepscheut, "A Distributed Implementation of a Task Pool," Research Directions in High Level Parallel Progamming Languages, J. Banatre and D. Le Metayer, eds. New York: Springer Verlag, 1992.
[7] G. Horton, "A Multi-Level Diffusion Method for Dynamic Load Balancing," Parallel Computing, vol. 19, pp. 209-218, 1993.
[8] M. Ivanov, G. Markelov, S. Taylor, and J. Watts, "Parallel DSMC Strategies for 3D Computations," Proc. Parallel CFD '96, pp. 485-492, 1996.
[9] G. Kohring, "Dynamic Load Balancing for Parallelized Particle Simulations on MIMD Computers," Parallel Computing, vol. 21, pp. 683-693, 1995.
[10] F.C.H. Lin and R.M. Keller, “The Gradient Model Load Balancing Method,” IEEE Trans. Software Eng., vol. 13, no. 1, pp. 32-38, Jan. 1987.
[11] F. Muniz and E.J. Zaluska, “Parallel Load Balancing: An Extension to the Gradient Model,” Parallel Computing, vol. 21, pp. 287-301, 1995.
[12] C. Papadimitriou, Computational Complexity.New York: Addison-Wesley, 1994.
[13] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C. Cambridge Univ. Press, 1992.
[14] M. Rieffel, S. Taylor, J. Watts, and S. Shankar, "Concurrent Simulation of Plasma Reactors," Proc. High Performance Computing '97, pp. 163-168. Soc. for Computer Simulation, 1997.
[15] R. Samanta Roy, D. Hastings, and S. Taylor, "Three-Dimensional Plasma Paricle-in-Cell Calculations of Ion Thruster Backflow Contamination," J. Computational Physics, vol. 128, pp. 6-18, 1996.
[16] M. Snir, S. Otto, S. Huss-Lederman, D. Walker, and J. Dongarra, “MPI: The Complete Reference,” MIT Press,, 1995.
[17] J. Song, "A Partially Asynchronous and Iterative Algorithm for Distributed Load Balancing," Parallel Computing, vol. 20, pp. 853-868, 1994.
[18] S. Taylor, J. Watts, M. Rieffel, and M. Palmer, "The Concurrent Graph: Basic Technology for Irregular Problems," IEEE Parallel and Distributed Technology, vol. 4, pp. 15-25, 1995.
[19] R. Van Driessche and D. Roose, "An Improved Spectral Bisection Algorithm and Its Application to Dynamic Load Balancing," Parallel Computing, vol. 21, pp. 29-48, 1995.
[20] C. Walshaw and M. Berzins, "Dynamic Load-Balancing for PDE Solvers on Adaptive Unstructured Meshes," Concurrency: Practice and Experience, vol. 7, pp. 17-28, 1995.
[21] J. Watts, M. Rieffel, and S. Taylor, "A Load Balancing Technique for Multiphase Computations," Proc. High Performance Computing '97, pp. 15-20. Soc. for Computer Simulation, 1997.
[22] M. Willebeck-LeMair and A. Reeves, “Strategies for Dynamic Load Balancing on Highly Parallel Computers,” IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 9, pp. 979-993, Sept. 1993.
[23] R. Williams, "Performance of Dynamic Load-Balancing Algorithm for Unstructured Mesh Calculations," Concurrency, vol. 3, pp. 457-481, 1991.
[24] C. Xu and F.C.M. Lau, Load Balancing in Parallel Computers: Theory and Practice. Boston: Kluwer Academic, 1997.

Index Terms:
Dynamic load balancing, diffusion, massively parallel computing, irregular problems.
Jerrell Watts, Stephen Taylor, "A Practical Approach to Dynamic Load Balancing," IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 3, pp. 235-248, March 1998, doi:10.1109/71.674316
Usage of this product signifies your acceptance of the Terms of Use.