This Article 
 Bibliographic References 
 Add to: 
Adaptive Allocation of Independent Tasks to Maximize Throughput
October 2007 (vol. 18 no. 10)
pp. 1420-1435
In this paper, we consider the task allocation problem for computing a large set of equal-sized independent tasks on a heterogeneous computing system where the tasks initially reside on a single computer (the root) in the system. This problem represents the computation paradigm for a wide range of applications such as SETI@home and Monte Carlo simulations. We consider the scenario where the systems have a general graph-structured topology, and the computers are capable of concurrent communications and overlapping communications with computation. We show that the maximization of system throughput reduces to a standard network flow problem. We then develop a decentralized adaptive algorithm that solves a relaxed form of the standard network flow problem and maximizes the system throughput. This algorithm is then approximated by a simple decentralized protocol to coordinate the resources adaptively. Simulations are conducted to verify the effectiveness of the proposed approach. For both uniformly distributed and power law distributed systems, close-to-optimal throughput is achieved and improved performance over a bandwidth-centric heuristic is observed. The adaptivity of the proposed approach is also verified through simulations.

[1] C. Banino, O. Beaumont, L. Carter, J. Ferrante, A. Legrand, and Y. Robert, “Scheduling Strategies for Master-Slave Tasking on Heterogeneous Processor Platforms,” IEEE Trans. Parallel Distributed Systems, vol. 15, no. 4, pp. 319-330, Apr. 2004.
[2] C. Banino, O. Beaumont, A. Legrand, and Y. Robert, “Scheduling Strategies for Master-Slave Tasking on Heterogeneous Processor Grids,” Proc. Int'l Conf. Applied Parallel Computing (PARA '02), pp.423-432, 2002.
[3] O. Beaumont, A. Legrand, L. Marchal, and Y. Robert, “Assessing the Impact and Limits of Steady-State Scheduling for Mixed Task and Data Parallelism on Heterogeneous Platforms,” Proc. Int'l Conf. Heterogeneous Computing (HeteroPar '04)/Proc. Int'l Symp. Parallel and Distributed Computing (ISPDC '04), 2004.
[4] O. Beaumont, A. Legrand, Y. Robert, L. Carter, and J. Ferrante, “Bandwidth-Centric Allocation of Independent Tasks on Heterogeneous Platforms,” Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '02), Apr. 2002.
[5] F. Berman, A. Chien, K. Cooper, J. Dongarra, I. Foster, D. Gannon, L. Johnsson, K. Kennedy, C. Kesselman, J. Mellor-Crummey, D. Reed, L. Torczon, and R. Wolski, “The GrADS Project: Software Support for High-Level Grid Application Development,” Int'l J. Supercomputer Applications, vol. 15, no. 4, 2001.
[6] T.D. Braun, H.J. Siegel, and N. Beck, “A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems,” J. Parallel and Distributed Computing, vol. 61, pp. 810-837, 2001.
[7] D. Collins and A. George, “Parallel and Sequential Job Scheduling in Heterogeneous Clusters: A Simulation Study Using Software in the Loop,” Simulation, vol. 77, no. 6, pp. 169-184, Dec. 2001.
[8] T.H. Cormen, C.E. Leiserson, and R.L. Rivest, Introduction to Algorithms. MIT Press, 1992.
[9] D.E. Culler, J.P. Singh, and A. Gupta, Parallel Computer Architecture, a Hardware/Software Approach. Morgan Kaufmann, 1999.
[10], http:/, 2007.
[11] M. Faloutsos, P. Faloutsos, and C. Faloutsos, “On Power-Law Relationships of the Internet Topology,” Proc. ACM Ann. Conf. Special Interest Group on Data Comm. (SIGCOMM '99), pp. 251-262, 1999.
[12] R.F. Freund and H.J. Siegel, “Heterogeneous Processing,” IEEE Computer, vol. 26, no. 6, pp. 13-17, 1993.
[13] M.G. Gouda and F.F. Haddix, “The Alternator,” Proc. Int'l Conf. Distributed Computing Systems (ICDCS '99) Workshop Self-Stabilizing Systems, pp. 48-53, 1999.
[14] B. Hong and V.K. Prasanna, “Maximum Data Gathering in Networked Sensor Systems,” Int'l J. Distributed Sensor Networks, vol. 1, no. 1, 2005.
[15] The Grid: Blueprint for a New Computing Infrastructure, I. Foster and C. Kesselman, eds. Morgan Kaufmann, 1999.
[16] O. Ibarra and C. Kim, “Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors,” J. ACM, vol. 24, no. 2, pp. 280-289, 1977.
[17] C. Inc. Cray XT3 Datasheet, http:/, 2007.
[18] E. Korpela, D. Werthimer, D. Anderson, J. Cobb, and M. Lebofsky, “SETI@home-Massively Distributed Computing for SETI,” Computing in Science and Eng., Jan. 2001.
[19] B. Kreaseck, L. Carter, H. Casanova, and J. Ferrante, “On the Interference of Communication on Computation in Java,” Proc. Int'l Workshop Performance Modeling, Evaluation, and Optimization on Parallel and Distributed Systems (PMEO-PDS '04), Apr. 2004.
[20] S.S. Kulkarni, C. Bolen, J. Oleszkiewicz, and A. Robinson, “Alternator in Read/Write Model,” Technical Report MSU-CSE-04-28, Dept. of Computer Science, Michigan State Univ., East Lansing, Michigan, July 2004.
[21] S.M. Larson, C.D. Snow, M. Shirts, and V.S. Pande, Folding@Home and Genome@Home: Using Distributed Computing to Tackle Previously Intractable Problems in Computational Biology, Computational Genomics, R. Grant, ed., Horizon Press, 2002.
[22] M. Maheswaran, S. Ali, H.J. Siegel, D. Hensgen, and R.F. Freund, “Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems,” J. Parallel and Distributed Computing, vol. 59, no. 2, pp. 107-131, 1999.
[23] A. Medina, I. Matta, and J. Byers, “On the Origin of Power Laws in Internet Topologies,” ACM Computer Comm. Rev., vol. 30, no. 2, pp.18-28, Apr. 2000.
[24] B.M.E. Moret, D.A. Bader, and T. Warnow, “High-Performance Algorithm Engineering for Computational Phylogeny,” J. Supercomputing, vol. 22, no. 1, pp. 99-111, 2002.
[25] R.V. Nieuwpoort, T. Kielmann, and H.E. Bal, “Efficient Load Balancing for Wide-Area Divide-and-Conquer Applications,” Proc. Eighth ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP '01), pp. 34-43, 2001.
[26] P. Pacheco, Parallel Programming with MPI. Morgan Kaufmann, 1996.
[27] U. Rencuzogullari and S. Dwardadas, “Dynamic Adaptation to Available Resources for Parallel Computing in an Autonomous Network of Workstations,” ACM SIGPLAN Notices, vol. 36, no. 7, pp. 72-81, 2001.
[28] G. Shao, F. Berman, and R. Wolski, “Master/Slave Computing on the Grid,” Proc. Ninth Heterogeneous Computing Workshop, May 2000.
[29] K. Taura and A.A. Chien, “A Heuristic Algorithm for Mapping Communicating Tasks on Heterogeneous Resources,” Proc. Heterogeneous Computing Workshop, pp. 102-115, 2000.
[30] D. Thain, T. Tannenbaum, and M. Livny, “Condor and the Grid,” Grid Computing: Making the Global Infrastructure a Reality, F.Berman, A.J.G. Hey, and G. Fox, eds., John Wiley & Sons, 2003.
[31] J.B. Weissman, “Scheduling Multi-Component Applications in Heterogeneous Wide-Area Networks,” Proc. Heterogeneous Computing Workshop, Int'l Parallel and Distributed Processing Symp. (IPDPS '00), May 2000.

Bo Hong, Viktor Prasanna, "Adaptive Allocation of Independent Tasks to Maximize Throughput," IEEE Transactions on Parallel and Distributed Systems, vol. 18, no. 10, pp. 1420-1435, Oct. 2007, doi:10.1109/TPDS.2007.1042
Usage of this product signifies your acceptance of the Terms of Use.