This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Adaptive Allocation of Independent Tasks to Maximize Throughput
October 2007 (vol. 18 no. 10)
pp. 1420-1435
In this paper, we consider the task allocation problem for computing a large set of equal-sized independent tasks on a heterogeneous computing system where the tasks initially reside on a single computer (the root) in the system. This problem represents the computation paradigm for a wide range of applications such as SETI@home and Monte Carlo simulations. We consider the scenario where the systems have a general graph-structured topology, and the computers are capable of concurrent communications and overlapping communications with computation. We show that the maximization of system throughput reduces to a standard network flow problem. We then develop a decentralized adaptive algorithm that solves a relaxed form of the standard network flow problem and maximizes the system throughput. This algorithm is then approximated by a simple decentralized protocol to coordinate the resources adaptively. Simulations are conducted to verify the effectiveness of the proposed approach. For both uniformly distributed and power law distributed systems, close-to-optimal throughput is achieved and improved performance over a bandwidth-centric heuristic is observed. The adaptivity of the proposed approach is also verified through simulations.

[1] C. Banino, O. Beaumont, L. Carter, J. Ferrante, A. Legrand, and Y. Robert, “Scheduling Strategies for Master-Slave Tasking on Heterogeneous Processor Platforms,” IEEE Trans. Parallel Distributed Systems, vol. 15, no. 4, pp. 319-330, Apr. 2004.
[2] C. Banino, O. Beaumont, A. Legrand, and Y. Robert, “Scheduling Strategies for Master-Slave Tasking on Heterogeneous Processor Grids,” Proc. Int'l Conf. Applied Parallel Computing (PARA '02), pp.423-432, 2002.
[3] O. Beaumont, A. Legrand, L. Marchal, and Y. Robert, “Assessing the Impact and Limits of Steady-State Scheduling for Mixed Task and Data Parallelism on Heterogeneous Platforms,” Proc. Int'l Conf. Heterogeneous Computing (HeteroPar '04)/Proc. Int'l Symp. Parallel and Distributed Computing (ISPDC '04), 2004.
[4] O. Beaumont, A. Legrand, Y. Robert, L. Carter, and J. Ferrante, “Bandwidth-Centric Allocation of Independent Tasks on Heterogeneous Platforms,” Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '02), Apr. 2002.
[5] F. Berman, A. Chien, K. Cooper, J. Dongarra, I. Foster, D. Gannon, L. Johnsson, K. Kennedy, C. Kesselman, J. Mellor-Crummey, D. Reed, L. Torczon, and R. Wolski, “The GrADS Project: Software Support for High-Level Grid Application Development,” Int'l J. Supercomputer Applications, vol. 15, no. 4, 2001.
[6] T.D. Braun, H.J. Siegel, and N. Beck, “A Comparison of Eleven Static Heuristics for Mapping a Class of Independent Tasks onto Heterogeneous Distributed Computing Systems,” J. Parallel and Distributed Computing, vol. 61, pp. 810-837, 2001.
[7] D. Collins and A. George, “Parallel and Sequential Job Scheduling in Heterogeneous Clusters: A Simulation Study Using Software in the Loop,” Simulation, vol. 77, no. 6, pp. 169-184, Dec. 2001.
[8] T.H. Cormen, C.E. Leiserson, and R.L. Rivest, Introduction to Algorithms. MIT Press, 1992.
[9] D.E. Culler, J.P. Singh, and A. Gupta, Parallel Computer Architecture, a Hardware/Software Approach. Morgan Kaufmann, 1999.
[10] Distributed.net, http:/www.distributed.net, 2007.
[11] M. Faloutsos, P. Faloutsos, and C. Faloutsos, “On Power-Law Relationships of the Internet Topology,” Proc. ACM Ann. Conf. Special Interest Group on Data Comm. (SIGCOMM '99), pp. 251-262, 1999.
[12] R.F. Freund and H.J. Siegel, “Heterogeneous Processing,” IEEE Computer, vol. 26, no. 6, pp. 13-17, 1993.
[13] M.G. Gouda and F.F. Haddix, “The Alternator,” Proc. Int'l Conf. Distributed Computing Systems (ICDCS '99) Workshop Self-Stabilizing Systems, pp. 48-53, 1999.
[14] B. Hong and V.K. Prasanna, “Maximum Data Gathering in Networked Sensor Systems,” Int'l J. Distributed Sensor Networks, vol. 1, no. 1, 2005.
[15] The Grid: Blueprint for a New Computing Infrastructure, I. Foster and C. Kesselman, eds. Morgan Kaufmann, 1999.
[16] O. Ibarra and C. Kim, “Heuristic Algorithms for Scheduling Independent Tasks on Nonidentical Processors,” J. ACM, vol. 24, no. 2, pp. 280-289, 1977.
[17] C. Inc. Cray XT3 Datasheet, http:/www.cray.com, 2007.
[18] E. Korpela, D. Werthimer, D. Anderson, J. Cobb, and M. Lebofsky, “SETI@home-Massively Distributed Computing for SETI,” Computing in Science and Eng., Jan. 2001.
[19] B. Kreaseck, L. Carter, H. Casanova, and J. Ferrante, “On the Interference of Communication on Computation in Java,” Proc. Int'l Workshop Performance Modeling, Evaluation, and Optimization on Parallel and Distributed Systems (PMEO-PDS '04), Apr. 2004.
[20] S.S. Kulkarni, C. Bolen, J. Oleszkiewicz, and A. Robinson, “Alternator in Read/Write Model,” Technical Report MSU-CSE-04-28, Dept. of Computer Science, Michigan State Univ., East Lansing, Michigan, July 2004.
[21] S.M. Larson, C.D. Snow, M. Shirts, and V.S. Pande, Folding@Home and Genome@Home: Using Distributed Computing to Tackle Previously Intractable Problems in Computational Biology, Computational Genomics, R. Grant, ed., Horizon Press, 2002.
[22] M. Maheswaran, S. Ali, H.J. Siegel, D. Hensgen, and R.F. Freund, “Dynamic Mapping of a Class of Independent Tasks onto Heterogeneous Computing Systems,” J. Parallel and Distributed Computing, vol. 59, no. 2, pp. 107-131, 1999.
[23] A. Medina, I. Matta, and J. Byers, “On the Origin of Power Laws in Internet Topologies,” ACM Computer Comm. Rev., vol. 30, no. 2, pp.18-28, Apr. 2000.
[24] B.M.E. Moret, D.A. Bader, and T. Warnow, “High-Performance Algorithm Engineering for Computational Phylogeny,” J. Supercomputing, vol. 22, no. 1, pp. 99-111, 2002.
[25] R.V. Nieuwpoort, T. Kielmann, and H.E. Bal, “Efficient Load Balancing for Wide-Area Divide-and-Conquer Applications,” Proc. Eighth ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP '01), pp. 34-43, 2001.
[26] P. Pacheco, Parallel Programming with MPI. Morgan Kaufmann, 1996.
[27] U. Rencuzogullari and S. Dwardadas, “Dynamic Adaptation to Available Resources for Parallel Computing in an Autonomous Network of Workstations,” ACM SIGPLAN Notices, vol. 36, no. 7, pp. 72-81, 2001.
[28] G. Shao, F. Berman, and R. Wolski, “Master/Slave Computing on the Grid,” Proc. Ninth Heterogeneous Computing Workshop, May 2000.
[29] K. Taura and A.A. Chien, “A Heuristic Algorithm for Mapping Communicating Tasks on Heterogeneous Resources,” Proc. Heterogeneous Computing Workshop, pp. 102-115, 2000.
[30] D. Thain, T. Tannenbaum, and M. Livny, “Condor and the Grid,” Grid Computing: Making the Global Infrastructure a Reality, F.Berman, A.J.G. Hey, and G. Fox, eds., John Wiley & Sons, 2003.
[31] J.B. Weissman, “Scheduling Multi-Component Applications in Heterogeneous Wide-Area Networks,” Proc. Heterogeneous Computing Workshop, Int'l Parallel and Distributed Processing Symp. (IPDPS '00), May 2000.

Citation:
Bo Hong, Viktor Prasanna, "Adaptive Allocation of Independent Tasks to Maximize Throughput," IEEE Transactions on Parallel and Distributed Systems, vol. 18, no. 10, pp. 1420-1435, Oct. 2007, doi:10.1109/TPDS.2007.1042
Usage of this product signifies your acceptance of the Terms of Use.