This Article 
 Bibliographic References 
 Add to: 
Optimal Schedules for Cycle-Stealing in a Network of Workstations with a Bag-of-Tasks Workload
February 2002 (vol. 13 no. 2)
pp. 179-191

Abstract—We refine the model underlying our prior work on scheduling bag-of-tasks (“embarrassingly parallel”) workloads via cycle-stealing in networks of workstations obtaining a model wherein the scheduling guidelines of produce optimal schedules for every such cycle-stealing opportunity. We thereby render prescriptive the descriptive model of those sources. Although computing optimal schedules usually requires the use of general function-optimizing methods, we show how to compute optimal schedules efficiently for the broad class of opportunities whose durations come from a concave probability distribution. Even when no such efficient computation of an optimal schedule is available, our refined model often suggests a natural notion of approximately optimal schedule, which may be efficiently computable. We illustrate such efficient approximability via the important class of cycle-stealing opportunities whose durations come from a heavy-tailed distribution. Such opportunities do not admit any optimal schedule—nor even a natural notion of approximately optimal schedule—within the model. Within our refined model, though, we derive computationally simple schedules for heavy-tailed opportunities, which can be “tuned” to accomplish an expected amount of work that is arbitrarily close to optimal.

[1] T.E. Anderson, D.E. Culler, and D.A. Patterson, “A Case for NOW (Networks of Workstations),” IEEE Micro, vol. 15, no. 1, pp. 54–64, 1995.
[2] M.J. Atallah, C.L. Black, D.C. Marinescu, H.J. Siegel, and T.L. Casavant, “Models and Algorithms for Coscheduling Compute-Intensive Tasks on a Network of Workstations,” J. Parallel Distributed Computing, vol. 16, pp. 319-327, 1992.
[3] B. Awerbuch, Y. Azar, A. Fiat, and F.T. Leighton, “Making Commitments in the Face of Uncertainty: How to Pick a Winner Almost Every Time,” Proc. 28th ACM Symp. Theory of Computing, pp. 519-530, 1996.
[4] M. Banikazemi and D.K. Panda, “Efficient Collective Communication on Heterogeneous Networks of Workstations,” technical report, Ohio State Univ., 2000.
[5] S.N. Bhatt, F.R.K. Chung, F.T. Leighton, and A.L. Rosenberg, “On Optimal Strategies for Cycle-Stealing in Networks of Workstations,” IEEE Trans. Computers, vol. 46, pp. 545-557, 1997.
[6] R.D. Blumofe and C.E. Leiserson, "Space-Efficient Scheduling of Multithreaded Computations," Proc. 25th Ann. ACM Symp. Theory of Computing, pp. 362-371,San Diego Calif., May 1993.
[7] R.D. Blumofe and C.E. Leiserson, "Scheduling Multithreaded Computations by Work Stealing," Proc. 35th Symp. Foundations of Computer Science, 1994.
[8] R. Blumofe and D.S. Park, “Scheduling Large-Scale Parallel Computations on Networks of Workstations,” Proc. Third Int'l Symp. High-Performance Distributed Computing, pp. 96-105, 1994.
[9] F. Cappello, P. Fraigniaud, B. Mans, and A.L. Rosenberg, “HiHCoHP—Toward a Realistic Communication Model for Hierarchical Hyperclusters of Heterogeneous Processors,” Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '01), 2001.
[10] W. Cirne and K. Marzullo, The Computational Co-Op: Gathering Clusters into a Metacomputer Proc. 13th Int'l Parallel Processing Symp., pp. 160-166, 1999.
[11] D. Culler, R. Karp, D. Patterson, A. Sahay, K.E. Schauser, E. Santos, R. Subramonian, and T. von Eicken, “LogP—A Practice Model of Parallel Computation,” Comm. ACM, vol. 39, no. 11, pp. 78-85, 1996.
[12] M. Harchol-Balter and A.B. Downey, "Exploiting Process Lifetime Distribution for Dynamic Load Balancing," Proc. ACM Sigmetrics Conf., ACM Press, 1996, pp. 13-24.
[13] W.E. Leland and T.J. Ott, “Load-Balancing Heuristics and Process Behavior,” Proc. Performance and ACM SIGMETRICS, vol. 14, pp. 54-69, 1986.
[14] M. Litzkow, M. Livny, and M.W. Mutka, “Condor—A Hunter of Idle Workstations,” Proc. Eighth Int'l Conf. Distributed Computing Systems, Jun. 1988.
[15] C.H. Papadimitriou and M. Yannakakis,"Towards an architecture-independent analysis of parallel algorithms," SIAM J. Computing, vol. 19, no. 2, pp. 322-328, Apr. 1990.
[16] G.F. Pfister, In Search of Clusters, second ed. New Jersey: Prentice Hall, 1998.
[17] A.L. Rosenberg, “Guidelines for Data-Parallel Cycle-Stealing in Networks of Workstations, I: On Maximizing Expected Output,” J. Parallel and Distributed Computing, vol. 59, pp. 31-53, 1999.
[18] A.L. Rosenberg, “Guidelines for Data-Parallel Cycle-Stealing in Networks of Workstations, II: On Maximizing Guaranteed Output,” Int'l J. Foundations of Computer Science, vol. 11, pp. 183-204, 2000.
[19] A.L. Rosenberg, “Sharing Partitionable Workloads in Heterogeneous NOWs: Greedier Is not Better,” Proc. IEEE Int'l Conf. Cluster Computing, pp. 124-131, 2001.
[20] S.W. White and D.C. Torney, “Use of a Workstation Cluster for the Physical Mapping of Chromosomes,” SIAM NEWS, pp. 14-17, Mar. 1993.

Index Terms:
Cycle-stealing, bag-of-tasks workloads, heavy-tailed distributions, networks of workstations (NOWs), optimal scheduling, scheduling parallel computations.
Arnold L. Rosenberg, "Optimal Schedules for Cycle-Stealing in a Network of Workstations with a Bag-of-Tasks Workload," IEEE Transactions on Parallel and Distributed Systems, vol. 13, no. 2, pp. 179-191, Feb. 2002, doi:10.1109/71.983945
Usage of this product signifies your acceptance of the Terms of Use.