This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
On Exploiting Task Duplication in Parallel Program Scheduling
September 1998 (vol. 9 no. 9)
pp. 872-892

Abstract—One of the main obstacles in obtaining high performance from message-passing multicomputer systems is the inevitable communication overhead which is incurred when tasks executing on different processors exchange data. Given a task graph, duplication-based scheduling can mitigate this overhead by allocating some of the tasks redundantly on more than one processor. In this paper, we focus on the problem of using duplication in static scheduling of task graphs on parallel and distributed systems. We discuss five previously proposed algorithms and examine their merits and demerits. We describe some of the essential principles for exploiting duplication in a more useful manner and, based on these principles, propose an algorithm which outperforms the previous algorithms. The proposed algorithm generates optimal solutions for a number of task graphs. The algorithm assumes an unbounded number of processors. For scheduling on a bounded number of processors, we propose a second algorithm which controls the degree of duplication according to the number of available processors. The proposed algorithms are analytically and experimentally evaluated and are also compared with the previous algorithms.

[1] A. Al-Maasarani, "Priority-Based Scheduling and Evaluation of Precedence Graphs With Communication Times," MS thesis, King Fahd Univ. of Petroleum and Minerals, Saudi Arabia, 1993.
[2] M.A. Al-Mouhamed, Lower Bound on the Number of Processors and Time for Scheduling Precedence Graphs with Communication Costs IEEE Trans. Software Eng., vol. 16, no. 12, pp. 1390-1401, Dec. 1990.
[3] V.A.F. Almeida, I.M. Vasconcelos, J.N.C. Arabe, and D.A. Menasce, "Using Random Task Graphs to Investigate the Potential Benefits of Heterogeneity in Parallel Systems," Proc. Supercomputing '92, pp. 683-691, Nov. 1992.
[4] Y.-C. Chung and S. Ranka,"Applications and performance analysis of a compile-time optimization approach for list scheduling algorithms on distributed memory multiprocessors," Proc. Supercomputing '92, pp. 512-521, 1992.
[5] J.Y. Colin and P. Chretienne, "C.P.M. Scheduling With Small Computation Delays and Task Duplication," Operations Research, pp. 680-684, 1991.
[6] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness.New York: W.H. Freeman, 1979.
[7] A. Gerasoulis and T. Yang, "A Comparison of Clustering Heuristics for Scheduling DAGs on Multiprocessors," J. Parallel and Distributed Computing, vol. 16, no. 4, pp. 276-291, Dec. 1992.
[8] R.L. Graham, E.L. Lawler, J.K. Lenstra, and A.H.G. Rinnoy Kan, "Optimization and Approximation in Deterministic Sequencing and Scheduling: A Survey," Annals of Discrete Mathematics, no. 5, pp. 287-326, 1979.
[9] D.S. Hochbaum and D.B. Shmoys, "Using Dual Approximation Algorithms for Scheduling Problems: Theoretical and Practical Results," J. ACM, vol. 34, no. 1, pp. 144-162, Jan. 1987.
[10] J.J. Hwang,Y.C. Chow,F.D. Anger, and C.Y. Lee,"Scheduling precedence graphs in systems with interprocessor communication times," SIAM J. Computing, vol. 18, no. 2, pp. 244-257, Apr. 1989.
[11] H. Kasahara and S. Narita, "Practical Multiprocessor Scheduling Algorithms for Efficient Parallel Processing," IEEE Trans. Computers, vol. 33, no. 11, pp. 1,023-1,029, Nov. 1984.
[12] B. Kruatrachue and T. Lewis,"Grain size determination for parallel processing," IEEE Software, pp. 23-32, Jan. 1988.
[13] Y.-K. Kwok and I. Ahmad, “Dynamic Critical-Path Scheduling: An Effective Technique for Allocating Task Graphs to Multiprocessors,” IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 5, pp. 506-521, May 1996.
[14] T.G. Lewis and H. El-Rewini, Introduction to Parallel Computing.New York: Prentice Hall, 1992.
[15] R.E. Lord, J.S. Kowalik, and S.P. Kumar, "Solving Linear Algebraic Equations on an MIMD Computer," J. ACM, vol. 30, no. 3, pp. 103-117, Jan. 1983.
[16] C. McCreary and H. Gill,"Automatic determination of grain size for efficient parallel processing," Comm. ACM, pp. 1,073-1,078, Sept. 1989.
[17] C.H. Papadimitriou and M. Yannakakis,"Towards an architecture-independent analysis of parallel algorithms," SIAM J. Computing, vol. 19, no. 2, pp. 322-328, Apr. 1990.
[18] C.D. Polychronopoulos and D.J. Kuck, “Guided Self-Scheduling: A Practical Scheduling Scheme for Parallel Supercomputers,” IEEE Trans. Computers, vol. 36, no. 12, pp. 1425-1439, Dec. 1987.
[19] H.E. Rewini and T.G. Lewis,"Scheduling parallel program tasks onto arbitrary target machines," J. Parallel and Distributed Computing, vol. 9, pp. 138-153, 1990.
[20] B. Shirazi, H. Chen, and J. Marquis, "Comparative Study of Task Duplication Static Scheduling versus Clustering and Non-Clustering Techniques," Concurrency: Practice and Experience, vol. 7, no. 5, pp. 371-390, Aug. 1995.
[21] B. Shirazi, M. Wang, and G. Pathak, “Analysis and Evaluation of Heuristic Methods for Static Task Scheduling,” J. Parallel and Distributed Computing, vol. 10, no. 3, pp. 222-232, Nov. 1990.
[22] G.C. Sih and E.A. Lee, “A Compile-Time Scheduling Heuristic for Interconnection-Constrained Heterogeneous Processor Architectures,” IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 2, pp. 175-186, Feb. 1993.
[23] M.Y. Wu and D.D. Gajski,"Hypertool: A programming aid for message-passing systems," IEEE Transactions on Parallel and Distributed Systems, vol. 1, no. 3, pp. 330-343, July 1990.
[24] T. Yang and A. Gerasoulis, “DSC: Scheduling Parallel Tasks on an Unbounded Number of Processors,” IEEE Trans. Parallel and Distributed Systems, vol. 5, pp. 951-967, 1994.

Index Terms:
Algorithms, distributed systems, multiprocessors, duplication-based scheduling, parallel scheduling, task graphs.
Citation:
Ishfaq Ahmad, Yu-Kwong Kwok, "On Exploiting Task Duplication in Parallel Program Scheduling," IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 9, pp. 872-892, Sept. 1998, doi:10.1109/71.722221
Usage of this product signifies your acceptance of the Terms of Use.