This Article 
 Bibliographic References 
 Add to: 
Task Allocation on a Network of Processors
December 2000 (vol. 49 no. 12)
pp. 1339-1353

Abstract—This paper studies the scheduling of tasks on a pool of identical workstations in a network where message passing is used for data transfer and communication between processors and where the precedence relations among tasks form a send-receive graph. Our parallel computation model differs from previous models by including all of the following practical considerations: 1) The sending and receiving of multiple messages from one processor to another is performed sequentially, 2) communication overhead is proportional to the message size, and 3) the starting and ending task must be performed on the same machine. These factors are crucial when performing parallel task execution using a pool of workstations whose communication primitives are provided by off-the-shelf packages, such as PVM, and whose message sizes are nontrivial. Although our model is new, using reduction from other well-known scheduling results shows that finding a scheduling with the optimal makespan is NP-hard. Our focus, therefore, is on developing and analyzing approximation algorithms for this problem. When the number of workstations in the network is abundant, a linear approximation algorithm is given with a proven performance bound of two times the optimal. When the number of available workstations is a fixed constant $k$ that is greater than $2$, we show an $O(n log n){\hbox{-}}{\rm time}$ approximation algorithm which always performs better than $(3+{\frac{k-2}{k}})$ times the optimal. Simulation results show that, on the average, both of our approximation algorithms perform much better than the worst case analysis and both generate schedulings whose makespans are very close to optimal.

[1] F.D. Anger, J.J. Hwang, and Y.C. Chow, “Scheduling with Sufficient Loosely Coupled Processors,” J. Parallel and Distributed Computing, vol. 9, pp. 87-92, 1990.
[2] A.V. Aho,J.E. Hopcroft, and J.D. Ullman,The Design and Analysis of Computer Algorithms.Reading, Mass.: Addison-Wesley, 1974.
[3] I. Ahmad and Y.-K. Kwok, “On Paralleling the Multiprocessor Scheduling Problem,” IEEE Trans. Parallel and Distributed Systems, vol. 10, pp. 414-432, 1999.
[4] J.Y. Colin and P. Chrétienne, “C.P.M. Scheduling with Small Communication Delays and Task Duplication,” Operations Research, vol. 39, no. 3, pp. 680-684, 1991.
[5] Scheduling Theory and Its Applications, P. Chrétienne, E.G. Coffman Jr., J.K. Lenstra, and Z. Liu, eds. John Wiley&Sons, 1995.
[6] P. Chrétienne, “A Polynomial Algorithm to Optimally Schedule Tasks on a Virtual Distributed System under Tree-Like Precedence Constraints,” European J. Operational Research, vol. 43, pp. 225-230, 1989.
[7] P. Chrétienne, “Task Scheduling over Distributed Memory Machines” Proc. Int'l Workshop Parallel and Distributed Algorithms, pp. 165-176, 1989.
[8] P. Chrétienne, “Task Scheduling with Interprocessor Communication Delays,” European J. Operational Research, vol. 57, pp. 348-354, 1992.
[9] D. Culler, R. Karp, D. Patterson, A. Sahay, K.E. Schauser, E. Santos, R. Subramonian, and T. von Eicken, “LogP—A Practice Model of Parallel Computation,” Comm. ACM, vol. 39, no. 11, pp. 78-85, 1996.
[10] Y. Cohen, M.S. Landy, and M. Pavel, “Hierarchical Coding of Binary Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 3, pp. 284-298, 1985.
[11] T.C.E. Cheng and C.C.S. Sin, “A State-of-the-Art Review of Parallel-Machine Scheduling Research,” European J. Operational Research, vol. 47, pp. 271-292, 1990.
[12] O. El-Dissouki and W. Huen, “Distributed Enumeration on Network Computers,” IEEE Trans. Computers, vol. 29, no. 9, pp. 818-825, 1980.
[13] A. Geist, A. Beguelin, J. Dongarra, W. Jiang, R. Manchek, and V. Sunderam, PVM 3 User's Guide and Reference Manual. Oak Ridge, Tenn.: Oak Ridge Nat'l Laboratory, May 1993.
[14] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness.New York: W.H. Freeman, 1979.
[15] P.B. Gibbons, Y. Matias, and V. Ramachandran, “The QRQW PRAM: Accounting for Contention in Parallel Algorithms,” SIAM J. Computing, vol. 28, no. 2, pp. 733-769, 1998.
[16] A. Gerasoulis and T. Yang, “A Comparison of Clustering Heuristics for Scheduling Directed Acyclic Graphs on Multiprocessors,” J. Parallel and Distributed Computing, vol. 16, pp. 276-291, 1992.
[17] A. Gerasoulis and T. Yang,"On the granularity and clustering of directed acyclic task graphs," IEEE Transactions on Parallel and Distributed Systems, vol. 4, no. 6, pp. 686-701, June 1993.
[18] J.J. Hwang,Y.C. Chow,F.D. Anger, and C.Y. Lee,"Scheduling precedence graphs in systems with interprocessor communication times," SIAM J. Computing, vol. 18, no. 2, pp. 244-257, Apr. 1989.
[19] L. Hollermann, T.-s. Hsu, D.R. Lopez, and K. Vertanen, “Scheduling Problems in a Practical Allocation Model,” J. Combinatorial Optimization, vol. 1, no. 2, pp. 129-149, 1997.
[20] V.M. Lo, “Task Assignment in Distributed Systems,” PhD thesis, Univ. of Illinois at Urbana-Champaign, Oct. 1983.
[21] V.M. Lo, "Heuristic Algorithms for Task Assignment in Distributed Systems," IEEE Trans. Computers, vol. 37, no. 11, pp. 1,384-1,397, Nov. 1988.
[22] D.R. Lopez, “Models and Algorithms for Task Allocation in a Parallel Environment,” PhD thesis, Texas A&M Univ., Dec. 1992.
[23] H.S. Malvar, “Lapped Transforms for Efficient Transformation/Subband Coding,” IEEE Trans. Acoustics, Speech, and Signal Processing, vol. 38, no. 6, pp. 969-978, 1990.
[24] D.R. McGregor, R.J. Fryer, P. Cockschott, and P. Murray, “Faster Fractal Compression,” Dr. Dobb's J., vol. 21, no. 1, pp. 34-38, 1996.
[25] P.R. Ma, E.Y. Lee, and M. Tsuchiya, “A Task Allocation Model for Distributed Computing Systems,” IEEE Trans. Computers, vol. 31, no. 1, pp. 41-47, Jan. 1982.
[26] M.G. Norman and P. Thanisch, “Models of Machines and Computation for Mapping in Multicomputers,” ACM Computing Surveys, vol. 25, no. 3, pp. 263-302, 1993.
[27] C.H. Papadimitriou and M. Yannakakis,"Towards an architecture-independent analysis of parallel algorithms," SIAM J. Computing, vol. 19, no. 2, pp. 322-328, Apr. 1990.
[28] S.A. Rees and J.P. Black, “An Experimental Investigation of Distributed Matrix Multiplication Techniques,” Software—Practice and Experience, vol. 21, no. 10, pp. 1,041-1,063, 1991.
[29] D. Saupe and M. Ruhl, “Evolutionary Fractal Image Compression,” Proc. IEEE Int'l Conf. Image Processing, pp. 1-4, 1996.
[30] H.S. Stone, “Multiprocessor Scheduling with the Aid of Network Flow Algorithms,” IEEE Trans. Software Eng., vol. 3, no. 1, pp. 85-93, 1977.
[31] P. Strobach, “Tree-Structured Scene Adaptive Coder,” IEEE Trans. Comm., vol. 38, no. 4, pp. 477-486, 1990.
[32] L.G. Valiant, “A Bridging Model for Parallel Computation,” Comm. ACM, vol. 33, no. 8, pp. 103-111, Aug. 1990.
[33] L.G. Valiant, “General Purpose Parallel Architectures,” Handbook of Theoretical Computer Science, J. van Leeuwen, ed., pp. 944-971, NorthHolland, 1990.
[34] T. Yang and A. Gerasoulis, “PYRROS: Static Scheduling and Code Generation for Message Passing Multiprocessors,” Proc. Sixth ACM Int'l Conf. Supercomputing, pp. 428-437, 1992.
[35] T. Yang and A. Gerasoulis, “DSC: Scheduling Parallel Tasks on an Unbounded Number of Processors,” IEEE Trans. Parallel and Distributed Systems, vol. 5, pp. 951-967, 1994.

Index Terms:
Scheduling, parallel/distributed systems, approximation algorithms.
Tsan-sheng Hsu, Joseph C. Lee, Dian Rae Lopez, William A. Royce, "Task Allocation on a Network of Processors," IEEE Transactions on Computers, vol. 49, no. 12, pp. 1339-1353, Dec. 2000, doi:10.1109/12.895858
Usage of this product signifies your acceptance of the Terms of Use.