This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Generalized Scheme for Mapping Parallel Algorithms
March 1993 (vol. 4 no. 3)
pp. 328-346

A generalized mapping strategy that uses a combination of graph theory, mathematicalprogramming, and heuristics is proposed. The authors use the knowledge from the givenalgorithm and the architecture to guide the mapping. The approach begins with agraphical representation of the parallel algorithm (problem graph) and the parallelcomputer (host graph). Using these representations, the authors generate a newgraphical representation (extended host graph) on which the problem graph is mapped.An accurate characterization of the communication overhead is used in the objectivefunctions to evaluate the optimality of the mapping. An efficient mapping scheme isdeveloped which uses two levels of optimization procedures. The objective functionsinclude minimizing the communication overhead and minimizing the total execution timewhich includes both computation and communication times. The mapping scheme istested by simulation and further confirmed by mapping a real world application ontoactual distributed environments.

[1] F. Berman and L. Snyder, "On mapping parallel algorithms into parallel architectures," inProc. Int. Conf. Parallel Processing, Aug. 1984, pp. 307-309.
[2] S. H. Bokhari, "On the mapping problem,"IEEE Trans. Comput., vol. C-30, no. 3, pp. 207-214, Mar. 1981.
[3] S. H. Bokhari, "Dual processor scheduling with dynamic reassignment,"IEEE Trans. Software Eng., vol. SE-5, no. 4, pp. 326-334, July 1979.
[4] S. H. Bokhari, "A shortest tree algorithm for optimal assignments across space and time in a distributed processor system,"IEEE Software Eng., vol. SE-7, no. 6, pp. 335-341, Nov. 1981.
[5] T. L. Casavant and J. G. Kuhl, "A taxonomy of scheduling in general-purpose distributed computing systems,"IEEE Trans. Software Eng., vol. 14, no. 2, pp. 42-45, Feb. 1988.
[6] T. L. Casavant and J. G. Kuhl, "Design of a loosely-coupled distributed multiprocessing network," inProc. Int. Conf. Parallel Processing, Aug. 1984, pp. 42-45.
[7] M. J. Gonzales, "Deterministic processor scheduling,"ACM Comput. Surveys, vol. 9, no. 3, Sept. 1977.
[8] E. L. Lawler, J. K. Lenstra, and A. H. G. R. Kan, "Recent developments in deterministic sequencing and scheduling: A survey," inDeterministic and Stochastic Scheduling, M. A. H. Dempsteret al., Eds. Reidel, 1982.
[9] S. J. Kim and J. C. Browne, "A general approach to mapping of parallel computations upon multiprocessor architectures," inProc. Int. Conf. Parallel Processing, Aug. 1988, pp. 1-8.
[10] S.-Y. Lee and J. K. Aggarwal, "A mapping strategy for parallel processing,"IEEE Trans. Comput., vol. C-36, pp. 433-442, Apr. 1987.
[11] E. G. Coffman,Computer and Job-Shop Scheduling Theory. New York: Wiley, 1974.
[12] J. D. Ullman, "NP-complete scheduling problems,"J. Comput. Syst. Sci., vol. 10, pp. 384-393, 1975.
[13] K. Vairavan and R. A. DeMillo, "On the computational complexity of a generalized scheduling problem,"IEEE Trans. Comput., vol. C-25, pp. 1067-1073, Nov. 1076.
[14] D. Bernstein, M. Rodeh, and I. Gertner, "On the complexity of scheduling problems for parallel/pipelined machines,"IEEE Trans. Comput., vol. C-38, no. 9, pp. 1308-1313, Sept. 1989.
[15] J. K. Lenstra and A. H. G. R. Kan, "Complexity of scheduling under precedence constraints,"Oper. Res., vol. 26, pp. 22-35, Jan. 1978.
[16] K. Efe, "Heuristic models of task assignment scheduling in distributed systems,"IEEE Comput. Mag., vol. 15, pp. 50-56, June 1982.
[17] J. C. Browne, "Formulation and programming of parallel computations: A unified approach," inProc. Int. Conf. Parallel Processing, Aug. 1985, pp. 624-631.
[18] J. C. Browne, "Framework for formulation and analysis of parallel computation structures,"Parallel Comput., vol. 3, pp. 1-9, 1986.
[19] L. D. Wittie and A. M. Tilborg, "MICROS, a distributed operating system for MICRONET, a reconfigurable network computer,"IEEE Trans. Comput., vol. C-29, no. 12, pp. 1133-1144, Dec. 1980.
[20] R. M. Bryant and R. A. Finkel, "A stable distributed scheduling algorithm," inProc. Int. Conf. Distributed Comput. Syst., Apr. 1981, pp. 314-323.
[21] T. C. K. Chou and J. A. Abraham, "Load balancing in distributed systems,"IEEE Trans. Softwre Eng., vol. SE-8, no. 4, pp. 401-412, July 1982.
[22] A. Gottlieb, R. Grishman, C. P. Kruskal, K. P. McAuliffe, L. Rudolph, and M. Snir, "The NYU Ultracomputer--Designing an MIMD shared memory parallel computer,"IEEE Trans. Comput., vol. C-32, no. 2, pp. 175-189, Feb. 1983.
[23] C. C. Shen and W. H. Tsai, "A graph matching approach to optimal task assignment in distributed computing systems using a minimax criterion,"IEEE Trans. Comput., vol. C-34, no. 3, pp. 197-203, Mar. 1985.
[24] Kashahara and Narita, "Practical multiprocessor scheduling algorithms for efficient parallel processing,"IEEE Trans. Comput., vol. C-33, no. 11, pp. 1023-1029, Nov. 1984.
[25] W. W. Chu, L. J. Hooloway, M. T. Lan, and K. Efe, "Task allocation in distributed data processing,"IEEE Comput. Mag., vol. 13, no. 11, pp. 57-69, Nov. 1980.
[26] T. C. Hu, "Parallel sequencing and assembly line problem,"Oper. Res., vol. 9, pp. 841-848, Nov. 1961.
[27] E. G. Coffman and R. L. Graham, "Optimal scheduling for two-processor systems,"Acta Informatica, vol. 1, pp. 200-213, 1972.
[28] A. K. Jones, R. J. Chansler, I. Durham, J. Mohan, K. Schwan, and S. Vegdahl, "Staros, a multiprocessor operating system," inProc. 7th ACM Symp. Operating System Principles, Asilomar, CA, Dec. 10-12, 1979, pp. 117-127.
[29] J. K. Ousterhout, A. Scelza, and P. S. Sindhu, "Medusa: An experiment in distributed operating system structure,"Commun. ACM, vol. 23, no. 2, pp. 92-105, Feb. 1980.
[30] J. A. Stankovicet al., "A review of current research and critical issues in distributed system software,"IEEE Computer Society Distributed Process. Tech. Committee Newslett., vol. 7, pp. 14-47, Mar. 1985.
[31] K. W. Doty, P. L. McEntire, and J. G. O'Reilly, "Task allocation in a distributed computer system," inProc. IEEE InfoCom, 1982, pp. 33-38.
[32] A. Gabrielian and D. B. Tyler, "Optimal object allocation in distributed computer systems," inProc. Int. Conf. Distributed Comput. Syst., May 1984, pp. 84-95.
[33] P. Y. R. Ma, E. Y. S. Lee, and J. Tsuchiya, "A task allocation model for distributed computing systems,"IEEE Trans. Comput., vol. C-31, no. 1, pp. 41-47, Jan. 1982.
[34] L. M. Ni and K. Hwang, "Optimal load balancing for a multiple processor system," inProc. Int. Conf. Parallel Processing, 1981, pp. 352-357.
[35] J. A. Bannister and K. S. Trivedi, "Task allocation in fault-tolerant distributed systems,"Acta Informatica, vol. 20, pp. 261-281, 1983.
[36] V. M. Lo, "Heuristic algorithms for task assignment in distributed systems," inProc. Int. Conf. Distributed Comput. Syst., May 1984, pp. 30-39.
[37] V. M. Lo, "Heuristic algorithms for task assignment in distributed systems,"IEEE Trans. Comput., vol. C-37, no. 11, pp. 1384-1397, Nov. 1988.
[38] C. C. Price and S. Krishnaprasad, "Software allocation models for distributed computing systems," inProc. Int. Conf. Distributed Comput. Sys., May 1984, pp. 40-48.
[39] C. P. Kruskal and A. Weiss, "Allocating independent subtasks on parallel processors, extended abstract," inProc. Int. Conf. Parallel Processing, Aug. 1984, pp. 236-240.
[40] M. O. Ward and D. J. Romero, "Assigning parallel executable intercommunicating subtasks to processors," inProc. Int. Conf. Parallel Processing, Aug. 1984, pp. 392-394.
[41] V. Chaudhary and J. K. Aggarwal, "Generalized mapping of parallel algorithms onto parallel architectures," inProc. Int. Conf. Parallel Processing, Aug. 1990, pp. 137-141.
[42] J. A. Stankovic and I. S. Sidhu, "An adaptive bidding algorithm of processes, clusters, and distributed groups," inProc. Int. Conf. Distributed Comput. Syst., May 1984, pp. 49-59.
[43] H. S. Stone, "Critical load factors in two-processor distributed systems,"IEEE Trans. Software Eng., vol. SE-4, no. 3, pp. 254-258, May 1978.
[44] H. S. Stone and S. H. Bokhari, "Control of distributed processes,"IEEE Comput. Mag., vol. 11, pp. 97-106, July 1978.
[45] L. Kleinrock,Queueing Systems, Vol. 2: Computer Applications. New York: Wiley, 1976.
[46] L. Kleinrock and A. Nilsson, "On optimal scheduling algorithms for time-shared systems,"J. ACM, vol. 28, no. 3, pp. 477-486, July 1981.
[47] J. A. Stankovicet al., "An evaluation of the applicability of different mathematical approaches to the analysis of decentralized control algorithms," inProc. IEEE COMPSAC, Nov. 1982, pp. 62-69.
[48] P. H. Enslow Jr., "What is a 'distributed' data processing system,"IEEE Comput. Mag., vol. 11, no. 1, pp. 13-21, Jan. 1978.
[49] Bolt Beranek and Newman Inc., "Butterfly (TM) parallel processor overview," Cambridge, MA, June 1985.
[50] H. Forsdick, R. Schantz, and R. Thomas, "Operating systems for computer networks,"IEEE Trans. Comput., no. 11, Jan. 1978.
[51] M. Hanan and J. M. Kurtzberg, "A review of the placement and quadratic assignment problems,"SIAM Rev., vol. 14, pp. 324-342, Apr. 1972.
[52] C. E. McDowell and W. F. Appelbe, "Processor scheduling for linearly connected parallel processors,"IEEE Trans. Comput., vol. C-35, pp. 632-638, 1986.
[53] S. Sahni, "Scheduling multipipeline and multiprocessor computers,"IEEE Trans. Comput., vol. C-33, no. 7, pp. 637-645, July 1984.
[54] R. P. Bianchini, Jr. and J. P. Shen, "Interprocessor traffic scheduling algorithm for multiple-processor networks,"IEEE Trans. Comput., vol. C-36, pp. 396-409, Apr. 1987.
[55] G. R. Andrews, D. P. Dobkin, and P. J. Downey, "Distributed allocation with pools of servers," inProc. ACM SIGACT-SIGOPS Symp. Principles Distributed Comput., Aug. 1982, pp. 73-83.
[56] L. M. Casey, "Decentralized scheduling,"Australian Comput. J., vol. 13, pp. 58-63, May 1981.
[57] C. Gao, J. W. S. Liu, and M. Railey. "Load balancing algorithms in homogeneous distributed systems," inProc. Int. Conf. Parallel Processing, Aug. 1984, pp. 302-306.
[58] D. Klappholz and H. C. Park, "Parallelized process scheduling for a tightly-coupled MIMD machine," inProc. Int. Conf. Parallel Processing, Aug. 1984, pp. 315-321.
[59] V. M. Lo, "Task assignment to minimize completion time," inProc. Int. Conf. Distributed Comput. Syst., May 1985, pp. 329-336.
[60] S. Majumdar and M. L. Green, "A distributed real time resource manager, " inProc. IEEE Symp. Distributed Data Acquisition, Comput. Contr., 1980, pp. 185-193.
[61] C. V. Ramamoorthyet al., "Optimal scheduling strategies in a multiprocessor system,"IEEE Trans. Comput., vol. C-21, no. 2, pp. 137-146, Feb. 1972.
[62] K. Ramamrithan and J. A. Stankovic, "Dynamic task scheduling in distributed hard real-time system," inProc. Int. Canf. Distributed Comput. Syst., May 1984, pp. 96-107.
[63] J. Reif and P. Spirakis, "Real-time resource allocation in a distributed system, " inProc. ACM SIGACT-SIGOPS Symp. Principles Distributed Comput., Aug. 1982, pp. 84-94.
[64] T. G. Saponis and P. L. Crews, "A model for decentralized control in a fully distributed processing system," inProc. COMPCON, 1980, pp. 307-312.
[65] J. A. Stankovic, "The analysis of a decentralized control algorithm for job scheduling utilizing bayesian decision theory," inProc. Int. Conf. Parallel Processing, 1981, pp. 333-337.
[66] J. A. Stankovic, "A heuristic of cooperation among decentralized controllers," inProc. IEEE INFOCOM, Apr. 1983, pp. 331-339.
[67] J. A. Stankovic, "An application of bayesian decision theory to decentralized control of job scheduling,"IEEE Trans. Comput., vol. C-34, no. 2, pp. 117-130, Feb. 1985.
[68] J. A. Stankovic, "Stability and distributed scheduling algorithms," inProc. ACM Nat. Conf., vol. 2, Mar. 1985.
[69] P. Tang, P. C. Yew, Z. Fang, and C. Q. Zhu, "Deadlock prevention in processor self-scheduling for parallel nested loops," inProc. Int. Conf. Parallel Processing, Aug. 1987, pp. 11-18.
[70] S. P. Lo and V. D. Gligor, "Properties of multiprocessor scheduling algorithms," inProc. Int. Conf. Parallel Processing, Aug. 1987, pp 867-870.
[71] C. E. Houstis, "Allocation of real-time applications to distributed systems," inProc. Int. Conf. Parallel Processing, Aug. 1987, pp. 863-866.
[72] K. M. Baumgartner and B. W. Wah, "Load balancing protocols on a local computer system with a multiaccess network," inProc. Int. Conf. Parallel Processing, Aug. 1987, pp. 851-858.
[73] D. C. Gerogiannis and S. C. Orphanoudakis, "Efficient embedding of interprocessor communication in parallel implementations of intermediate level vision tasks," inProc. Int. Conf. Comput. Vision Pattern Recognition, June 1990, pp. 368-372.
[74] M. Yasberi, J. C. Browne, and D. P. Agrawal, "Analysis and design of parallel algorithms and implementations for some image processing operations," inProc. Int. Conf. Parallel Processing, Aug. 1987, pp. 901-908.
[75] T. C. K. Chou and J. A. Abraham, "Load redistribution under failure in distributed systems,"IEEE Trans. Comput., vol. C-32, no. 9, pp. 799-808, Sept. 1983.
[76] Y. C. Chow and W. H. Kohler, "Models for dynamic load balancing in a heterogeneous multiple processor system,"IEEE Trans. Comput, vol. C-28, no. 5, pp. 354-361, May 1979.
[77] R. Manner, "Hardware task/processor scheduling in a polyprocessor environment,"IEEE Trans. Comput., vol. C-33, no. 7, pp. 626-636, July 1984.
[78] L. M. Ni and K. Abani, "Nonpreemptive load balancing in a class of local area networks," inProc. Comput. Networking Symp., Dec. 1981, pp. 113-118.
[79] M. L. Powell and B. P. Miller, "Process migration in DEMOS/MP," inProc. Ninth Symp. Oper. Syst. Principles, Bretton Woods, NH, Oct. 1983, pp. 110-119.
[80] J. A. Stankovic, "Simulations of three adaptive, decentralized controlled, job scheduling algorithms,"Comput. Networks, pp. 199-217, June 1984.
[81] N. Deo,Graph Theory with Applications to Engineering and Computer Science. New Delhi, India: Prentice-Hall of India, 1987.
[82] V. Chaudhary and J. K. Aggarwal, "On the complexity of parallel image component labeling," inProc. Int. Conf. Parallel Processing, Aug. 1991, pp. 183-187.
[83] V. Chaudhary and J. K. Aggarwal, "Parallelism in computer vision-A review," inParallel Algorithms for Machine Intelligence and Computer Vision, V. Kumar and P. Gopalakrishnan and L. Kanal, Eds. New York: Springer-Verlag, 1990, pp. 271-309.
[84] V. Chaudhary and J. K. Aggarwal, "Parallelism in low-level vision-A review," inData Analysis in Astronomy III, V. De Gesu, L. Scarsi, P. Crane, J. H. Friedman, S. Levialdi, and M. C. Maccarone, New York: Plenum, 1988, pp. 255-270.
[85] M. H. Sunwoo, B. S. Baroody, and J. K. Aggarwal, "A parallel algorithm for region labeling," inProc. IEEE Workshop Comput. Architecture for Pattern Anal. Mach. Intell., 1987, pp. 27-34.
[86] I. Herstein,Abstract Algebra, New York: Wiley, 1984, 212 pp.
[87] Y. C. Kim and J. K. Aggarwal, "Positioning three dimensional objects using stereo images,"IEEE J. Robotics Automat., vol. RA-3, no. 6, pp. 361-373, Aug. 1987.

Index Terms:
Index Termsgeneralized scheme; mapping parallel algorithms; graph theory; mathematicalprogramming; heuristics; graphical representation; communication overhead; optimality;simulation; distributed environments; graph theory; mathematical programming; parallelalgorithms
Citation:
V. Chaudhary, J.K. Aggarwal, "A Generalized Scheme for Mapping Parallel Algorithms," IEEE Transactions on Parallel and Distributed Systems, vol. 4, no. 3, pp. 328-346, March 1993, doi:10.1109/71.210815
Usage of this product signifies your acceptance of the Terms of Use.