This Article 
 Bibliographic References 
 Add to: 
Analysis of Processor Allocation in Multiprogrammed, Distributed-Memory Parallel Processing Systems
April 1994 (vol. 5 no. 4)
pp. 401-420

A main objective of scheduling independent jobs composed of multiple sequential tasks in shared-memory and distributed-memory multiprocessor computer systems is theassignment of these tasks to processors in a manner that ensures efficient operation ofthe system. Achieving this objective requires the analysis of a fundamental tradeoffbetween maximizing parallel execution, suggesting that the tasks of a job be spreadacross all system processors, and minimizing synchronization and communicationoverheads, suggesting that the job's tasks be executed on a single processor. Theauthors consider a class of scheduling policies that represent the essential aspects ofthis processor allocation tradeoff, and model the system as a distributed fork-joinqueueing system. They derive an approximation for the expected job response time,which includes the important effects of various parallel processing overheads (such astask synchronization and communication) induced by the processor allocation policy.

[1] F. Baccelli, W. A. Massey, and D. Towsley, "Acyclic fork/join queueing networks,"J. ACM, vol. 36, pp. 615-642, 1989.
[2] J. Bennett, J. Carter, and W. Zwaenepoel, "Munin: Distributed Shared Memory Based on Type-Specific Memory Coherence,"Proc. 1990 Conf. Principles and Practice of Parallel Programming, ACM Press, New York, N.Y., 1990, pp. 168-176.
[3] L. Brochard, "Efficiency of some parallel numerical algorithms on distributed systems,"Parallel Computing, vol. 12, pp. 21-44, 1989.
[4] L. Brochard, "Scalability, granularity and parallelism of numerical algorithms," Tech. Rep. RC-14786, IBM Res. Div., 1989.
[5] S. L. Brumelle, "Some inequalities for parallel-server queues,"Operations Res., vol. 19, pp. 402-413, 1971.
[6] M. L. Chaudhary and J. G. C. Templeton,A First Course in Bulk Queues. New York: Wiley, 1983.
[7] M. Crovella, P. Das, C. Dubnicki, T. LeBlanc, and E. Markatos, "Multiprogramming on multiprocessors," Tech. Rep., Dept. of Comput. Sci., Univ. of Rochester, NY, May 1991.
[8] R. Cytron, "Useful parallelism in a multiprocessing environment,"Proc. Int. Conf. Parallel Processing, 1985, pp. 450-457.
[9] H. A. David,Order Statistics, 2nd ed. New York: Wiley, 1981.
[10] L. W. Dowdy and M. R. Leuze, "On modeling partitioned multiprocessor systems,"Int. J. High Speed Computing, vol. 5, Sept. 1993.
[11] L. Durivault and R. D. Nelson, "An expression for the maximum of Erlang random variables," Tech. Rep. RC-16751, IBM Res. Div., Jan. 1991.
[12] K. Dussa, B. Carlson, L. Dowdy, and K.-H. Park, "Dynamic partitioning in transputer environments,"Proc. ACM SIGMETRICS Conf., 1990, pp. 203-213.
[13] D. Eager, E. Lazowska, and J. Zahorjan, "Adaptive load sharing in homogeneous distributed systems,"IEEE Trans. Software Eng., vol. SE-12, no. 5, pp. 662-675, May 1986.
[14] M. Foxet al., Solving Problems on Concurrent Processors, vol. 1. Englewood Cliffs, NJ: Prentice-Hall, 1988.
[15] D. Ghosal, G. Serazzi, and S. K. Tripathi, "The processor working set and its use in scheduling multiprocessor systems,"IEEE Trans. Software Eng., vol. 17, pp. 443-453, May 1991.
[16] D. Gross and C. Harris,Fundamentals of Queueing Theory, second ed. New York: John Wiley, 1985.
[17] B. Indurkhya, H. S. Stone, and L. Xi-Cheng, "Optimal partitioning of randomly generated distributed programs,"IEEE Trans. Software Eng., vol. SE-12, pp. 483-495, Mar. 1986.
[18] J. Keilson and A. Seidmann, "M/G/∞with batch arrivals,"Operations Res. Lett., pp. 219-222, Oct. 1988.
[19] L. Kleinrock,Queueing Systems: Applications, Vol. II. New York: Wiley, 1976.
[20] S.T. Leutenegger and M.K. Vernon, "The Performance of Multiprogrammed Multiprocessor Scheduling Policies,"Proc. 1990 ACM SIGMetrics Conf. Measurement and Modeling of Computer Systems, ACM Press, New York, 1990, pp. 226-236.
[21] K. Li and P. Hudak, "Memory Coherence in Shared Virtual Memory Systems,"ACM Trans. Computer Systems, Vol. 7, No. 4, Nov. 1989, pp. 321-359.
[22] S. Majumdar, D.L. Eager, and R. Bunt, "Scheduling in multiprogrammed parallel systems,"ACM SIGMETRICS, pp. 104-113, 1988.
[23] A. Makowski and R. D. Nelson, "Distributed parallelism considered harmful," Tech. Rep. RC-17448, IBM Res. Div., 1991.
[24] A. Makowski and R. D. Nelson, "Optimal scheduing for a distributed parallel processing model," Tech. Rep. RC-17449, IBM Res. Div., 1991.
[25] C. McCann, R. Vaswani, and J. Zahorjan, "A dynamic processor allocation policy for multiprogrammed shared memory multiprocessors,"ACM Trans. Comput. Syst., vol. 11, pp. 146-178, May 1993.
[26] N. H. Naik, V. K. Naik, and M. Nicoules, "Parallelization of implicit finite difference schemes in computational fluid dynamics," Tech. Rep. RC-16054, IBM Res. Div., 1990.
[27] V. K. Naik, S. K. Setia, and M. S. Squillante, "Performance analysis of job scheduling policies in parallel supercomputing environments,"Proc. Supercomputing'93, 1993, pp. 824-833.
[28] V. K. Naik, S. K. Setia, and M. S. Squillante, "Scheduling of large scientific applications on distributed memory multiprocessor systems,"Proc. 6th SIAM Conf. Parallel Processing Scientific Computing, 1993, pp. 913-922.
[29] R. D. Nelson and T. K. Philips, "An approximation for the mean response time for shortest queue routing with general interarrival and service times,"Performance Evaluation, vol. 17, pp. 123-139, 1993.
[30] R. D. Nelson and A. N. Tantawi, "Approximate analysis of fork-join synchronization,"IEEE Trans. Comput., vol. 37, pp. 739-743, June 1988.
[31] R. D. Nelson and D. Towsley, "A performance evaluation of several priority policies for parallel processing systems,"J. ACM, vol. 40, pp. 714-740, July 1993.
[32] R. D. Nelson, D. Towsley, and A. N. Tantawi, "Performance analysis of parallel processing systems,"IEEE Trans. Software Eng., vol. 14, pp. 532-540, Apr. 1988.
[33] M. F. Neuts,Matrix-geometric Solutions in Stochastic Models. Baltimore, MD: Johns Hopkins Univ. Press, 1981.
[34] J. K. Ousterhout, "Scheduling techniques for concurrent systems,"Proc. 3rd Int. Conf. Distrib. Computing Syst., 1982, pp. 22-30.
[35] C. D. Polychronopoulos, "Multiprocessing versus multiprogramming,"Proc. Int. Conf. on Parallel Processing, vol. II, 1989, pp. 223-230.
[36] S. M. Ross,Stochastic Processes. New York: Wiley, 1983.
[37] S. K. Setia, M. S. Squillante, and S. K. Tripathi, "Analysis of processor allocation in multiprogrammed parallel processing systems," Tech. Rep. RC-17712 (80337), IBM Res. Div., Nov. 1991; revised Sept. 1992.
[38] S. K. Setia and S. K. Tripathi, "A comparative analysis of static processor partitioning policies for parallel computers,"Proc. MASCOTS'93, Jan. 1993.
[39] S. K. Setia, "Scheduling on Multiprogrammed, Distributed Memory Parallel Computers," Tech. Rep. CS-TR-3169, Ph.D. dissertation, Dept. of Comput. Sci., Univ. of Md., College Park, MD, 1993.
[40] K. C. Sevcik, "Characterization of parallelism in applications and their use in scheduling,"ACM SIGMETRICS, pp. 171-180, 1989.
[41] H. Shore, "Simple approximations for the GI/G/c queue-I: The steady-state probabilities,"J. Operational Res. Soc., vol. 39, pp. 279-284, Mar. 1988.
[42] M. S. Squillante and E. D. Lazowska, "Using processor-cache affinity information in shared-memory multiprocessor scheduling,"IEEE Trans. Parallel Distrib. Syst., vol. 4, pp. 131-143, Feb. 1993.
[43] M. S. Squillante and R. D. Nelson, "Analysis of task migration in shared-memory multiprocessor scheduling," inProc. ACM SIGMETRICS Conf. Measurement and Modeling of Comput. Syst., May 1991, pp. 143-155.
[44] H. C. Tijms, M. H. Van Hoorn, and A. Federgruen, "Approximations for the steady state probabilities in the M/G/c queue,"Adv. Applied Probability, vol. 13, pp. 186-206, 1981.
[45] D. Towsley, C. G. Rommel, and J. Stankovic, "Analysis of fork-join program response times on multiprocessors,"IEEE Trans. Parallel Distrib. Syst., vol. 1, pp. 286-303, July 1990.
[46] A. Tucker and A. Gupta, "Process control and scheduling issues on a network of multiprocessors," inProc. Twelfth ACM Symp. Operating System Principles, Litchfielf Park, AZ, Dec. 1989, pp. 159-166.
[47] S. Varma, "Heavy and light traffic approximations for queues with synchronization constraints," Tech. Rep., Ph.D. dissertation, Dept. of Elec. Eng., Univ. of Md., College Park, MD, 1990.
[48] J. Zahorjan and C. McCann, "Processor scheduling in shared memory multiprocessors,"Proc. ACM Sigmetrics Conf., 1990, pp. 214-225.
[49] S. Zhou and T. Brecht, "processor pool-based scheduling for large-scale NUMA multiprocessors,"Proc. ACM Sigmetrics, 1991, pp. 133-142.

Index Terms:
Index Termsparallel architectures; shared memory systems; distributed memory systems; queueingtheory; scheduling; synchronisation; processor allocation; multiprogrammed;distributed-memory; parallel processing systems; shared-memory; scheduling;independent jobs; multiple sequential tasks; job response time; distributed fork-joinqueueing system; processor allocation policy; bulk arrival queues; communicationoverhead; distributed memory; fork-join queues; modeling and analysis;multiprogramming; operating systems; parallel processing; synchronization delay
S.K. Setia, M.S. Squillante, S.K. Tripathi, "Analysis of Processor Allocation in Multiprogrammed, Distributed-Memory Parallel Processing Systems," IEEE Transactions on Parallel and Distributed Systems, vol. 5, no. 4, pp. 401-420, April 1994, doi:10.1109/71.273047
Usage of this product signifies your acceptance of the Terms of Use.