This Article 
 Bibliographic References 
 Add to: 
Guaranteeing Good Memory Bounds for Parallel Programs
October 1996 (vol. 22 no. 10)
pp. 762-773

Abstract—The amount of memory required by a parallel program may be spectacularly larger then the memory required by an equivalent sequential program, particularly for programs that use recursion extensively. Since most parallel programs are nondeterministic in behavior, even when computing a deterministic result, parallel memory requirements may vary from run to run, even with the same data. Hence, parallel memory requirements may be both large (relative to memory requirements of an equivalent sequential program) and unpredictable.

Assume that each parallel program has an underlying sequential execution order that may be used as a basis for predicting parallel memory requirements. We propose a simple restriction that is sufficient to ensure that any program that will run in n units of memory sequentially can run in mn units of memory on m processors, using a scheduling algorithm that is always within a factor of two of being optimal with respect to time.

Any program can be transformed into one that satisfies the restriction, but some potential parallelism may be lost in the transformation. Alternatively, it is possible to define a parallel programming language in which only programs satisfying the restriction can be written.

[1] R.D. Blumofe and C.E. Leiserson, "Space-Efficient Scheduling of Multithreaded Computations," Proc. 25th Ann. ACM Symp. Theory of Computing, pp. 362-371,San Diego Calif., May 1993.
[2] R. Sethi, "Algorithms for Minimal-Length Schedules," Computer and Job-Shop Scheduling Theory, E.G. Coffman, Jr., ed., chapter 2, pp. 51-99. John Wiley&Sons, 1976.
[3] R.L. Graham, "Bounds on the Performance of Scheduling Algorithms," Computer and Job-Shop Scheduling Theory, E.G. Coffman, Jr., ed., chapter 5, pp. 165-227. John Wiley&Sons, 1976.
[4] D.L. Eager, J. Zahorian, and E.D. Lazowska, "Speedup versus Efficiency in Parallel Systems," IEEE Trans. Computers, vol. 38, no. 3, pp. 408-423, Mar. 1989.
[5] F.W. Burton, "Storage Management in Virtual Tree Machines," IEEE Trans. Computers, vol. 37, no. 3, Mar. 1988.
[6] R.D. Blumofe and C.E. Leiserson, "Scheduling Multithreaded Computations by Work Stealing," Proc. 35th Symp. Foundations of Computer Science, 1994.
[7] P. Brinch Hansen, "Efficient Parallel Recursion," ACM SIGPLAN Notices, vol. 30, pp. 9-16, Dec. 1995.
[8] F.W. Burton and M.R. Sleep, "Executing Functional Programs on a Virtual Tree of Processors," Proc. Conf. Functional Programming Languages and Computer Architecture, pp. 187-194,Portsmouth N.H., Oct. 1981.
[9] C.A. Ruggiero and J. Sargeant, "Control of Parallelism in the Manchester Dataflow Machine," Functional Programming Languages and Computer Architecture, vol. 274, Lecture Notes Computer Science, pp. 1-15. Springer-Verlag, 1987.
[10] D.E. Culler and Arvind, "Resource Requirements of Dataflow Programs," Proc. Intl'l Symp. Computer Architecture, pp. 141-150, 1988.
[11] R.H. Halstead Jr., "Multilisp: A language for concurrent symbolic computation," ACM Trans. Programming Languages and Systems, vol. 7, pp. 501-538, Oct. 1985.
[12] S. Jagannathan and J. Philbin, "A Foundation for an Efficient Multithreaded Scheme System," Proc. 1992 ACM Conf. Lisp and Functional Programming, San Francisco, Calif., pp. 345-357, 1992.
[13] G. Blelloch, P. Gibbons, and Y. Matias, "Provably Efficient Scheduling for Languages with Fine-Grained Parallelism," Proc. Symp. Parallel Algorithms and Architectures, pp. 1-12, July 1995.

F. Warren Burton, "Guaranteeing Good Memory Bounds for Parallel Programs," IEEE Transactions on Software Engineering, vol. 22, no. 10, pp. 762-773, Oct. 1996, doi:10.1109/32.544353
Usage of this product signifies your acceptance of the Terms of Use.