This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Lazy Task Creation: A Technique for Increasing the Granularity of Parallel Programs
July 1991 (vol. 2 no. 3)
pp. 264-280

When a parallel algorithm is written naturally, the resulting program often produces tasks of a finer grain than an implementation can exploit efficiently. Two solutions to the granularity problem that combine parallel tasks dynamically at runtime are discussed. The simpler load-based inlining method, in which tasks are combined based on dynamic bad level, is rejected in favor of the safer and more robust lazy task creation method, in which tasks are created only retroactively as processing results become available. The strategies grew out of work on Mul-T, an efficient parallel implementation of Scheme, but could be used with other languages as well. Mul-T implementations of lazy task creation are described for two contrasting machines, and performance statistics that show the method's effectiveness are presented. Lazy task creation is shown to allow efficient execution of naturally expressed algorithms of a substantially finer grain than possible with previous parallel Lisp systems.

[1] A. Agarwalet al., "The MIT Alewife machine: A large-scale distributedmemory multiprocessor," inScalable Shared Memory Multiprocessors, M. DuBois and S. Thakkar, Eds. Boston, MA: Kluwer Academic, 1991.
[2] A. Agrawal et al., "APRIL: A Processor Architecture for Multiprocessing,"17th Int'l Symp. Computer Architectures, 1990, IEEE Computer Soc. Press, Los Alamitos, Calif., Order No. 2047, pp. 104-114.
[3] Arvind and D.E. Culler, "Dataflow Architectures," inAnnual Reviews in Computer Science, 1, Annual Reviews Inc., Palo Alto, Calif., 1986, pp. 225-253.
[4] G. Bilardi and A. Nicolau, "Adaptive bitonic sorting: An optimal parallel algorithm for shared-memory machines,"SIAM J. Computing, vol. 18, no. 2, pp. 216-228, 1989.
[5] A. Agarwal et al., "Limitless Directories: A Scalable Cache Coherence Scheme,"Proc. Fourth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, ACM, New York, 1991, pp. 224-234.
[6] D. E. Culler, "Managing parallelism and resources in scientific dataflow programs," Ph.D. dissertation, M.I.T. Dep. Elec. Eng. Comput. Sci., Cambridge, MA, June 1989.
[7] R.P. Gabriel and J. McCarthy, "Queue-Based Multiprocessing Lisp,"Conf. Record 1984 ACM Conf. Lisp and Functional Programming, ACM, New York, 1984, pp. 25-44.
[8] B. Goldberg, "Multiprocessor execution of functional programs,"Int. J. Parallel Programming, vol. 17, no. 5, pp. 425-473, Oct. 1988.
[9] R. Goldman and R.P. Gabriel, "Preliminary Results with the Initial Implemention of Qlisp,"Proc. ACM Symp. Lisp and Functional Programming, ACM, New York, 1988, pp. 143-152.
[10] R. Goldman, R. Gabriel, and C. Sexton, "Qlisp: An interim report," inProc. U.S./Japan Workshop Parallel Lisp, (Springer-Verlag Lecture Notes in Computer Science 441), T. Ito and R. Halstead, Eds., Sendai, Japan, June 1989, pp. 161-181.
[11] J. R. Gurd, C. C. Kirkham, and I. Watson, "The Manchester prototype dataflow computer,"Commun. ACM, vol. 28, no. 1, pp. 34-52, Jan. 1985.
[12] M. Feeley, "Fine grain parallelism in Multilisp on a shared memory MIMD computer," Ph.D. dissertation, Brandeis Univ., in preparation.
[13] R. Halstead, "Multilisp: A Language for Concurrent Symbolic Computation,"ACM Trans. Programming Languages and Systems, Vol. 7, No. 4, Oct. 1985, pp. 501-538.
[14] R. Halstead, "An assessment of Multilisp: Lessons from experience,"Int. J. Parallel Programming, vol. 15, no. 6, pp. 459-501, Dec. 1986.
[15] R. W. Hockney and C. R. Jesshope,Parallel Computers 2. Bristol, England: Adam Hilger, 1988, pp. 475-483.
[16] P. Hudak and B. Goldberg, "Serial combinators: 'Optimal' grains of parallelism," inFunctional Programming Languages and Computer Architecture, Springer-Verlag LNCS 201, Sept. 1985, pp. 382-388.
[17] D. Kranz, R. Halstead, and E. Mohr, "Mul-T, A high-performance Parallel Lisp," inProc. ACM SIGPLAN '89 Conf. Programming Language Design Implementation, Portland, OR, June 1989, pp. 81-90.
[18] D. Kranz et al., "Orbit: An Optimizing Compiler for Scheme,"Proc. SIGPlan '86 Symp. Compiler Construction, ACM, June 1986, pp. 219-233.
[19] D.A. Kranz,Orbit: An Optimizing Compiler for Scheme, doctoral dissertation, Yale Univ., New Haven, Conn., 1988.
[20] E. Mohr, D. Kranz, and R. Halstead, "Lazy task creation: A technique for increasing the granularity of parallel programs," inProc. ACM Symp. Lisp and Functional Programming, June 1990, pp. 185-197.
[21] E. Mohr, D. Kranz, and R. Halstead, "Lazy task creation: A technique for increasing the granularity of parallel programs," DEC Cambridge Research Lab Tech. Rep. CRL 90/7, Nov. 1990.
[22] J. E. B. Moss, "Managing stack frames in Smalltalk," inProc. SIGPLAN '87 Symp. Interpreters and Interpretive Techniques, June 1987, pp. 229-240.
[23] C. A. Ruggiero and J. Sargeant, "Control of parallelism in the Manchester dataflow computer,"Lecture Notes Comput. Sci., no. 274, pp. 1-15, 1987.
[24] G. Sabot,The Paralation Model. Cambridge, MA: M.I.T. Press, 1988.
[25] V. Sarkar and J. Hennessy, "Compile-time partitioning and scheduling of parallel programs," inProc. SIGPLAN Symp. Compiler Construction, July 1986, pp. 17-26.
[26] G.L. Steele, Jr., and W.D. Hillis, "Connection Machine Lisp: Fine-Grained Parallel Symbolic Processing,"Proc. 1986 ACM Conf. Lisp and Functional Programming, ACM, New York, 1986, pp. 279-297.
[27] M. Vandevoorde and E. Roberts, "WorkCrews: An abstraction for controlling parallelism,"Int. J. Parallel Programming, vol. 17, no. 4, pp. 347-366, Aug. 1988.
[28] R. C. Waters, "Series," inCommon Lisp: the Language, Second Ed., G. Steele, Jr., ed. Bedford, MA: Digital, pp. 923-955.
[29] J. D. Pehoushek and J. S. Weening, "Low-cost process creation and dynamic partitioning in Qlisp," inProc. U.S./Japan Workshop Parallel Lisp, Springer-Verlag Lecture Notes in Computer Science 441, T. Ito and R. Halstead, Eds., Sendai, Japan, June 1989, pp. 182-199.
[30] J. Weening, "Parallel execution of Lisp programs," Stanford Comput. Sci. Rep. STANCS-89-1265, June 1989.

Index Terms:
Index Termsparallel algorithm; granularity problem; parallel tasks; simpler load-based inlining method; dynamic bad level; robust lazy task creation method; processing results; Mul-T; parallel implementation; Scheme; performance statistics; naturally expressed algorithms; parallel Lisp systems; parallel algorithms; parallel languages; parallel programming
Citation:
E. Mohr, D.A. Kranz, R.H. Halstead, Jr., "Lazy Task Creation: A Technique for Increasing the Granularity of Parallel Programs," IEEE Transactions on Parallel and Distributed Systems, vol. 2, no. 3, pp. 264-280, July 1991, doi:10.1109/71.86103
Usage of this product signifies your acceptance of the Terms of Use.