This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Abstractions for Portable, Scalable Parallel Programming
January 1998 (vol. 9 no. 1)
pp. 71-86

Abstract—In parallel programming, the need to manage communication, load imbalance, and irregularities in the computation puts substantial demands on the programmer. Key properties of the architecture, such as the number of processors and the cost of communication, must be exploited to achieve good performance, but coding these properties directly into a program compromises the portability and flexibility of the code because significant changes are then needed to port or enhance the program. We describe a parallel programming model that supports the concise, independent description of key aspects of a parallel program—including data distribution, communication, and boundary conditions—without reference to machine idiosyncrasies. The independence of such components improves portability by allowing the components of a program to be tuned independently, and encourages reuse by supporting the composition of existing components. The isolation of architecture-sensitive aspects of a computation simplifies the task of porting programs to new platforms. Moreover, the model is effective in exploiting both data parallelism and functional parallelism. This paper provides programming examples, compares this work to related languages, and presents performance results.

[1] G. Alverson, W. Griswold, D. Notkin, and L. Snyder, "A Flexible Communication Abstraction for Non-Shared Memory Parallel Computing," Proc. Supercomputing '90, Nov. 1990.
[2] G. Alverson and D. Notkin, "Program structuring for effective parallel portability," Proc. IEEE Trans. Parallel and Distributed Systems, vol. 4, pp. 1,041-1,059, Sept. 1993
[3] T.S. Axelrod, P.F. Dubois, and P.G. Eltgroth, "A Simulator for MIMD Performance Prediction-Application to the S-1 MkIIa Multiprocessor," Proc. Int'l Conf. Parallel Processing, pp. 350-358, 1983.
[4] G.E. Blelloch, "NESL: A Nested Data-Parallel Language," Technical Report CMU-CS-92-103, School of Computer Science, Carnegie Mellon Univ., Jan. 1992.
[5] N. Carriero and D. Gelernter, "Linda in Context," Comm. ACM, vol. 32, no. 4, Apr. 1989, pp. 444-458.
[6] B. Chapman, P. Mehrotra, and H. Zima, "Vienna Fortran-A Fortran Language Extension for Distributed Memory Multiprocessors," Technical Report 91-72, Inst. for Computer Applications in Science and Engineering, Sept. 1990.
[7] W. Crowley, C.P. Hendrickson, and T.I. Luby, "The Simple Code," Technical Report UCID-17715, Lawrence Livermore Laboratory, 1978.
[8] D. Culler,R. Karp,D. Patterson,A. Sahay,K.E. Schauser,E. Santos,R. Subramonian,, and T. von Eicken,“LogP: Towards a realistic model of parallel computation,” Fourth Symp. Principles and Practices Parallel Programming, SIGPLAN’93, ACM, May 1993.
[9] D.E. Culler and Arvind, "Resource Requirements of Dataflow Programs," Proc. Intl'l Symp. Computer Architecture, pp. 141-150, 1988.
[10] D.E. Culler, A. Dusseau, S.C. Goldstein, A. Krishnamurthy, S. Lumetta, T. von Eicken, and K. Yelick, "Parallel Programming in Split-C," Supercomputing, 1993.
[11] T. Dunigan, "Hypercube Performance," Proc. Second Conf. on Hypercube Architectures, pp. 178-192, 1987.
[12] T. Dunigan, "Performance of the Intel iPSC/860 and NCUBE 6400 Hypercubes," Technical Report ONRL/TM-11790, Oak Ridge Nat'l Laboratory, 1991.
[13] K. Ekanadham and Arvind, "SIMPLE: Part I, An Exercise in Future Scientific Programming," Technical Report 273, MIT CSG, 1987.
[14] W. Fenton, B. Ramkumar, V. Saletore, A. Sinha, and L. Kale, "Supporting Machine Independent Programming on Diverse Parallel Architectures," Proc. Int'l Conf. Parallel Processing, vol. II, pp. 193-201, 1991.
[15] J. Feo, D. Cann, and R. Oldehoeft,“A report on the Sisal language project,”J. Parallel, Distrib. Comput., vol. 10, pp. 349–365, Dec. 1990.
[16] D. Gannon and J. Panetta, "SIMPLE on a CHiP," Technical Report 469, Computer Science Dept., Purdue Univ., 1984.
[17] D. Gannon and J. Panetta, "Restructuring Simple for the CHiP Architecture," Parallel Computing, vol. 3, pp. 305-326, 1986.
[18] K. Gates, "SIMPLE: An Exercise in Programming in Poker," technical report, Applied Mathematics Dept., Univ. of Washington, 1989.
[19] W. Griswold, G. Harrison, D. Notkin, and L. Snyder, "Scalable Abstractions for Parallel Programming," Proc. Fifth Distributed Memory Computing Conf., Charleston, S.C., 1990.
[20] R.H. Halstead Jr., "Multilisp: A language for concurrent symbolic computation," ACM Trans. Programming Languages and Systems, vol. 7, pp. 501-538, Oct. 1985.
[21] P.J. Hatcher, M.J. Quinn, R.J. Anderson, A.J. Lapadula, B.K. Seevers, and A.F. Bennett, "Architecture-Independent Scientific Programming in Dataparallel C: Three Case Studies," Proc. Supercomputing '91, pp. 208-217, 1991.
[22] High Performance Fortran Specification, High Performance Fortran Forum, Nov. 1994.
[23] R.E. Hiromoto, O.M. Lubeck, and J. Moore, "Experiences with the Denelcor HEP," Parallel Computing, vol. 1, pp. 197-206, 1984.
[24] T.J. Holman, "Processor Element Architecture for Non-Shared Memory Parallel Computers," PhD thesis, Dept. of Computer Science, Univ. of Washington, 1988.
[25] iPSC/2 User's Guide. Intel Corp., Oct. 1989.
[26] C. Koelbel and P. Mehrotra, "Compiling Global Name-Space Parallel Loops for Distributed Execution," IEEE Trans. Parallel and Distributed Systems, vol. 2, no. 10, pp. 440-451, Oct. 1991.
[27] S.R. Kohn and S.B. Baden, "Lattice Parallelism: A Parallel Programming Model for Non-Uniform, Structured Scientific Computations," Technical Report CS92-261, Dept. of Computer Science and Eng., Univ. of California, San Diego, Sept. 1992.
[28] S.R. Kohn and S.B. Baden, "An Implementation of the LPAR Parallel Programming Model for Scientific Computation," Proc. Sixth SIAM Conf. Parallel Processing for Scientific Computing, Mar. 1993.
[29] M.S. Lam and M.C. Rinard, "Coarse-Grained Parallel Programming in Jade," Third ACM SIGPLAN Symp. Principles and Practice of Parallel Programming, Apr. 1991.
[30] C. Lin, "The Portability of Parallel Programs Across MIMD Computers," PhD thesis, Dept. of Computer Science and Eng., Univ. of Washington, 1992.
[31] C. Lin, "ZPL Reference Manual," Technical Report 94-10-06, Dept. of Computer Science and Eng., Univ. of Washington, 1994.
[32] C. Lin and L. Snyder, "A Portable Implementation of SIMPLE," Int'l J. Parallel Programming, vol. 20, no. 5, pp. 363-401, 1991.
[33] C. Lin and L. Snyder, "Portable Parallel Programming: Cross Machine Comparisons for SIMPLE," Proc. Fifth SIAM Conf. Parallel Processing, 1991.
[34] C. Lin and L. Snyder, "Data Ensembles in Orca C," Languages and Compilers for Parallel Computing, U. Banerjee, D. Gelernter, A. Nicolau, and D. Padua, eds., pp. 112-123. Springer-Verlag, 1993.
[35] C. Lin and L. Snyder, "ZPL: An Array Sublanguage," Languages and Compilers for Parallel Computing, U. Banerjee, D. Gelernter, A. Nicolau, and D. Padua, eds., pp. 96-114. Springer-Verlag, 1993.
[36] C. Lin and L. Snyder, "Accommodating Polymorphic Data Decompositions in Explicity Parallel Programs," Proc. Eighth Int'l Parallel Processing Symp., pp. 68-74, 1994.
[37] C. Lin and L. Snyder, "Simple Performance Results in ZPL," Languages and Compilers for Parallel Computing, K. Pingali et al., eds., Springer-Verlag, Berlin, 1994, pp. 361-375.
[38] C. Lin and W.D. Weathersby, "Towards a Machine-Independent Solution of Sparse Cholesky Factorization," Proc. Parallel Computing '93, 1993.
[39] P. Mehrotra and J. Rosendale, "Compiling High Level Constructs to Distributed Memory Architectures," Technical Report ICASE Report 89-20, Inst. for Computer Applications in Science and Eng., Mar. 1989.
[40] J.M. Meyers, "Analysis of the SIMPLE Code for Dataflow Computation," Technical Report MIT/LCS/TR-216, MIT, 1979.
[41] D. Notkin et al., "Experiences with Poker," Proc. ACM Sigplan Symp. Parallel Programming: Experience with Applications, Languages, and Systems, ACM Press, New York, 1988.
[42] K. Pingali and A. Rogers, "Compiler Parallelization of SIMPLE for a Distributed Memory Machine," Technical Report 90-1084, Cornell Univ., 1990.
[43] J. Rose and G.L. Steele Jr., "C*: An Extended Language for Data Parallel Programming," Proc. Second Int'l Conf. Supercomputing, Mar. 1987.
[44] M. Rosing, R. Schnabel, and R. Weaver, "The Dino Parallel Programming Language," Technical Report CU-CS-457-90, Dept. of Computer Science, Univ. of Colorado, Apr. 1990.
[45] L. Snyder, "Type Architecture, Shared Memory, and the Corollary of Modest Potential," Ann. Rev. Computer Science, Annual Reviews, Inc., Palo Alto, Calif., 1986, pp. 289-318.
[46] L. Snyder, "The XYZ Abstraction Levels of Poker-like Languages," Languages and Compilers for Parallel Computing, D. Gelernter, A. Nicolau, and D. Padua, eds., pp. 470-489. MIT Press, 1990.
[47] L. Snyder, "A ZPL Programming Guide," Technical Report 94-12-02, Dept. of Computer Science and Eng., Univ. of Washington, 1994.
[48] L.G. Valiant, “A Bridging Model for Parallel Computation,” Comm. ACM, vol. 33, no. 8, pp. 103-111, Aug. 1990.
[49] J. Wiggs, "A Parallel Implementation of the Car-Parrinello Method," technical report general exam, Dept. of Chemistry, Univ. of Washington, June 1993.

Index Terms:
MIMD, parallel, portable, programming model, scalable.
Citation:
Gail A. Alverson, William G. Griswold, Calvin Lin, David Notkin, Lawrence Snyder, "Abstractions for Portable, Scalable Parallel Programming," IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 1, pp. 71-86, Jan. 1998, doi:10.1109/71.655246
Usage of this product signifies your acceptance of the Terms of Use.