This Article 
 Bibliographic References 
 Add to: 
Data Management and Control-Flow Aspects of an SIMD/SPMD Parallel Language/Compiler
February 1993 (vol. 4 no. 2)
pp. 222-234

Features of an explicitly parallel programming language targeted for reconfigurable parallel processing systems, where the machine's N processing elements (PEs) are capable of operating in both the SIMD and SPMD modes of parallelism, are described. The SPMD (single program-multiple data) mode of parallelism is a subset of the MIMD mode where all processors execute the same program. By providing all aspects of the language with an SIMD mode version and an SPMD mode version that are syntactically and semantically equivalent, the language facilitates experimentation with and exploitation of hybrid SIMD/SPMD machines. Language constructs (and their implementations) for data management, data-dependent control-flow, and PE-address-dependent control-flow are presented. These constructs are based on experience gained from programming a parallel machine prototype and are being incorporated into a compiler under development. Much of the research presented is applicable to general SIMD machines and MIMD machines.

[1] Ames Research Center,CFD: a FORTRAN-based language for Illiac IV, NASA, 1974.
[2] M. Auguin and F. Boeri, "The OPSILA computer," inParallel Languages and Architectures, M. Consard, Ed. Amsterdam, The Netherlands, Elsevier Science, 1986, pp. 143-153.
[3] M. Auguin, F. Boeri, J. P. Dalban, and A. Vincent-Carrefour, "Experience using a SIMD/SPMD multiprocessor architecture,"Microprocessing Microprogramming, vol. 21, pp. 171-177, Aug. 1987.
[4] G. H. Barnes, R. Brown, M. Kato, D. J. Kuck, D. L. Slotnick, and R. A. Stokes, "The Illiac IV computer,"IEEE Trans. Comput., vol. C-17, no. 8, pp. 746-757, Aug. 1968.
[5] K. E. Batcher, "Design of a massively parallel processor,"IEEE Trans. Comput., vol. C-29, no. 9, pp. 836-844, Sept. 1980.
[6] K. E. Batcher, "Bit serial parallel processing systems,"IEEE Trans. Comput., vol. C-31, no. 5, pp. 377-384, May 1982.
[7] Berg, T.B., and H.J. Siegel, "Instruction Execution Trade-offs for SIMD versus MIMD versus Mixed-Mode Parallelism,"Proc. Fifth Int'l Parallel Processing Symp., IEEE CS Press, Los Alamitos, Calif., Order No. 2167, 1991, pp. 301-308.
[8] T. Blank, "The MasPar MP-1 Architecture," inProc. 35th IEEE Comput. Soc. Int. Conf.--Spring Compcon 90, San Francisco, CA, Feb. 1990.
[9] E. C. Bronson, T. L. Casavant, and L. H. Jamieson, "Experimental application-driven architecture analysis of an SIMD/MIMD parallel processing system."IEEE Trans. Parallel Distributed Syst., vol. 1, no. 2, pp. 195-205, Apr. 1990.
[10] J. C. Browne, A. R. Tripathi, S. Fedak, A. Adiga, and R. N. Kapur, "A language for specification and programming of reconfigurable parallel computation structures," inProc. 1982 Int. Conf. Parallel Processing, Aug. 1982, pp. 142-149.
[11] P. Christy, "Software to support massively parallel computing on the MasPar MP-1," inProc. IEEE Compcon, Feb. 1990, pp. 29-33.
[12] C. L. Cline and H. J. Siegel, "Augmenting Ada for SIMD parallel processing,"IEEE Trans. Software Eng., vol. SE-11, no. 9, pp. 970-977, Sept. 1985.
[13] W. Crowther, J. Goodhue, R. Thomas, W. Milliken, and T. Blackadar, "Performance measurements on a 128-node butterfly parallel processor," inProc. 1985 Int. Conf. Parallel Processing, Aug. 1985, pp. 531-540.
[14] F. Darema-Rodgers, D. A. George, V. A. Norton, and G. F. Pfister,Environment and System Interface for VM/EPEX, Res. Rep. RC11381 (#51260), IBM T. J. Watson Research Center, 1985.
[15] F. Darema, D. A. George, V. A. Norton, and G. F. Pfister, "A single-program-multiple-data computational model for EPEX/FORTRAN,"Parallel Comput., vol. 7, no. 1, pp. 11-24, Apr. 1988.
[16] P. Duclos et al., "Image Processing on a SIMD/SPMD Architecture: Opsila,"Proc. Ninth Int'l Conf. Pattern Recognition, IEEE CS Press, Los Alamitos, Calif., Order No. 878, 1988, pp. 430-433.
[17] S.A. Fineberg, T.L. Casavant, and H.J. Siegel, "Experimental Analysis of a Mixed-Mode Parallel Architecture Using Bitonic Sequence Sorting,"J. Parallel and Distributed Computing, Vol. 11, No. 3, Mar. 1991, pp. 239-251.
[18] M. J. Flynn, "Very high-speed computing systems,"Proc. IEEE, vol. 54, no. 12, pp. 1901-1909, Dec. 1966.
[19] R. Freund, "Optimal Selection Theory for Superconcurrency,"Proc. 89 Supercomputing, IEEE CS Press, Los Alamitos, Calif., Order No. M2021 (microfiche), 1989, pp. 13-17.
[20] J.P. Hayes and T.N. Mudge, "Hypercube Supercomputers,"Proc. IEEE, Vol. 77, No. 12, Dec. 1989, pp. 1,829-1,841.
[21] W. D. Hillis,The Connection Machine. Cambridge, MA: MIT Press, 1985.
[22] D. J. Hunt, "AMT DAP-A processor array in a workstation environment,"Comput. Syst. Sci. and Eng., vol. 4, no. 2, pp. 107-114, Apr. 1989.
[23] L. H. Jamieson, "Characterizing parallel algorithms," inThe Characteristics of Parallel Algorithms, L. H. Jamieson, D. B. Gannon, and R. J. Douglass, Eds. Cambridge, MA: M.I.T. Press, 1987, pp. 65-100.
[24] H.F. Jordan, "Structuring Parallel Algorithms in a MIMD, Shared-Memory Environment,"Parallel Computing, May 1986, pp. 93-110.
[25] H. F. Jordan, "The Force," inThe Characteristics of Parallel Algorithms, L. H. Jamieson, D. B. Gannon, and R. J. Douglass, Eds. Cambridge, MA: M.I.T. Press, 1987, pp. 395-436.
[26] B. W. Kernighan and D. M. Ritchie,The C Programming Language. Englewood Cliffs, NJ: Prentice-Hall, 1978.
[27] S. D. Kim, M. A. Nichols. and H. J. Siegel, "Modeling overlapped operation between the control unit and processing elements in an SIMD machine,"J. Parallel and Distributed Comput., Special Issue on Modeling of Parallel Computers, vol. 12, no. 4, pp. 329-342, Aug. 1991.
[28] J. T. Kuehn and H. J. Siegel, "Extensions to the C programming language for SIMD/MIMD parallelism," inProc. 1985. Int. Conf. Parallel processing, Aug. 1985, pp. 232-235.
[29] D. H. Lawrie, T. Layman, D. Baer, and J. M. Randall. "Glypnir-A programming language for Illiac IV,"Commun. ACM, vol. 18, no. 3, pp. 157-164, Mar. 1975.
[30] G.J. Lipovski and M. Malek,Parallel Computing: Theory and Comparisons, John Wiley&Sons, New York, 1987.
[31] P. Mehrotra and J. Van Rosendale, "The BLAZE language: A parallel language for scientific programming,"Parallel Comput., vol. 5, no. 3, pp. 339-361, Nov. 1987.
[32] W. G. Nation, S. A. Fineberg, M. D. Allemang, T. Schwederski. T. L. Casavant, and H. J. Siegel, "Efficient masking techniques for large-scale SIMD architectures," inProc. Frontiers '90: Third Symp. Frontiers Massively Parallel Computation. Oct. 1990, pp. 259-264.
[33] M. A. Nichols, H. J. Siegel, H. G. Dietz, R. W. Quong, and W. G. Nation, "Eliminating memory fragmentation within partitionable SIMD/SPMD machines,"IEEE Trans. Parallel Distributed Systems, Special Issue on Parallel Languages and Compilers, vol. 2, no. 3, pp. 290-303, July 1991.
[34] M. A. Nichols, H. J. Siegel, and H. G. Dietz, "Execution mode management in an SIMD/SPMD parallel language/compiler," inProc. COMPSAC '91: Fifteenth Annu. Int. Comput. Software and Appl. Conf., Sept. 1991, pp. 392-397.
[35] S. F. Nugent, "The iPSC/2 Direct-Connect communications technology," inProc. Third Conf. Hypercube Comput. Appl., Pasadena, CA, Jan. 1988, pp. 56-60.
[36] T. J. Parr, H. G. Dietz, and W. E. Cohen, "Purdue Compiler-Construction Tool Set," Tech. Rep. TR-EE 90-14, School of Elec. Eng., Purdue Univ., 1990.
[37] R. H. Perrott, R. W. Lyttle, and P. S. Dhillon, "The design and implementation of a Pascal-based language for array processor architectures."J. Parallel Distributed Comput., vol. 4, no. 3, pp. 266-287, June 1987.
[38] R. H. Perrott. "A language for array and vector processors."ACM Trans. Programming Languages and Syst., vol. 1, no. 2, pp. 177-195, Oct. 1979.
[39] R.H. Perrott,Parallel Programming, Addison-Wesley, Reading, Mass., 1987.
[40] M. J. Phillip and H. G. Dietz, "Toward semantic self-consistency in explicitly parallel languages," inProc. Fourth Int. Conf. Supercomput., May 1989, pp. 398-407.
[41] M. J. Phillip, "Unification of synchronous and asynchronous models for parallel programming languages," M.S.E.E. thesis, School of Elec. Eng., Purdue Univ., 1989.
[42] A. P. Reeves, "Parallel Pascal: An extended Pascal for parallel computers,"J. Parallel Distributed Comput., vol. 1, no. 1, pp. 64-80, Aug. 1984.
[43] M. D. Rice, S. B. Seidman, and P. Y. Wang. "A high-level language for SIMD computation," inProc. CONPAR 88, Sept. 1988, pp. 384-391.
[44] J. R. Rose and G. L., Jr. Steele, "C*: An extended C language for data parallel programming." inProc. Second Int. Conf. Supercomput., vol. 2, May 1987, pp. 2-16.
[45] T. Schwederski, W. G. Nation, H. J. Siegel, and D. G. Meyer, "Design and implementation of the PASM prototype control hierarchy." inProc. Second Int. Conf. Supercomput., vol. I, May 1987, pp. 418-427.
[46] M. C. Sejnowski, E. T. Upchurch, R. N. Kapur, D. P. S. Charlu, and G. J. Lipovski, "An overview of the Texas Reconfigurable Array Computer." inProc. AFIPS 1980 Nat. Comput. Conf., June 1980, pp. 631-641.
[47] H. J. Siegel, J. B. Armstrong, and D. W. Watson, "Mapping computer-vision-related tasks onto reconfigurable parallel processing systems,"IEEE Comput. Mag., Special issue on Parallel Processing for Computer Vision and Image Understanding, vol. 25, no. 2, pp. 54-63, Feb. 1992.
[48] H. J. Siegel, "Analysis techniques for SIMD machine interconnection networks and the effects of processor address masks,"IEEE Trans. Comput., vol. C-26, no. 2, pp. 153-161, Feb. 1977.
[49] H. J. Siegel. "A model of SIMD machines and a comparison of various interconnection networks,"IEEE Trans Comput., vol. C-28, no. 12, pp. 907-917, Dec. 1979.
[50] H. J. Siegel,Interconnectron Networks for Large-Scale Parallel Processing: Theory and Case Studies, second ed. New York: McGraw-Hill, 1990.
[51] H. J. Siegel, L. J. Siegel, F. C. Kemmerer, P. T. Mueller, Jr., H. E. Smalley, Jr., and S. D. Smith, "PASM: A partitionable SIMD/MIMD system for image processing and pattern recognition,"IEEE Trans. Comput., vol. C-30, no. 12, pp. 934-947, Dec. 1981.
[52] H. J. Siegel, T. Schwederski, J. T. Kuehn, and N. J. Davis IV, "An overview of the PASM parallel processing system," inComputer Architecture, D. D. Gajski. V. M. Milutinovic, H. J. Siegel, and B. P. Furht, Eds. Washington. DC: IEEE Computer Society Press, 1987, pp. 387-407.
[53] R. H. Thomas and W. Crowther,The Uniform System: An approach to runtime support for large scale shared memory parallel processors, BBN Advanced Computers, Inc., Cambridge, MA, 1987.
[54] L. W. Tucker and G. G. Robertson, "Architecture and applications of the Connection Machine"IEEE Comput. Mag., vol. 21, no. 8, pp. 26-38, Aug. 1988.
[55] P. Y. Wang, S. B. Seidman, M. D. Rice, and T. E. Gerasch, "An objectmethod programming language for data parallel computation," inProc. 22nd Hawaii Int. Conf. Syst. Sci., vol. 2, Jan. 1989, pp. 745-750.

Index Terms:
Index Termsdata management; control-flow aspects; SIMD/SPMD parallel language/compiler; explicitly parallel programming language; reconfigurable parallel processing systems; processing elements; single program-multiple data; mode version; data-dependent control-flow; PE-address-dependent control-flow; parallel machine prototype; database management systems; parallel languages; parallel programming; program compilers
M.A. Nichols, H.J. Siegel, H.G. Dietz, "Data Management and Control-Flow Aspects of an SIMD/SPMD Parallel Language/Compiler," IEEE Transactions on Parallel and Distributed Systems, vol. 4, no. 2, pp. 222-234, Feb. 1993, doi:10.1109/71.207596
Usage of this product signifies your acceptance of the Terms of Use.