This Article 
 Bibliographic References 
 Add to: 
Unifying and Optimizing Parallel Linear Algebra Algorithms
December 1993 (vol. 4 no. 12)
pp. 1382-1397

Two issues in linear algebra algorithms for multicomputers are addressed. First, how tounify parallel implementations of the same algorithm in a decomposition-independent way. Second, how to optimize naive parallel programs maintaining the decompositionindependence. Several matrix decompositions are viewed as instances of a more generalallocation function called subcube matrix decomposition. By this meta-decomposition, aprogramming environment characterized by general primitives that allow one to designmeta-algorithms independently of a particular decomposition. The authors apply such aframework to the parallel solution of dense matrices. This demonstrates that most of theexisting algorithms can be derived by suitably setting the primitives used in themeta-algorithm. A further application of this programming style concerns the optimization of parallel algorithms. The idea to overlap communication and computation has been extended from 1-D decompositions to 2-D decompositions. Thus, a first attempt towards a decomposition-independent definition of such optimization strategies is provided.

[1] M. Angelaccio and M. Colajanni, "Subcube matrix decomposition: A unifying view for LU factorization on MIMD multiprocessors," Res. Rep. RI.91.04, Univ. Rome "Tor Vergata," Italy, Sept. 1991; also to appear inParallel Comput.
[2] R. Bisseling and J. van de Vorst, "Parallel LU decomposition on a transputer network," inProc. Shell Conf. Parallel Comput., Lecture Notes Comput. Sci. Berlin: Springer-Verlag, 1989.
[3] E. Chu and A. George, "Gaussian elimination with partial pivoting and load balancing on a multiprocessor,"Parallel Comput., vol. 5, pp. 65-74, 1987.
[4] M. Cosnard, B. Tourancheau, and G. Villard, "Gaussian elimination on message passing architecture," inProc. Supercomput., 1987, Athens, Lecture Notes Comput. Sci. New York: Springer-Verlag, 1987.
[5] G. Davis, "Column LU factorization with pivoting on a hypercube multiprocessor,"SIAM J. Algebraic Discrete Methods, vol. 7, 1986.
[6] J. Dongarra, R. van de Geijn, and D. Walker, "A look at scalable dense linear algebra libraries," inProc. Scalable High Performance Comput. Conf. SHPCC-92. Williamsburg, VA: IEEE Computer Society Press, Apr. 1992.
[7] M. Foxet al., Solving Problems on Concurrent Processors, vol. 1. Englewood Cliffs, NJ: Prentice-Hall, 1988.
[8] G. C. Fox, S. Hiranandani, K. Kennedy, C. Koelbel, U. Kremer, C. W. Tseng, and M. Y. Wu, "Fortran D language specification," Rice COMP TR90-141, Rice Univ., Dec. 1990.
[9] K. A. Gallivan, M. T. Heath, E. Ng, J. M. Ortega, B. W. Peyton, C. H. Romine, A. H. Sameh, and R. G. Voigt,Parallel Algorithms for Matrix Computations. Philadelphia, PA: SIAM, 1990.
[10] G. A. Geist and C. H. Romine, "LU factorization algorithms on distributed-memory multiprocessor architectures,"SIAM J. Sci. Statist. Comput., vol. 9, no. 4, pp. 639-649, 1988.
[11] M.T. Heath and C. H. Romine, "Parallel solution of triangular systems on distributed-memory multiprocessors,"SIAM J. Sci. Statist. Comput., vol. 9, no. 3, pp. 558-600, 1988.
[12] S. Hiranandani, K. Kennedy, and C.-W. Tseng, "Compiling Fortran D for MIMD distributed-memory machines,"Commun. ACM, vol. 35, no. 8, Aug. 1992.
[13] G. Li and T. F. Coleman, "A parallel triangular solver for a distributed memory multiprocessor,"SIAM J. Sci. Statist. Comput., vol. 9, no. 3, pp. 485-502, 1988.
[14] D. O'Leary and G. Stewart, "Data-flow algorithms for parallel matrix computations,"Commun. ACM, vol. 28, pp. 840-853, 1985.
[15] J. M. Ortega,Introduction to Parallel and Vector Solution of Linear Systems. New York: Plenum, 1988.
[16] J. M. Ortega and C. H. Romine, "The ijk forms of factorization methods II. Parallel systems,"Parallel Comput., vol. 7, pp. 149-162, 1988.
[17] Y. Robert,The Impact of Vector and Parallel Architectures on the Gaussian Elimination Algorithm. Manchester, U.K.: Manchester University Press, 1990.
[18] C. H. Romine and J. M. Ortega, "Parallel solution of triangular systems of equations,"Parallel Comput., vol. 6, pp. 109-114, 1988.
[19] Y. Saad, "Gaussian elimination on hypercubes," Yale Tech. Rep. YALEU/DCS/RR-462, Mar. 1986.

Index Terms:
Index Termsparallel linear algebra algorithms; linear algebra algorithms; multicomputers; parallelimplementations; parallel programs; subcube matrix decomposition; meta-algorithm;communication; computation; decomposition-independent definition; optimization; linearalgebra; matrix algebra; optimisation; parallel algorithms
M. Angelaccio, M. Colajanni, "Unifying and Optimizing Parallel Linear Algebra Algorithms," IEEE Transactions on Parallel and Distributed Systems, vol. 4, no. 12, pp. 1382-1397, Dec. 1993, doi:10.1109/71.250119
Usage of this product signifies your acceptance of the Terms of Use.