This Article 
 Bibliographic References 
 Add to: 
Scalability of Parallel Algorithm-Machine Combinations
June 1994 (vol. 5 no. 6)
pp. 599-613

Scalability has become an important consideration in parallel algorithm and machinedesigns. The word scalable, or scalability, has been widely and often used in the parallelprocessing community. However, there is no adequate, commonly accepted definition ofscalability available. Scalabilities of computer systems and programs are difficult toquantify, evaluate, and compare. In this paper, scalability is formally defined foralgorithm-machine combinations. A practical method is proposed to provide a quantitative measurement of the scalability. The relation between the newly proposed scalability and other existing parallel performance metrics is studied. A harmony between speedup and scalability has been observed. Theoretical results show that a large class ofalgorithm-machine combinations is scalable and the scalability can be predicted throughpremeasured machine parameters. Two algorithms have been studied on an nCUBE 2multicomputer and on a MasPar MP-1 computer. These case studies have shown howscalabilities can be measured, computed, and predicted. Performance instrumentation andvisualization tools also have been used and developed to understand the scalabilityrelated behavior.

[1] J. Gustafson, G. Montry, and R. Benner, "Development of parallel methods for a 1024-processor hypercube,"SIAM J. SSTC, vol. 9, July 1988.
[2] J. L. Gustafson, "Re-evaluating Amdahl's Law,"Commun. ACM, vol. 31, no. 5, pp. 532-533, 1988.
[3] V. Kumar and A. Gupta, "Analysis of scalability of parallel algorithms and architectures: A survey," inProc. Int. Conf. on Supercomputing, June 1991.
[4] V. Kumar and V. Singh, "Scalability of parallel algorithms for the allpairs shortest path problem: A summary of results," inProc. of Conf. on Parallel Processing, Chicago, IL, 1990, pp. III 136-140.
[5] M. Willebeek-LeMair, A. P. Reeves, and C. H. Ning, "Characterization of multicomputer system: a transfer ration approach," inProc. Int. Conf. on Parallel Processing, Chicago, IL, 1990, pp. II 171-178.
[6] D. Nussbaum and A. Agarwal, "Scalability of Parallel Machines,"Comm. ACM, Vol. 34, No. 3, Mar. 1991, pp. 57-61.
[7] D. Rover, V. Tsai, Y. Chow, and J. Gustafson, "Signal processing algorithms on parallel architectures: A performance update,"J. Parallel Distrib. Computing, vol. 13, pp. 237-245, Nov. 1991.
[8] C. M. Goral, K. E. Torrance, D. P. Greenberg, and B. Battaile, "Modeling the interaction of light between diffuse surfaces," inProc. SIGGRAPH '84, 1984, pp. 213-222.
[9] X.-H. Sun and L.M. Ni, "Another View of Parallel Speedup,"Proc. Supercomputing '90, IEEE Computer Soc. Press, Los Alamitos, Calif., 1990, pp. 324-333.
[10] X.-H. Sun and L. Ni, "Scalable Problems and Memory-Bound Speedup,J. Parallel and Distributed Computing, Vol. 19, No. 1, Sept. 1993, pp. 21-37.
[11] G. Amdahl, "Validity of the single-processor approach to achieving large scale computing capabilities," inProc. AFIPS Conf., 1967, pp. 483-485.
[12] X.-H. Sun, L. M. Ni, F. A. Salam, and S. Guo, "Compute-exchange computation for solving power flow problems: The model and application," inProc. 4th SIAM Conf. Parallel Processing for Scientific Computing, J. Dongarra, P. Messina, D. C. Sorensen, and R. G. Voigt, Eds., Dec. 1989, pp. 198-203.
[13] R. W. Hackney, "Parametrization of computer performance,"Parellel Computing, vol. 5, 1987.
[14] X.-H. Sun and J. Gustafson, "Toward a better parallel performance metric,"Parallel Computing, vol. 17, Dec. 1991.
[15] J. Gustafson, D. Rover, S. Elbert, and M. Carter, "The design of a scalable, fixed-time computer benchmark,"J. Parallel Distrib. Computing, vol. 11, Aug. 1991.
[16] G. Geist, M. Heath, B. Peyton, and P. Worley, "A machine-independent communication library," inProc. 4th Conf. Hypercubes, Concurrent Computers, and Applications, 1989, pp. 565-568.
[17] M. T. Heath and J. A. Etheridge, "Visualizing the performance of parallel programs,"IEEE Software, vol. 8, pp. 29-39, Sept. 1991.

Index Terms:
Index Termsparallel algorithms; parallel machines; performance evaluation; software metrics; parallelalgorithm; scalability; algorithm-machine combinations; parallel machine; quantitativemeasurement; parallel performance metrics; nCUBE 2; MasPar MP-1; case studies
X.H. Sun, D.T. Rover, "Scalability of Parallel Algorithm-Machine Combinations," IEEE Transactions on Parallel and Distributed Systems, vol. 5, no. 6, pp. 599-613, June 1994, doi:10.1109/71.285606
Usage of this product signifies your acceptance of the Terms of Use.