This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Parallel Computational Model for Heterogeneous Clusters
December 2006 (vol. 17 no. 12)
pp. 1390-1400

Abstract—Heterogeneous clusters claim for new models and algorithms. In this paper, a new parallel computational model is presented. The model, based on the LogGP model, has been extended to be able to deal with heterogeneous parallel systems. For that purpose, the LogGP's scalar parameters have been replaced by vector and matrix parameters to take into account the different nodes' features. The work presented here includes the parametrization of a real cluster, which illustrates the impact of node heterogeneity over the model's parameters. Finally, the paper presents some experiments that can be used for assessing the method's validity, together with the main conclusions and future work.

[1] A. Aggarwal, A.K. Chandra, and M. Snir, “On Communication Latency in PRAM Computations,” Proc. ACM Symp. Parallel Algorithms and Architectures, pp. 11-21, June 1989, preliminary version.
[2] A. Aggarwal, A.K. Chandra, and M. Snir, “Communication Complexity of PRAMs,” Theoretical Computer Science, vol. 71, no. 1, pp. 3-28, Mar. 1990.
[3] A. Alexandrov, M. Ionescu, K.E. Schauser, and C. Scheiman, “LogGP: Incorporating Long Messages into the logP Model—One Step Closer towards a Realistic Model for Parallel Computation,” Proc. Seventh Ann. ACM Symp. Parallel Algorithms and Architectures (SPAA '95), pp. 95-105, July 1995.
[4] A. Alexandrov, M.F. Ionescu, K.E. Schauser, and C. Scheiman, “LogGP: Incorporating Long Messages into the LogP Model for Parallel Computation,” J. Parallel and Distributed Computing, vol. 44, no. 1, pp. 71-79, July 1997.
[5] B. Awerbuch, Y. Azar, A. Fiat, and T. Leighton, “Making Commitments in the Face of Uncertainty: How to Pick a Winner Almost Every Time (Extended Abstract),” Proc. 28th Ann. ACM Symp. Theory of Computing, pp. 519-530, May 1996.
[6] M. Banikazemi, V. Moorthy, and D.K. Panda, “Efficient Collective Communication on Heterogeneous Networks of Workstations,” Proc. 27th Int'l Conf. Parallel Processing (ICPP '98), Aug. 1998.
[7] A. Bar-Noy and S. Kipnis, “Designing Broadcasting Algorithms in the Postal Model for Message-Passing Systems,” Proc. Fourth Ann. ACM Symp. Parallel Algorithms and Architectures (SPAA '92), pp. 13-22, June 1992.
[8] G. Bell and J. Gray, “What's Next in High-Performance Computing?” Comm. ACM, vol. 45, no. 2, pp. 91-95, Feb. 2002.
[9] P. Bhat, C.S. Raghavendra, and V. Prasanna, “Efficient Collective Communication in Distributed Heterogeneous Systems,” Proc. 19th Int'l Conf. Distributed Computing Systems (ICDCS '99), May 1999.
[10] P.B. Bhat, V.K. Prasanna, and C.S. Raghavendra, “Adaptive Communication Algorithms for Distributed Heterogeneous Systems,” J. Parallel and Distributed Computing, vol. 59, no. 2, pp. 252-279, Nov. 1999.
[11] S.N. Bhatt, F.R.K. Chung, F.T. Leighton, and A.L. Rosenberg, “On Optimal Strategies for Cycle-Stealing in Networks of Workstations,” IEEE Trans. Computers, vol. 46, no. 5, pp. 545-557, May 1997.
[12] G. Bilardi, K.T. Herley, A. Pietracaprina, and G. Pucci, “On Stalling in LogP,” Lecture Notes in Computer Science, vol. 1800, 2000.
[13] G.E. Blelloch, Vector Models for Data-Parallel Computing. MIT Press, 1990.
[14] R.D. Blumofe and D.S. Park, “Scheduling Large-Scale Parallel Computations on Networks of Workstations,” Proc. Third Int'l Symp. High-Performance Distributed Computing, pp. 96-105, Aug. 1994.
[15] J.L. Bosque and L. Pastor, “Hloggp: A New Parallel Computational Model for Heterogeneous Clusters,” Proc. IEEE/ACM Int'l Conf. Cluster Computing and the Grid, Apr. 2004.
[16] F. Cappello, P. Fraigniaud, B. Mans, and A.L. Rosenberg, “HiHCoHP: Toward a Realistic Communication Model for Hierarchical HyperClusters of Heterogeneous Processors,” Proc. 15th Int'l Parallel and Distributed Processing Symp. (IPDPS '01), pp.42-42, Apr. 2001.
[17] R. Cole and O. Zajicek, “The APRAM: Incorporating Asynchrony into the PRAM Model,” Proc. First Ann. ACM Symp. Parallel Algorithms and Architectures, pp. 169-178, June 1989.
[18] D. Culler, R. Karp, D. Patterson, A. Sahay, K.E. Schauser, E. Santos, R. Subramonian, and T. von Eicken, “Log P: Towards a Realistic Model of Parallel Computation,” Proc. Fourth ACM SIGPLAN Symp. Principles & Practice of Parallel Programming (PPOPP '90), ACM SIGPLAN Notices, pp. 1-12, July 1993.
[19] D.E. Culler, L.T. Liu, R.P. Martin, and C. Yoshikawa, “LogP Performance Assessment of Fast Network Interfaces,” IEEE Micro, Feb. 1996.
[20] S.R. Donaldson, J.M.D. Hill, and D.B. Skillicorn, “Predictable Communication on Unpredictable Networks: Implementing BSP over TCP/IP and UDP/IP,” Concurrency: Practice and Experience, vol. 11, no. 11, pp. 687-700, Sept. 1999.
[21] A.C. Dusseau, D.E. Culler, K.E. Schauser, and R.P. Martin, “Fast Parallel Sorting under LogP: Experience with the CM-5,” IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 8, pp. 791-805, Aug. 1996.
[22] S. Fortune and J. Wyllie, “Parallelism in Random Access Machines,” Proc. 10th ACM Symp. Theory of Computing, pp. 114-118, 1978.
[23] M. Forum, “A Message-Passing Interface Standard,” 1995, http:/www.mpi-forum.org.
[24] P.B. Gibbons, “A More Practical PRAM Model,” Proc. First Ann. ACM Symp. Parallel Algorithms and Architectures, pp. 158-168, June 1989.
[25] B.H.H. Juurlink and H.A.G. Wijshoff, “A Quantitative Comparison of Parallel Computation Models,” ACM Trans. Computer Systems, vol. 16, no. 3, pp. 271-318, Aug. 1998.
[26] T. Kalinowski, I. Kort, and D. Trystram, “List Scheduling of General Task Graphs under LogP,” Parallel Computing, vol. 26, no. 9, pp. 1109-1128, July 2000.
[27] T. Kielmann, H.E. Bal, and K. Verstoep, “Fast Measurement of LogP Parameters for Message Passing Platforms,” Lecture Notes in Computer Science, vol. 1800, 2000.
[28] W. Löwe and W. Zimmermann, “Scheduling Balanced Task-Graphs to LogP-Machines,” Parallel Computing, vol. 26, no. 9, pp.1083-1108, July 2000.
[29] A.L. Rosenberg, “Sharing Partitionable Workloads in Heterogeneous Nows: Greedier Is Not Better,” Proc. Third IEEE Int'l Conf. Cluster Computing (Cluster '01), pp. 12-131, 2001.
[30] J.M. Squyres, K.L. Meyer, M. McNally, and A. Lumsdaine, LAM/MPI User Guide, 1998.
[31] E.J. Stollnitz, T.D. DeRose, and D.H. Salesin, Wavelets for Computer Graphics: Theory and Applications. Morgan Kauffman, 1996.
[32] L.G. Valiant, “A Bridging Model for Parallel Computation,” Comm. ACM, vol. 22, no. 8, pp. 103-111, Aug. 1990.
[33] J. Verriet, “Scheduling Outtrees of Height One in the LogP Model,” Parallel Computing, vol. 26, no. 9, pp. 1065-1082, July 2000.
[34] J. Watts, M. Rieffel, and S. Taylor, “A Load Balancing Technique for Multiphase Computations,” Proc. HisG Performance Computing Conf., pp. 15-20, 1997.

Index Terms:
Parallel computational models, performance evaluation, heterogeneous systems, cluster computing, LogGP model.
Citation:
Jose Luis Bosque, Luis Pastor, "A Parallel Computational Model for Heterogeneous Clusters," IEEE Transactions on Parallel and Distributed Systems, vol. 17, no. 12, pp. 1390-1400, Dec. 2006, doi:10.1109/TPDS.2006.165
Usage of this product signifies your acceptance of the Terms of Use.