The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2010 vol.21)
pp: 698-709
Jesper Larsson Träff , University of Vienna, Vienna
William D. Gropp , University of Illinois at Urbana-Champaign, Urbana
Rajeev Thakur , Argonne National Laboratory, Argonne
ABSTRACT
Message passing using the Message-Passing Interface (MPI) is at present the most widely adopted framework for programming parallel applications for distributed memory and clustered parallel systems. For reasons of (universal) implementability, the MPI standard does not state any specific performance guarantees, but users expect MPI implementations to deliver good and consistent performance in the sense of efficient utilization of the underlying parallel (communication) system. For performance portability reasons, users also naturally desire communication optimizations performed on one parallel platform with one MPI implementation to be preserved when switching to another MPI implementation on another platform. We address the problem of ensuring performance consistency and portability by formulating performance guidelines and conditions that are desirable for good MPI implementations to fulfill. Instead of prescribing a specific performance model (which may be realistic on some systems, under some MPI protocol and algorithm assumptions, etc.), we formulate these guidelines by relating the performance of various aspects of the semantically strongly interrelated MPI standard to each other. Common-sense expectations, for instance, suggest that no MPI function should perform worse than a combination of other MPI functions that implement the same functionality, no specialized function should perform worse than a more general function that can implement the same functionality, no function with weak semantic guarantees should perform worse than a similar function with stronger semantics, and so on. Such guidelines may enable implementers to provide higher quality MPI implementations, minimize performance surprises, and eliminate the need for users to make special, nonportable optimizations by hand. We introduce and semiformalize the concept of self-consistent performance guidelines for MPI, and provide a (nonexhaustive) set of such guidelines in a form that could be automatically verified by benchmarks and experiment management tools. We present experimental results that show cases where guidelines are not satisfied in common MPI implementations, thereby indicating room for improvement in today's MPI implementations.
INDEX TERMS
Parallel processing, message passing, message-passing interface, MPI, performance portability, performance prediction, performance model, public benchmarking, performance guidelines.
CITATION
Jesper Larsson Träff, William D. Gropp, Rajeev Thakur, "Self-Consistent MPI Performance Guidelines", IEEE Transactions on Parallel & Distributed Systems, vol.21, no. 5, pp. 698-709, May 2010, doi:10.1109/TPDS.2009.120
REFERENCES
[1] W. Augustin and T. Worsch, "Usefulness and Usage of SKaMPI-Bench," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: 10th European PVM/MPI Users' Group Meeting, pp. 63-70, 2003.
[2] M. Barnett, S. Gupta, D.G. Payne, L. Schuler, R. van de Geijn, and J. Watts, "Building a High-Performance Collective Communication Library," Proc. Conf. Supercomputing '94, pp. 107-116, 1994.
[3] E.W. Chan, M.F. Heimlich, A. Purkayastha, and R.A. van de Geijn, "On Optimizing Collective Communication," Proc. IEEE Int'l Conf. Cluster Computing (CLUSTER), 2004.
[4] D.E. Culler, R.M. Karp, D. Patterson, A. Sahay, E.E. Santos, K.E. Schauser, R. Subramonian, and T. von Eicken, "LogP: A Practical Model of Parallel Computation," Comm. ACM, vol. 39, no. 11, pp. 78-85, 1996.
[5] J. Dean and S. Ghemawat, "MapReduce: Simplified Data Processing on Large Clusters," Comm. ACM, vol. 51, no. 1, pp. 107-113, 2008.
[6] T. El-Ghazawi, W. Carlson, T. Sterling, and K. Yelick, UPC: Distributed Shared Memory Programming. Wiley, 2005.
[7] A.H. Gebremedhin, M. Essaïdi, I.G. Lassous, J. Gustedt, and J.A. Telle, "PRO: A Model for the Design and Analysis of Efficient and Scalable Parallel Algorithms," Nordic J. Computing, vol. 13, pp. 215-239, 2006.
[8] S. Girona, J. Labarta, and R.M. Badia, "Validation of Dimemas Communication Model for MPI Collective Operations," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: Seventh European PVM/MPI Users' Group Meeting, pp. 39-46, 2000.
[9] S. Gorlatch, "Toward Formally-Based Design of Message Passing Programs," IEEE Trans. Software Eng., vol. 26, no. 3, pp. 276-288, Mar. 2000.
[10] S. Gorlatch, "Send-Receive Considered Harmful: Myths and Realities of Message Passing," ACM Trans. Programming Languages and Systems, vol. 26, no. 1, pp. 47-56, 2004.
[11] W. Gropp, S. Huss-Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir, MPI—The Complete Reference, vol. 2. MIT Press, 1998.
[12] W.D. Gropp, D. Kimpe, R. Ross, R. Thakur, and J.L. Träff, "Self-Consistent MPI IO Performance Requirements and Expectations," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: 15th European PVM/MPI Users' Group Meeting, pp. 167-176, 2008.
[13] J. Hein, S. Booth, and M. Bull, "Exchanging Multiple Messages via MPI," Technical Report HPCxTR0308, EPCC, The Univ. of Edinburgh, 2003.
[14] T. Hoefler, T. Mehlan, A. Lumsdaine, and W. Rehm, "Netgauge: A Network Performance Measurement Framework," Proc. High Performance Computing and Comm. (HPPC), pp. 659-671, 2007.
[15] D. Jiang, H. Shan, and J.P. Singh, "Application Restructuring and Performance Portability on Shared Virtual Memory and Hardware-Coherent Multiprocessors," Proc. Sixth ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP), pp. 217-229, 1997.
[16] D.J. Kerbyson and A. Hoisie, "S05—A Practical Approach to Performance Analysis and Modeling of Large-Scale Systems," Proc. ACM/IEEE SC Conf. High Performance Networking and Computing, p. 206, 2006.
[17] T. Kielmann, H.E. Bal, and K. Verstoep, "Fast Measurement of LogP Parameters for Message Passing Platforms," Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '00), pp. 1176-1183, 2000.
[18] M. Kühnemann, T. Rauber, and G. Rünger, "Optimizing MPI Collective Communication by Orthogonal Structures," Cluster Computing, vol. 9, no. 3, pp. 257-279, 2006.
[19] C. Lin and L. Snyder, Principles of Parallel Programming. Pearson/Addison-Wesley, 2008.
[20] M.M. Mathis, D.J. Kerbyson, and A. Hoisie, "A Performance Model of Non-Deterministic Particle Transport on Large-Scale Systems," Future Generation Computer Systems, vol. 22, no. 3, pp. 324-335, 2006.
[21] L.C. McInnes, J. Ray, R. Armstrong, T.L. Dahlgren, A. Malony, B. Norris, S. Shende, J.P. Kenny, and J. Steensland, "Computational Quality of Service for Scientific CCA Applications: Composition, Substitution, and Reconfiguration," Technical Report ANL/MCS-P1326-0206, Argonne Nat'l Laboratory, Feb. 2006.
[22] MPI Forum. MPI: A Message-Passing Interface Standard. Version 2.1, www.mpi-forum.org, Sept. 2008.
[23] B. Norris, L. McInnes, and I. Veljkovic, "Computational Quality of Service in Parallel CFD," Proc. 17th Int'l Conf. Parallel Computational Fluid Dynamics, pp. 24-27, May 2006.
[24] M. Plummer and K. Refson, "An LPAR-Customized MPI_Alltoallv for the Materials Science Code CASTEP," Technical Report HPCxTR0401, EPCC, The Univ. of Edinburgh, 2004.
[25] R. Reussner, "Using SKaMPI for Developing High-Performance MOI Programs with Performance Portability," Future Generation Computing Systems, vol. 19, no. 5, pp. 749-759, 2003.
[26] R. Reussner, P. Sanders, L. Prechelt, and M. Müller, "SKaMPI: A Detailed, Accurate MPI Benchmark," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: Fifth European PVM/MPI Users' Group Meeting, pp. 52-59, 1998.
[27] R. Reussner, P. Sanders, and J.L. Träff, "SKaMPI: A Comprehensive Benchmark for Public Benchmarking of MPI," Scientific Programming, vol. 10, no. 1, pp. 55-65, 2002.
[28] H. Ritzdorf and J.L. Träff, "Collective Operations in NEC's High-Performance MPI Libraries," Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '06), p. 100, 2006.
[29] G. Rodríguez, R.M. Badia, and J. Labarta, "Generation of Simple Analytical Models for Message Passing Applications," Proc. Euro-Par '04 Parallel Processing Conf., pp. 183-188, 2004.
[30] V.A. Saraswat, V. Sarkar, and C. von Praun, "X10: Concurrent Programming for Modern Architectures," Proc. ACM SIGPLAN Symp. Principles and Practice of Parallel Programming (PPoPP), p. 271, 2007.
[31] M. Snir, S. Otto, S. Huss-Lederman, D. Walker, and J. Dongarra, MPI—The Complete Reference, second ed., vol. 1. MIT Press, 1998.
[32] R. Thakur, W.D. Gropp, and R. Rabenseifner, "Improving the Performance of Collective Operations in MPICH," Int'l J. High Performance Computing Applications, vol. 19, pp. 49-66, 2004.
[33] J.L. Träff, "Hierarchical Gather/Scatter Algorithms with Graceful Degradation," Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS '04), p. 80, 2004.
[34] J.L. Träff, "An Improved Algorithm for (Non-Commutative) Reduce-Scatter with an Application," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: 12th European PVM/MPI Users' Group Meeting, pp. 129-137, 2005.
[35] J.L. Träff, W. Gropp, and R. Thakur, "Self-Consistent MPI Performance Requirements," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: 14th European PVM/MPI Users' Group Meeting, pp. 36-45, 2007.
[36] L.G. Valiant, "A Bridging Model for Parallel Computation," Comm. ACM, vol. 33, no. 8, pp. 103-111, 1990.
[37] P. Worley and J. Drake, "Performance Portability in the Physical Parameterizations of the Community Atmospheric Model," Int'l J. High Performance Computing Applications, vol. 19, no. 3, pp. 187-202, 2005.
[38] P.H. Worley, "Comparison of Cray XT3 and XT4 Scalability," Proc. 49th Cray User's Group Meeting (CUG), 2007.
[39] J. Worringen, "Experiment Management and Analysis with Perfbase," Proc. IEEE Int'l Conf. Cluster Computing, 2005.
[40] J. Worringen, "Automated Performance Comparison," Proc. Recent Advances in Parallel Virtual Machine and Message Passing Interface: 13th European PVM/MPI Users' Group Meeting, pp. 402-403, 2006.
[41] K. Yelick, P. Hilfinger, S. Graham, D. Bonachea, J. Su, A. Kamil, P.C.K. Datta, and T. Wen, "Parallel Languages and Compilers: Perspective from the Titanium Experience," Int'l J. High Performance Computing Applications, vol. 21, pp. 266-290, 2007.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool