This Article 
 Bibliographic References 
 Add to: 
The Design and Implementation of a Domain-Specific Language for Network Performance Testing
October 2007 (vol. 18 no. 10)
pp. 1436-1449
Abstract—CONCEPTUAL is a toolset designed specifically to help measure the performance of high-speed interconnection networks such as those used in workstation clusters and parallel computers. It centers around a high-level, domain-specific language which makes it easy for a programmer to express, measure, and report the performance of complex communication patterns. The primary challenge in implementing a compiler for such a language is that the generated code must be extremely efficient so as not to misattribute overhead costs to the messaging library. At the same time, the language itself must not sacrifice expressiveness for compiler efficiency or there would be little point in using a high-level language for performance testing.This paper describes the CONCEPTUAL language and the CONCEPTUAL compiler’s novel code-generation framework. The language provides primitives for a wide variety of idioms needed for performance testing and emphasizes a readable syntax. The core code-generation technique, based on unrolling CONCEPTUAL programs into sequences of communication events, is simple yet enables the efficient implementation of a variety of high-level constructs. The paper further explains how CONCEPTUAL implements time-bounded loops—even those which comprise blocking communication—in the absence of a timeout mechanism as this is a somewhat unique language/implementation feature.

[1] L. Monk, R. Games, J. Ramsdell, A. Kanevsky, C. Brown, and P. Lee, “Real-Time Communications Scheduling: Final Report,” Technical Report MTR 97B0000069, The MITRE Corp., Center for Integrated Intelligence Systems, Bedford, Mass., , May 1997.
[2] F. Darema, D.A. George, V.A. Norton, and G.F. Pfister, “A Single-Program-Multiple-Data Computational Model for EPEX/FORTRAN,” Parallel Computing, vol. 7, no. 1, pp. 11-24, Apr. 1988.
[3] Methods for Testing and Specification (MTS); The Testing and Test Control Notation version 3; Part 1: TTCN-3 Core Language, European Technological Standards Inst., Sophia Antipolis Cedex, http://www.ttcn-3.orgSpecifications.htm, 2005.
[4] H. Zimmermann, “OSI Reference Model—The ISO Model of Architecture for Open Systems Interconnection,” IEEE Trans. Comm., vol. 28, no. 4, pp. 425-432, , Apr. 1980.
[5] E. Kohler, M.F. Kaashoek, and D.R. Montgomery, “A Readable TCP in the Prolac Protocol Language,” Proc. ACM SIGCOMM '99, pp. 3-13, , 1999.
[6] C.A.R. Hoare, “Communicating Sequential Processes,” Comm. ACM, vol. 21, no. 8, pp. 666-677, Aug. 1978.
[7] K. Perumalla, A. Ogielski, and R. Fujimoto, “TeD—A Language for Modeling Telecommunication Networks,” ACM SIGMETRICS Performance Evaluation Rev., vol. 25, no. 4, pp. 4-11, , Mar. 1998.
[8] R.L. Bagrodia and W.-T. Liao, “Maisie: A Language for the Design of Efficient Discrete-Event Simulations,” IEEE Trans. Software Eng., vol. 20, no. 4, pp. 225-238, , Apr. 1994.
[9] S. Keshav, “REAL: A Network Simulator,” Technical Report CSD-88-472, Computer Science Division, Univ. of California, Berkeley, pdf/RightsManagement_eid=136833.pdfhttp:/ / pdfhttp:/ / pads/PAPERS/ted-sigmetrics. pshttp:/ / / Dienst/Repository/2.0/Body/ncstrl.ucb/ CS D-88-472pdf, Dec. 1988.
[10] L. Arber and S. Pakin, “The Impact of Message-Buffer Alignment on Communication Performance,” Parallel Processing Letters, vol. 15, no. 1, pp. 49-65, papersArber2005:alignment.pdf, Mar. 2005.
[11] Intel MPI Benchmarks: Users Guide and Methodology Description, Intel GmbH, asmo-na/eng/cluster/clustertoolkit219848.htm , Nov. 2004.
[12] R. Reussner, P. Sanders, L. Prechelt, and M. Müller, “SKaMPI: A Detailed, Accurate MPI Benchmark,” Recent Advances in Parallel Virtual Machine and Message Passing Interface: Proc. Fifth European PVM/MPI Users' Group Meeting (EuroPVM/MPI '98), pp. 52-59, ps.gz , Sept. 1998.
[13] Q.O. Snell, A.R. Mikler, and J.L. Gustafson, “NetPIPE: A Network Protocol Independent Performance Evaluator,” Proc. IASTED/ISMM Int'l Conf. Intelligent Information Management Systems,, June 1996.
[14] W. Gropp and E. Lusk, “Reproducible Measurements of MPI Performance Characteristics,” Recent Advances in Parallel Virtual Machine and Message Passing Interface: Proc. Sixth European PVM/MPI Users' Group Meeting (EuroPVM/MPI '99), pp. 11-18, 1999/pvmmpi99mpptest.pdf, Sept. 1999.
[15] S. Araki, A. Bilas, C. Dubnicki, J. Edler, K. Konishi, and J. Philbin, “User-Space Communication: A Quantitative Study,” Proc. ACM/IEEE Conf. Supercomputing (SC '98), http://www.cs.princeton. edu/dubnicki/papers araki98userspace.pdf, 1998.
[16] R.A.F. Bhoedjang, T. Rühl, and H.E. Bal, “User-Level Network Interface Protocols,” Computer, vol. 31, no. 11, pp. 53-60, , Nov. 1998.
[17] J. Beecroft, D. Addison, F. Petrini, and M. McLaren, “Quadrics QsNet II: A Network for Supercomputing Applications,” Proc. Hot Chips 15 Conf., / fabrizio/papershot03.pdf, Aug. 2003.
[18] MPI: A Message-Passing Interface Standard, Message Passing Interface Forum,, June 1995.
[19] S. Pakin, “Reproducible Network Benchmarks with coNCePTuaL,” Proc. 10th Int'l Euro-Par Conf., pp. 64-71, 2004, papersPakin2004: reproducible.pdf.
[20] S. Pakin, “Rapid Development of Application-Specific Network Performance Tests,” Proc. Int'l Conf. Computational Science (ICCS '05) Workshop Tools for Program Development and Analysis in Computational Science, papersPakin2005:conc-library.pdf, May 2005.
[21] S. Pakin, “coNCePTuaL User's Guide,” Technical Report LA-UR 03-7356, Los Alamos Nat'l Laboratory, Los Alamos, New Mexico, conceptual.pdf, Oct. 2003.
[22] S.J. Deitz, B.L. Chamberlain, and L. Snyder, “Abstractions for Dynamic Data Distribution,” Proc. 18th Int'l Parallel and Distributed Processing Symp. (IPDPS '04), Ninth Int'l Workshop High-Level Parallel Programming Models and Supportive Environments (HIPS '04), pp. 42-51, , Apr. 2004.
[23] M. Matsumoto and T. Nishimura, “Mersenne Twister: A 623-Dimensionally Equidistributed Unif Orm Pseudorandom Number Generator,” ACM Trans. Modeling and Computer Simulations, vol. 8, no. 1, pp. 3-30, , Jan. 1998.
[24] A. Hoisie, O.M. Lubeck, and H.J. Wasserman, “Scalability Analysis of Multidimensional Wavefront Algorithms on Large-Scale SMP Clusters,” Proc. Seventh Symp. Frontiers of Massively Parallel Computing (Frontiers '99), pp. 4-15, , Feb. 1999.
[25] W.R. Stevens, B. Fenner, and A.M. Rudoff, “The Sockets Networking API,” Unix Network Programming, vol. 1, third ed., Addison-Wesley, Nov. 2003.
[26] E.R. Gansner and S.C. North, “An Open Graph Visualization System and Its Applications to Software Engineering,” Software—Practice and Experience, vol. 30, no. 11, pp. 1203-1233, , Sept. 2000.
[27] H. Voß, PSTricks: Grafik mit PostScript für $\TeX$ und $\LaTeX$ . Lehmanns Fachbuchhandlung, 2005.
[28] E. Strohmaier and H. Shan, “Apex-Map: A Global Data Access Benchmark to Analyze HPC Systems and Parallel Programming Paradigms,” Proc. ACM/IEEE Supercomputing Conf. (SC '05), papers/data/Deitz04.pdf nisimura/random/doc/mt.pshttp:/ /www.c3.lanl. gov/pal/publications/ papers/Hoisie1999:Sweep3D.pdf sw/tools/graphviz/GN99.pdfhttp:/ / pdfpap280.pdf, Nov. 2005.
[29] Information Networks Division, Netperf: A Network Performance Benchmark, Revision 2.1, Hewlett-Packard Company, , Feb. 1996.
[30] J. Armstrong, R. Virding, C. Wikström, and M. Williams, Concurrent Programming in Erlang, second ed., Prentice Hall, (part 1 only), Jan. 1996.
[31] J. Liu, B. Chandrasekaran, W. Yu, J. Wu, D. Buntinas, S. Kini, D.K. Panda, and P. Wyckoff, “Microbenchmark Performance Comparison of High-Speed Cluster Interconnects,” IEEE Micro, pp. 2-12, journal-papers/2004liuj-ieeemicro04.pdf , Jan.-Feb. 2004.
[32] F. Petrini, W. Feng, A. Hoisie, S. Coll, and E. Frachtenberg, “The Quadrics Network: High-Performance Clustering Technology,” IEEE Micro, vol. 22, no. 1, pp. 46-57, paperspetrini02:qsnet-micro.pdf, Jan.-Feb. 2002.

Index Terms:
Interprocessor communications, Measurement techniques, Specialized application languages
Scott Pakin, "The Design and Implementation of a Domain-Specific Language for Network Performance Testing," IEEE Transactions on Parallel and Distributed Systems, vol. 18, no. 10, pp. 1436-1449, Oct. 2007, doi:10.1109/TPDS.2007.1065
Usage of this product signifies your acceptance of the Terms of Use.