Issue No. 04 - July/August (2008 vol. 10)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MCSE.2008.98
Mohamed Sayeed , Purdue University
Hansang Bae , Purdue University
Yili Zheng , Purdue University
Brian Armstrong , Purdue University
Rudolf Eigenmann , Purdue University
Faisal Saied , Purdue University
The authors discuss the important questions that benchmarking must answer and the degree to which such answers can be given by existing kernel versus real application benchmarks. They describe the state of the art and challenges that must be met to base needed performance measurements on real applications. Finally, they quantify their claims by measuring and comparing several real applications and kernel benchmarks. An important finding is that all three measured computer platforms performed both the best and the worst across the selected applications; "best performance" significantly depended on the problem being solved, questioning the value of computer rankings that use simplistic metrics.
high-performance computing, performance analysis, performance evaluation, performance modeling, kernel benchmarks, real application benchmarks
Y. Zheng, H. Bae, M. Sayeed, R. Eigenmann, B. Armstrong and F. Saied, "Measuring High-Performance Computing with Real Applications," in Computing in Science & Engineering, vol. 10, no. , pp. 60-70, 2008.