This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
SPEC as a Performance Evaluation Measure
August 1995 (vol. 28 no. 8)
pp. 33-42
System Performance Evaluation Cooperative (SPEC) measures are commonly considered the most effective indicators for evaluating engineering workstation performance. Indeed, these measures have gradually become a de facto industry standard for CPU performance evaluation. But do they truly provide added information value? The authors examine SPEC measures with the aim of learning their limitations and the areas in which caution should be exercised. They present results and evaluations of a comprehensive statistical analysis of SPEC rankings published in the SPEC Newsletter over a number of years. The major findings show that several programs in the benchmark suite contribute little additional information to the measures. In fact, the potency profile of the system can be demonstrated by just a few representative programs. Moreover, the method of aggregating the results to create a single measure (or two measures) leads to distortion in presenting the overall system performance. In evaluating the system, SPEC focuses on the CPU clock rate and thus may not reflect the performance of the entire system. This suggests that the added value of the SPEC information is not very high. The authors thus advise that SPEC should be employed cautiously with full awareness of its shortcomings.
Citation:
Ran Giladi, Niv Ahituv, "SPEC as a Performance Evaluation Measure," Computer, vol. 28, no. 8, pp. 33-42, Aug. 1995, doi:10.1109/2.402073
Usage of this product signifies your acceptance of the Terms of Use.