Issue No.02 - July-Dec. (2014 vol.13)
Stijn Eyerman , , Ghent University, Belgium
Lieven Eeckhout , , Ghent University, Belgium
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/L-CA.2013.9
Weighted speedup is nowadays the most commonly used multiprogram workload performance metric. Weighted speedup is a weighted-IPC metric, i.e., the multiprogram IPC of each program is first weighted with its isolated IPC. Recently, Michaud questions the validity of weighted-IPC metrics by arguing that they are inconsistent and that weighted speedup favors unfairness . Instead, he advocates using the arithmetic or harmonic mean of the raw IPC values of the programs in the multiprogram workload. We show that weighted-IPC metrics are not inconsistent, and that weighted speedup is fair in giving equal importance to each program. We argue that, in contrast to raw-IPC metrics, weighted-IPC metrics have a system-level meaning, and that raw-IPC metrics are affected by the inherent behavior of the programs. We also show that the choice of a metric may adversely affect the conclusions from an experiment. We suggest to use two weighted-IPC metrics — system throughput (STP) and average normalized turnaround time (ANTT) — for evaluating multiprogram workload performance, and to avoid raw-IPC metrics.
Benchmark testing, Throughput, Harmonic analysis, Degradation, Multicore processing, Weight measurement,C.4.c Measurement techniques, C Computer Systems Organization, C.1 Processor Architectures, C.1.4 Parallel Architectures, C.1.4.e Multi-core/single-chip multiprocessors, C Computer Systems Organization, C.1 Processor Architectures, C.1.3 Other Architecture Styles, C.1.3.h Multithreaded processors, C Computer Systems Organization, C.4 Performance of Systems
Stijn Eyerman, Lieven Eeckhout, "Restating the Case for Weighted-IPC Metrics to Evaluate Multiprogram Workload Performance", IEEE Computer Architecture Letters, vol.13, no. 2, pp. 93-96, July-Dec. 2014, doi:10.1109/L-CA.2013.9