DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/L-CA.2013.9
Stijn Eyerman , Ghent University, Ghent
Lieven Eeckhout , Ghent University, Ghent
Weighted speedup is nowadays the most commonly used multiprogram workload performance metric. Weighted speedup is a weighted-IPC metric, i.e., the multiprogram IPC of each program is first weighted with its isolated IPC. Recently, Michaud questions the validity of weighted-IPC metrics by arguing that they are inconsistent and that weighted speedup favors unfairness. Instead, he advocates using the arithmetic or harmonic mean of the raw IPC values of the programs in the multiprogram workload. We show that weighted-IPC metrics are not inconsistent, and that weighted speedup is fair in giving equal importance to each program. We argue that, in contrast to raw-IPC metrics, weighted-IPC metrics have a system-level meaning, and that raw-IPC metrics are affected by the inherent behavior of the programs. We also show that the choice of a metric may adversely affect the conclusions from an experiment. We suggest to use two weighted-IPC metrics---system throughput (STP) and average normalized turnaround time (ANTT)---for evaluating multiprogram workload performance, and to avoid raw-IPC metrics.
C.4.c Measurement techniques, C Computer Systems Organization, C.1 Processor Architectures, C.1.4 Parallel Architectures, C.1.4.e Multi-core/single-chip multiprocessors, C Computer Systems Organization, C.1 Processor Architectures, C.1.3 Other Architecture Styles, C.1.3.h Multithreaded processors, C Computer Systems Organization, C.4 Performance of Systems
Stijn Eyerman, Lieven Eeckhout, "Restating the Case for Weighted-IPC Metrics to Evaluate Multiprogram Workload Performance", IEEE Computer Architecture Letters, vol. , no. , pp. 0, 5555, doi:10.1109/L-CA.2013.9