The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - July-Dec. (2013 vol.12)
pp: 63-66
Pierre Michaud , INRIA, Rennes
ABSTRACT
Several different metrics have been proposed for quantifying the throughput of multicore processors. There is no clear consensus about which metric should be used. Some studies even use several throughput metrics. We show that there exists a relation between single-thread average performance metrics and throughput metrics, and that throughput metrics inherit the meaning or lack of meaning of the corresponding single-thread metric. We show that two popular throughput metrics, the weighted speedup and the harmonic mean of speedups, are inconsistent: they do not give equal importance to all benchmarks. Moreover we demonstrate that the weighted speedup favors unfairness. We show that the harmonic mean of IPCs, a seldom used throughput metric, is actually consistent and has a physical meaning. We explain under which conditions the arithmetic mean or the harmonic mean of IPCs can be used as a strong indicator of throughput increase.
INDEX TERMS
Computer architecture, Program processors, Parallel architectures, Multicore processing, Performance evaluation, Modeling,simulation of multiple-processor systems, Computer Systems Organization, Processor Architectures, Computer Systems Organization, Processor Architectures, Parallel Architectures, Multi-core/single-chip multiprocessors, Computer Systems Organization, Performance of Systems, Measurement, evaluation, modeling
CITATION
Pierre Michaud, "Demystifying multicore throughput metrics", IEEE Computer Architecture Letters, vol.12, no. 2, pp. 63-66, July-Dec. 2013, doi:10.1109/L-CA.2012.25
REFERENCES
1. D. Citron,A. Hurani,, and, A. Gnadrey,“The harmonic or geometric mean: Does it really matter ?” ACM SIGARCH Comput. Archit. News, vol. 34, no. 4, Sep. 2006
2. P. J. Fleming and J. J. Wallace,How not to lie with statistics: The correct way to summarize benchmark results Commun. ACM, vol. 29, no. 3, pp. 218-221 Mar. 1986
3. M. Gomaa,M. D. Powell,, and T. N. Vijaykumar,“Heat-and-run: Leveraging SMT and CMP to manage power density through the operating system,” in ASPLOS, 2004.
4. M. F. Iqbal, L. K. John,“Confusion by all means,” in Workshop on Unique Chips and Systems (UCAS), 2010.
5. L. K. John,More on finding a single number to indicate overall performance of a benchmark suite, ACM SIGARCH Comput. Archit. News, vol. 32, no. 1, Mar. 2004.
6. K. Luo,J. Gummaraju,, and M. Franklin,“Balancing throughput and fairness in SMT processors,” in ISPASS, 2001.
7. J. R. Mashey,War of the benchmark means: Time for a truce,” ACM SIGARCH Comput. Archit. News, vol. 32, no. 4, Sep. 2004.
8. Y. Sazeides and T. Juan,“How to compare the performance of two SMT microarchitectures,” in ISPASS, 2001.
9. J. E. Smith,“Characterizing computer performance with a single number,” Commun. ACM, vol. 31, no. 10 pp. 1202-1206, Oct. 1988.
10. A. Snavely and D. M. Tullsen,“Symbiotic jobscheduling for a simultaneous multithreading architecture,” in ASPLOS, 2000.
11. D. M. Tullsen,S. J. Eggers,, and H. M. Levy,“Simultaneous multithreading: Maximizing on-chip parallelism,” in ISCA, 1995.
12. H. Vandierendonck and A. Seznec,“Fairness metrics for multithreaded processors,” IEEE CAL vol. 10, no. 1, Jan. 2011.
8 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool