The Community for Technology Leaders
RSS Icon
Issue No.09 - September (2010 vol.43)
pp: 62-67
Todd Mytkowicz , University of Colorado, Boulder
Amer Diwan , University of Colorado , Boulder Boulder
Matthias Hauswirth , University of Lugano, Lugano
Peter Sweeney , IBM T.J. Watson Research Center, Hawthorne
Experiments that do not account for all relevant variables can produce misleading results. The authors identify two variables that compiler optimization evaluations often omit, resulting in significant omitted-variable bias, and discuss how to identify new OVB sources and incorporate them in an experimental design.
Omitted-variable bias, Design and test, Computer performance, Measurement errors, Cache memories
Todd Mytkowicz, Amer Diwan, Matthias Hauswirth, Peter Sweeney, "The Effect of Omitted-Variable Bias on the Evaluation of Compiler Optimizations", Computer, vol.43, no. 9, pp. 62-67, September 2010, doi:10.1109/MC.2010.214
1. W.H. Greene, Econometric Analysis, 6th ed., Prentice-Hall, 2007.
2. J.R. Taylor, An Introduction to Error Analysis: The Study of Uncertainties in Physical Measurements, 2nd ed., University Science Books, 1997.
3. S.M. Blackburn, P. Cheng, and K.S. McKinley, "Myths and Realities: The Performance Impact of Garbage Collection," ACM SIGMETRICS Performance Evaluation Rev., June 2004, pp. 25-36.
4. R.P. Feynman and R. Leighton, "Surely You're Joking, Mr. Feynmann!" Adventures of a Curious Character, W.W. Norton & Co., 1997.
5. J.P.A. Ioannidis,, "Contradicted and Initially Stronger Effects in Highly Cited Clinical Research," JAMA,13 July 2005, pp. 218-228.
6. N.L. Binkert et al., "The M5 Simulator: Modeling Networked Systems," IEEE Micro, July/Aug. 2006, pp. 52-60.
7. T. Mytkowicz et al., "Producing Wrong Data without Doing Anything Obviously Wrong!" Proc. 14th Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS 09), ACM Press, 2009, pp. 265-276.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool