loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
The Seventh Workshop on Hot Topics in Operating Systems
The Case for Application-Specific Benchmarking
Rio Rico, Arizona
March 28-March 30
ISBN: 0-7695-0237-7
Margo Seltzer, Harvard University
David Krinsky, Harvard University
Keith Smith, Harvard University
Xiaolan Zhang, Harvard University
Most performance analysis today uses either microbenchmarks or standard macrobenchmarks (e.g., SPEC, LADDIS, the Andrew benchmark). However, the results of such benchmarks provide little information to indicate how well a particular system will handle a particular application. Such results are, at best, useless and, at worst, misleading. In this paper, we argue for an application-directed approach to benchmarking, using performance metrics that reflect the expected behavior of a particular application across a range of hardware or software platforms. We present three different approaches to application-specific measurement, one using vectors that characterize both the underlying system and an application, one using trace-driven techniques, and a hybrid approach. We argue that such techniques should become the new standard.
Index Terms:
benchmark, performance, application-specific, measurement
Citation:
Margo Seltzer, David Krinsky, Keith Smith, Xiaolan Zhang, "The Case for Application-Specific Benchmarking," hotos, pp.102, The Seventh Workshop on Hot Topics in Operating Systems, 1999
Usage of this product signifies your acceptance of the Terms of Use.