Issue No. 11 - November (2007 vol. 56)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TC.2007.70748
<p><b>Abstract</b>—Commercial workloads form an important class of applications and have performance characteristics that are distinct from scientific and technical benchmarks such as SPEC CPU. However, due to the prohibitive simulation time of commercial workloads, it is extremely difficult to use them in computer architecture research. In this paper, we study the efficacy of using statistical sampling based simulation methodology for two classes of commercial workloads - a Java server benchmark, SPECjbb2000, and an Online Transaction Processing (OLTP) benchmark, DBT-2. Our results show that although SPECjbb2000 shows distinct garbage collection phases, there are no large-scale phases in the OLTP benchmark. We take advantage of this stationary behavior in steady phase, and propose a statistical sampling based simulation technique, DynaSim, with two dynamic stopping rules. In this approach, the simulation terminates once the target accuracy has been met. We apply DynaSim to simulate commercial workloads and show that with the simulation of only a few million total instructions, the error can be within 3% at a confidence level of 99%. DynaSim compares favorably with random sampling and representative sampling in terms of the total number of instructions simulated (time cost) and with representative sampling in terms of the number of checkpoints (storage cost). DynaSim increases the usability of a sampling based simulation approach for commercial workloads, and will encourage the use of commercial workloads in computer architecture research.</p>
simulation, representative sampling, random sampling, commercial workloads
Y. Luo, A. Joshi and L. K. John, "Applying Statistical Sampling for Fast and Efficient Simulation of Commercial Workloads," in IEEE Transactions on Computers, vol. 56, no. , pp. 1520-1533, 2007.