This Article 
 Bibliographic References 
 Add to: 
An Analytical Model for Designing Memory Hierarchies
October 1996 (vol. 45 no. 10)
pp. 1180-1194

Abstract—Memory hierarchies have long been studied by many means: system building, trace-driven simulation, and mathematical analysis. Yet little help is available for the system designer wishing to quickly size the different levels in a memory hierarchy to a first-order approximation. In this paper, we present a simple analysis for providing this practical help and some unexpected results and intuition that come out of the analysis. By applying a specific, parameterized model of workload locality, we are able to derive a closed-form solution for the optimal size of each hierarchy level. We verify the accuracy of this solution against exhaustive simulation with two case studies: a three-level I/O storage hierarchy and a three-level processor-cache hierarchy. In all but one case, the configuration recommended by the model performs within 5% of optimal. One result of our analysis is that the first place to spend money is the cheapest (rather than the fastest) cache level, particularly with small system budgets. Another is that money spent on an n-level hierarchy is spent in a fixed proportion until another level is added.

[1] S. Blumson, P. Honeyman, T.E. Ragland, and M.T. Stolarchuk, "AFS Server Logging," Technical Report CITI-93-10, Univ. of Michigan, Nov. 1993.
[2] C.H. Edwards Jr. and D.E. Penney, Calculus and Analytic Geometry.Englewood Cliffs, N.J.: Prentice Hall, 1982.
[3] C.K. Chow, "On Optimization of Storage Hierarchies," IBM J. Research and Development, pp. 194-203, May 1974.
[4] C.K. Chow, "Determination of Cache's Capacity and Its Matching Storage Hierarchy," IEEE Trans. Computers, vol. 25, no. 2, pp. 157-164, Feb. 1976.
[5] E.G. Coffman and P.J. Denning,Operating Systems Theory, Prentice-Hall Inc., Englewood Cliffs, N.J., 1973.
[6] A.L. Drapeau and R.H. Katz, “Striped Tape Arrays,” Proc. 12th IEEE Symp. Mass Storage Systems, pp. 257-265, 1993.
[7] H. Garcia-Molina, A. Park, and L.R. Rogers, "Performance Through Memory," Proc. 1987 ACM Sigmetrics Conf. Measurement and Modeling of Computer Systems, pp. 122-131, 1987.
[8] B.L. Jacob, "Optimization of Mass Storage Hierarchies," Technical Report CSE-TR-228-95, Univ. of Michigan, May 1994.
[9] N. Jouppi and S. Wilton, ``Tradeoffs in Two-Level On-Chip Caching,'' Proc. 21st ISCA, pp. 34-45, Apr. 1994.
[10] J.E. MacDonald and K.L. Sigworth, "Storage Hierarchy Optimization Procedure," IBM J. Research and Development, pp. 133-140, Mar. 1975.
[11] R. Pike, D. Presotto, K. Thompson, and H. Trickey, "Plan 9 from Bell Labs," Proc. Summer 1990 UKUUG Conf., pp. 1-9,London, July 1990.
[12] S.A. Przybylski, Cache and Memory Hierarchy Design—A Performance-Directed Approach, pp. 181-186. Morgan Kaufmann, 1990.
[13] S. Quinlan, "A Cached WORM File System," Software-Practice and Experience, vol. 21, no. 12, pp. 1,289-1,299, Dec. 1991.
[14] S. Ranade, Mass Storage Technologies.Westport, Conn.: Meckler Publishing, 1991.
[15] S.L. Rege, "Cost, Performance and Size Trade-Offs for Different Levels in a Memory Hierarchy," Computer, vol. 19, pp. 43-51, Apr. 1976.
[16] A.J. Smith, "Two Methods for the Efficient Analysis of Memory Address Trace Data," IEEE Trans. Software Eng., vol. 3, no. 1, pp. 94-101, Jan. 1977.
[17] A.J. Smith, "Cache Memories," ACM Computing Surveys, Vol. 14, 1982, pp. 473-540.
[18] A.J. Smith, "Disk Cache—Miss Ratio Analysis and Design Considerations," ACM Trans. Computer Systems, Aug. 1985, pp. 161-203.
[19] H.S. Stone, High Performance Computer Architecture. Addison Wesley, 1993.
[20] T.A. Welch, "Memory Hierarchy Configuration Analysis," IEEE Trans. Computers, vol. 27, no. 5, pp. 408-413, May 1978.

Index Terms:
Cache, memory, and storage hierarchies; trace-driven simulations; optimization of cache configurations.
Bruce L. Jacob, Peter M. Chen, Seth R. Silverman, Trevor N. Mudge, "An Analytical Model for Designing Memory Hierarchies," IEEE Transactions on Computers, vol. 45, no. 10, pp. 1180-1194, Oct. 1996, doi:10.1109/12.543711
Usage of this product signifies your acceptance of the Terms of Use.