This Article 
 Bibliographic References 
 Add to: 
Optimal Partitioning of Cache Memory
September 1992 (vol. 41 no. 9)
pp. 1054-1068

A model for studying the optimal allocation of cache memory among two or more competing processes is developed and used to show that, for the examples studied, the least recently used (LRU) replacement strategy produces cache allocations that are very close to optimal. It is also shown that when program behavior changes, LRU replacement moves quickly toward the steady-state allocation if it is far from optimal, but converges slowly as the allocation approaches the steady-state allocation. An efficient combinatorial algorithm for determining the optimal steady-state allocation, which, in theory, could be used to reduce the length of the transient, is described. The algorithm generalizes to multilevel cache memories. For multiprogrammed systems, a cache-replacement policy better than LRU replacement is given. The policy increases the memory available to the running process until the allocation reaches a threshold time beyond which the replacement policy does not increase the cache memory allocated to the running process.

[1] J.-L. Baer and W.-H. Wang, "On the Inclusion Properties for Multi-Level Cache Hierarchies,"Proc. 15th Ann. Int'l Symp. Computer Architecture, IEEE-CS Press, Los Alamitos, Calif., Order No. 861, 1988, pp. 73-80.
[2] L. Belady, "A study of replacement algorithms for a virtual-store computer,"IBM Syst. J., vol. 5, no. 2, pp. 78-101, 1966.
[3] C. K. Chow, "On optimization of storage hierarchy,"IBM J. Res. Develop., vol. 18, pp. 194-203, May 1974.
[4] B. Fox, "Discrete optimization via marginal analysis,"Management Sci., vol. 13, pp. 909-918, 1966.
[5] G. N. Frederickson and D. B. Johnson, "The complexity of selection and ranking in X + Y and matrices with sorted columns,"J. Comput. Syst. Sci., vol. 24, pp. 197-208, 1982.
[6] Z. Galil and N. Megiddo, "A fast selection algorithm and the problem of optimum distribution of effort,"J. ACM, vol. 26, pp. 58-64, 1979.
[7] M. Z. Ghanem, "Dynamic partitioning of the main memory using the working set concept,"IBM J. Res. Develop., pp. 445-450, Sept. 1975.
[8] J.L. Hennessy and David A. Patterson,Computer Architecture: A Quantitative Approach, Morgan Kaufmann, San Mateo, Calif., 1990.
[9] M. D. Hill, "A case for direct-mapped caches,"IEEE Comput. Mag., vol. 21, no. 12, pp. 25-40, Dec. 1988.
[10] T. Ibaraki and N. Katoh,Resource Allocation Problems. Cambridge, MA: MIT Press, 1988.
[11] D. B. Kirk, "Process dependent static cache partitioning for real time systems," inProc. Real Time Syst. Symp., 1988, pp. 181-190.
[12] S. Przybylski, M. Horowitz, and J. Hennessy, "Characteristics of performance optimal multilevel cache hierarchies," inProc. 16th Annu. Int. Symp. Comput. Architecture, 1989, pp. 114-121.
[13] A. Smith, "Cache Memories,"Computing Surveys, Vol. 14, No. 3, Sept. 1982, pp. 473- 530.
[14] A. J. Smith, "Line (block) size choice for CPU cache memories,"IEEE Trans. Computers, vol. 36, no. 9, pp. 1063-1074, 1987.
[15] J. E. Smith and J. R. Goodman, "Instruction cache replacement policies and organizations,"IEEE Trans. Comput., vol. C-34, no. 3, pp. 234-241, Mar. 1985.
[16] K. So and R. N. Rechtschaffen, "Cache operations by MRU change,"IEEE Trans. Comput., vol. 37, no. 6, pp. 700-709, June 1988.
[17] H. S. Stone, D. F. Thiebaut, and J. L. Wolf, "Improving disk cache hitratios through cache partitioning,"IEEE Trans. Comput., vol. 41, no. 6, pp. 665-676, 1992. The paper originally appeared as IBM Research Report RC 15072, Oct. 26, 1989.
[18] W. D. Strecker, "Transient behavior of cache memories,"ACM Trans. Comput. Syst., vol. 1, no. 4, pp. 281-293, Nov. 1983.
[19] A. N. Tantawi, D. Towsley, and J. L. Wolf, "An algorithm for a class-constrained resource allocation problem, " inProc. 1988 ACM Sigmetrics Conf., May 1988, pp. 253-260.
[20] D. F. Thiebaut, "On the fractal dimension of computer programs and its application to the prediction of the cache miss ratio,"IEEE Trans. Comput., vol. 38, no. 7, pp. 1012-1026, July 1989.
[21] D. F. Thiebaut and H. S. Stone, "Footprints in the cache,"ACM Trans. Comput. Syst., vol. 5, no. 4, pp. 305-329, Nov. 1987.
[22] D. F. Thiebaut, H. S. Stone, and J. L. Wolf, "A theory of cache behavior,"IBM Res. Rep. RC 13309, Nov. 10, 1987.
[23] M. K. Vernon, R. Jog, and G. S. Sohi, "Performance analysis of hierarchical cache consistent multiprocessors," inPerform. Distributed and Parallel Systems, Proc. IFIP TC 7/WG 7.3, 1989, pp. 111-126.
[24] J. L. Wolf, D. Dias, and P. S. Yu, "An effective algorithm for parallelizing sort merge joins in the presence of data skew," inProc. Second Int. Conf. Databases in Parallel and Distributed Syst., Dublin, Ireland, July 1990, pp. 103-115.
[25] J. L. Wolf, B. R. Iyer, K. R. Pattipati, and J. Turek, "Optimal buffer partitioning for the nested block join algorithm," inProc. Seventh Int. Conf. Data Eng., Kobe, Japan, Apr. 1991, pp. 510-519.

Index Terms:
optimal partitioning; cache memory; optimal allocation; competing processes; least recently used; replacement strategy; cache allocations; program behavior; steady-state allocation; combinatorial algorithm; multilevel cache memories; multiprogrammed systems; cache-replacement policy; running process; threshold time; buffer storage; storage allocation.
H.S. Stone, J. Turek, J.L. Wolf, "Optimal Partitioning of Cache Memory," IEEE Transactions on Computers, vol. 41, no. 9, pp. 1054-1068, Sept. 1992, doi:10.1109/12.165388
Usage of this product signifies your acceptance of the Terms of Use.