This Article 
 Bibliographic References 
 Add to: 
Analytical Modeling of Set-Associative Cache Behavior
October 1999 (vol. 48 no. 10)
pp. 1009-1024

Abstract—Cache behavior is complex and inherently unstable, yet it is a critical factor affecting program performance. A method of evaluating cache performance is required, both to give quantitative predictions of miss-ratio and information to guide optimization of cache use. Traditional cache simulation gives accurate predictions of miss-ratio, but little to direct optimization. Also, the simulation time is usually far greater than the program execution time. Several analytical models have been developed, but concentrate mainly on direct-mapped caches, often for specific types of algorithm, or to give qualitative predictions. In this work, novel analytical models of cache phenomena are presented, applicable to numerical codes consisting mostly of array operations in looping constructs. Set-associative caches are considered, through an extensive hierarchy of cache reuse and interference effects, including numerous forms of temporal and spatial locality. Models of each effect are given which, when combined, predict the overall miss-ratio. An advantage is that the models also indicate sources of cache interference. The accuracy of the models is validated through example program fragments. The predicted miss-ratios are compared with simulations and shown typically to be within 15 percent. The evaluation time of the models is shown to be independent of the problem size, generally several orders of magnitude faster than simulation.

[1] A. Agarwal, M. Horowitz, and J. Hennessy, "An Analytical Cache Model," ACM Trans. Computer Systems, vol. 7, pp. 184-215, May 1989.
[2] F. Bodin and A. Seznec, “Skewed Associativity Improves Program Performance and Enhances Predictability,” IEEE Trans. Computers, vol. 46, no. 5, pp. 530-544, May 1997.
[3] S. Coleman and K. McKinley, “Tile Size Selection Using Cache Organization and Data Layout,” Proc. SIGPLAN Conf. Programming Language Design and Implementation, June 1995.
[4] T. Fahringer, “Automatic Cache Performance Prediction in a Parallelizing Compiler,” Proc. AICA '93—International Section, Sept. 1993.
[5] C. Fricker, O. Temam, and W. Jalby, “Influence of Cross-Interferences on Blocked Loops: A Case Study with Matrix-Vector Multiply,” ACM Trans. Programming Languages and Systems, vol. 17, no. 4, pp. 561-575, July 1995.
[6] S. Ghosh, M. Martonosi, and S. Malik, "Cache Miss Equations: An Analytical Representation of Cache Misses," Proc. Int'l Conf. Supercomputing (ICS 97), IEEE Computer Soc. Press, Los Alamitos, Calif., 1997, pp. 317-324.
[7] J.S. Harper, D.J. Kerbyson, and G.R. Nudd, “Predicting the Cache Miss Ratio of Loop-Nested Array References,” Research Report CS-RR-336, Dept. of Computer Science, Univ. of Warwick, Coventry, U.K., Dec. 1997.
[8] J.S. Harper, D.J. Kerbyson, and G.R. Nudd, “Efficient Analytical Modelling of Multi-Level Set-Associative Caches,” Proc. Int'l Conf. HPCN Europe '99, pp. 473-482, 1999.
[9] M.D. Hill, Aspects of Cache Memory and Instruction Buffer Performance, PhD thesis, UCB/CSD 87/381, Univ. of California at Berkeley, Nov. 1987.
[10] M. Lam, E. Rothberg, and M. Wolf, “The Cache Performance and Optimizations of Blocked Algorithms,” Proc. Fourth Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS '91), 1991.
[11] K.S. McKinley and O. Temam, “A Quantitative Analysis of Loop Nest Locality,” Proc. Seventh Int'l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 94-104, Oct. 1996.
[12] G.R. Nudd, D.J. Kerbyson, E. Papaefstathiou, J.S. Harper, S.C. Perry, and D.V. Wilcox, “PACE: A Toolset for the Performance Prediction of Parallel and Distributed Systems,” Int'l J. High Performance Computing, 1999.
[13] O. Temam, C. Fricker, and W. Jalby, "Cache Interference Phenomena," Proc. ACM SIGMetrics Conf. Measurement and Modeling of Computer Systems, ACM Press, New York, 1994.
[14] R.A. Uhlig and T.N. Mudge, "Trace-Driven Memory Simulation: A Survey," ACM Computing Surveys, Vol. 29, No. 2, June 1997, pp. 128-170.
[15] M. Wolf and M. Lam, “A Data Locality Optimizing Algorithm,” Proc. SIGPLAN Conf. Programming Language Design and Implementation, pp. 30-44, June 1991.

Index Terms:
Cache modeling, set-associative, cache interference, performance evaluation, data locality, analytical modeling.
John S. Harper, Darren J. Kerbyson, Graham R. Nudd, "Analytical Modeling of Set-Associative Cache Behavior," IEEE Transactions on Computers, vol. 48, no. 10, pp. 1009-1024, Oct. 1999, doi:10.1109/12.805152
Usage of this product signifies your acceptance of the Terms of Use.