This Article 
 Bibliographic References 
 Add to: 
Second-Level Buffer Cache Management
June 2004 (vol. 15 no. 6)
pp. 505-519

Abstract—Buffer caches are commonly used in servers to reduce the number of slow disk accesses or network messages. These buffer caches form a multilevel buffer cache hierarchy. In such a hierarchy, second-level buffer caches have different access patterns from first-level buffer caches because accesses to a second-level are actually misses from a first-level. Therefore, commonly used cache management algorithms such as the Least Recently Used (LRU) replacement algorithm that work well for single-level buffer caches may not work well for second-level. This paper investigates multiple approaches to effectively manage second-level buffer caches. In particular, it reports our research results in 1) second-level buffer cache access pattern characterization, 2) a new local algorithm called Multi-Queue (MQ) that performs better than nine tested alternative algorithms for second-level buffer caches, 3) a set of global algorithms that manage a multilevel buffer cache hierarchy globally and significantly improve second-level buffer cache hit ratios over corresponding local algorithms, and 4) implementation and evaluation of these algorithms in a real storage system connected with commercial database servers (Microsoft SQL Server and Oracle) running industrial-strength online transaction processing benchmarks.

[1] V. Almeida, A. Bestavros, M. Crovella, and A. de Oliveira, Characterizing Reference Locality in the WWW Proc. IEEE Conf. Parallel and Distributed Information Systems, Dec. 1996.
[2] O.I. Aven, E.G.I. Coffmann, and I.A. Kogan, Stochastic Analysis of Computer Storage. Amsterdam: Reidel, 1987.
[3] E. Bachmat and J. Schindler, Analysis of Methods for Scheduling Low Priority Disk Drive Tasks Proc. Int'l Conf. Measurement and Modeling of Computer Systems, 2002.
[4] J.-L. Baer and W.-H. Wang,"On the Inclusion Property for Multi-Level Cache Hierarchies," Proc. 15th Ann. Int'l Symp. Computer Architecture, 1988, pp.73-80.
[5] L.A. Belady, A Study of Replacement Algorithms for a Virtual-Storage Computer IBM Systems J., vol. 5, no. 2, pp. 78-101, 1966.
[6] R. Carr and J. Hennessy, Wsclock A Simple and Efficient Algorithm for Virtual Memory Management Proc. Eighth Symp. Operating Systems Principles, Dec. 1981.
[7] E. Coffman Jr. and P. Denning, Operating Systems Theory. Prentice-Hall, 1973.
[8] M. Dahlin, R. Wang, T.E. Anderson, and D.A. Patterson, Cooperative Caching: Using Remote Client Memory to Improve File System Performance Operating Systems Design and Implementation, pp. 267-280, 1994.
[9] M.D. Dahlin, C.J. Mather, R.Y. Wang, T.E. Anderson, and D.A. Patterson, A Quantitative Analysis Scalability for Network File Systems Proc. SIGMETRICS, 1994.
[10] A. Dan, D.M. Dias, and P.S. Yu, Analytical Modelling of a Hierarchical Buffer for a Data Sharing Environment Proc. ACM SIGMETRICS Int'l Conf. Measurement and Modeling of Computer Systems ACM SIGMETRICS Performance Evaluation Rev., vol. 19, no. 1, May 1991.
[11] A. Dan and D. Towsley, An Approximate Analysis of the LRU and FIFO Buffer Replacement Schemes 1990.
[12] P.J. Denning, The Working Set Model for Program Behavior Comm. ACM, vol. 11, no. 5, pp. 323-333, May 1968.
[13] P.J. Denning and S.C. Schwartz, Properties of the Working-Set Model Comm. ACM, vol. 15, no. 3, pp. 191-198, Mar. 1972.
[14] EMC Corp., Symmetrix 3000 and 5000 Enterprise Storage Systems, Product Description Guide, pdfs/ pdgsymm_3_5_pdg.pdf, 1999.
[15] M.J. Feeley, W.E. Morgan, F.H. Pighin, A.R. Karlin, H.M. Levy, and C.A. Thekkath, Implementing Global Memory Management in a Workstation Cluster Proc. Symp. Operating Systems Principles, pp. 201-212, 1995.
[16] M.J. Franklin, M.J. Carey, and M. Livny, Global Memory Management in Client-Server DBMS Architectures Proc. Int'l Conf. Very Large Data Bases, pp. 596-609, Aug. 1992.
[17] J. Gecsei, Determining Hit Ratios for Multilevel Hierarchies IBM J. Research and Development, vol. 18, no. 4, pp. 316-327, July 1974.
[18] IBM, IBM Enterprise Storage Server, ess.htm, IBM Corp., 1999.
[19] S. Jiang and X. Zhang, LIRS: An Efficient Low Inter-Reference Recency Set Replacement Policy to Improve Buffer Cache Performance Proc. SIGMETRICS, pp. 31-42, 2002.
[20] T. Johnson and D. Shasha, 2Q: A Low Overhead High Performance Buffer Management Replacement Algorithm Proc. Very Large Databases Conf., pp. 439-450, 1995.
[21] N.P. Jouppi, “Improving Direct-Mapped Cache Performance by the Addition of a Small Fully Associative Cache and Prefetch Buffers,” Proc. 17th Int'l Symp. Computer Architecture, pp. 364-373, May 1990.
[22] J. Kim, J. Choi, J. Kim, S. Noh, S. Min, Y. Cho, and C. Kim, A Low-Overhead High-Performance Unified Buffer Management Scheme that Exploits Sequential and Looping References Proc. Symp. Operating Systems Design and Implementation, 2000.
[23] C. Lam and S.E. Madnick, Properties of Storage Hierarchy Systems with Multiple Page Sizes and Redundant Data ACM Trans. Database Systems, vol. 4, no. 3, pp. 345-367, Sept. 1979.
[24] D. Lee, J. Choi, J.-H. Kim, S.L. Min, Y. Cho, C.S. Kim, and S.H. Noh, On the Existence of a Spectrum of Policies That Subsumes the Least Recently Used (LRU) and Least Frequently Used (LFU) Policies Proc. ACM SIGMETRICS Int'l Conf. Measurement and Modeling of Computing Systems, SIGMETRICS Performance Evaluation Rev., vol. 27, no. 1, pp. 134-143, May 1999.
[25] S.T. Leutenegger and D. Dias, A Modeling Study of the TPC-C Benchmark SIGMOD Record, vol. 22, no. 2, pp. 22-31, June 1993.
[26] C. Lumb, J. Schindler, G.R. Ganger, E. Riedel, and D.F. Nagle, Towards Higher Disk Head Utilization: Extracting‘Free’Bandwidth from Busy Disk Drives Proc. Symp. Operating Systems Design and Implementation, 2000.
[27] R.L. Mattson, J. Gecsei, D.R. Slutz, and I.L. Traiger, Evaluation Techniques for Storage Hierarchies IBM Systems J., vol. 9, no. 2, pp. 78-117, 1970.
[28] N. Megiddo and D.S. Modha, Arc: A Self-Tuning, Low Overhead Replacement Cache Proc. Second USENIX Conf. File and Storage Technologies, 2003.
[29] D. Muntz and P. Honeyman, Multi-Level Caching in Distributed File Systems-or-Your Cache Ain't Nuthin' but Trash Proc. Usenix Winter 1992 Technical Conf., pp. 305-314, Jan. 1991.
[30] V.F. Nicola, A. Dan, and D.M. Dias, Analysis of the Generalized Clock Buffer Replacement Scheme for Database Transaction Processing Proc. 1992 ACM SIGMETRICS and PERFORMANCE Int'l Conf. Measurement and Modeling of Computer Systems, vol. 20, no. 1, p. 35, June 1992.
[31] E.J. O'Neil, P.E. O'Neil, and G. Weikum, The LRU-K Page Replacement Algorithm for Database Disk Buffering Proc. ACM SIGMOD Int'l Conf. Management of Data, pp. 297-306, May 1993.
[32] Oracle 8i Concepts, Oracle Corp., 1999.
[33] B.G.V. Phalke, An Inter-Reference Gap Model for Temporal Locality in Program Behavior Proc. Joint Int'l Conf. Measurement and Modeling of Computer Systems, pp. 291-300, May 1995.
[34] J. Robinson and M. Devarakonda, Data Cache Management Using Frequency-Based Replacement Proc. ACM SIGMETRICS Conf. Measurement and Modeling of Computer Systems, 1990.
[35] C. Ruemmler and J. Wilkes, A Trace-Driven Analysis of Disk Working Set Sizes Technical Report HPL-OSR-93-23, Hewlett-Packard Laboratories, Palo Alto, Calif., Apr. 1993.
[36] C. Ruemmler and J. Wilkes, UNIX Disk Access Patterns Proc. Winter 1993 USENIX Conf., 1993.
[37] P. Sarkar and J. Hartman, Efficient Cooperative Caching Using Hints Proc. Second ACM Symp. Operating Systems Design and Implementation, 1996.
[38] R.A. Shillner and E.W. Felten, Simplifying Distributed File Systems Using a Shared Logical Disk Technical Report TR-524-96, CS Dept., Princeton Univ., 1996.
[39] D. Sleator and R. Tarjan, Amortized Efficiency of List Update and Paging Rules Comm. ACM, vol. 28, no. 2, pp. 202-208, Feb. 1985.
[40] A.J. Smith, Cache Memories ACM Computing Surveys, vol. 14, no. 3, pp. 473-530, Sept. 1982.
[41] A.J. Smith, Disk Cache Miss Ratio Analysis and Design Considerations ACM Trans. Computer Systems, vol. 3, pp. 161-203, 1985.
[42] B.J. Smith, A Pipelined, Shared Resource MIMD Computer Proc. Int'l Conf. Parallel Processing, pp. 6-8, 1978.
[43] J.R. Spirn, Distance String Models for Program Behavior Computer, vol. 9, no. 11, pp. 14-20, Nov. 1976.
[44] Transaction Processing Performance Council, TPC Benchmark C, Shanley Public Relations, 777 N. First Street, Suite 600, San Jose, Calif. 95112-6311, May 1991.
[45] Virtual Interface Architecture Specication Version 1.0, VI-Architecture Organization,http:/, 1997.
[46] W.-H. Wang, J.-L. Baer, and H.M. Levy, “Organization and Performance of a Two-Level Virtual-Real Cache Hierarchy,” Proc. 16th Ann. Int'l Symp. Computer Architecture (ISCA '89), pp. 140-148, June 1989.
[47] D.L. Willick, D.L. Eager, and R.B. Bunt, Disk Cache Replacement Policies for Network Fileservers Proc. 13th Int'l Conf. Distributed Computing Systems, May 1993.
[48] T. Wong and J. Wilkes, My Cache or Yours? Making Storage More Exclusive Proc. USENIX Ann. Technical Conf., 2002.
[49] Y. Zhou, Memory Management for Networked Servers PhD thesis, technical report, Computer Science Dept., Princeton Univ., Nov. 2000.
[50] Y. Zhou, A. Bilas, S. Jagannathan, C. Dubnicki, J.F. Philbin, and K. Li, Experiences with VI Communication for Database Storage Proc. Int'l Symp. Computer Architecture., May 2002.
[51] Y. Zhou, J.F. Philbin, and K. Li, The Multi-Queue Replacement Algorithm for Second Level Buffer Caches Proc. Usenix Technical Conf., June 2001.

Index Terms:
Cache memories, storage hierarchy, storage management.
Yuanyuan Zhou, Zhifeng Chen, Kai Li, "Second-Level Buffer Cache Management," IEEE Transactions on Parallel and Distributed Systems, vol. 15, no. 6, pp. 505-519, June 2004, doi:10.1109/TPDS.2004.13
Usage of this product signifies your acceptance of the Terms of Use.