The Community for Technology Leaders
RSS Icon
Issue No.11 - November (2010 vol.59)
pp: 1508-1519
Song Jiang , Wayne State University, Detroit
Xuechen Zhang , Wayne State University, Detroit
Shuang Liang , Ohio State University, Columbus
Kei Davis , Los Alamos National Laboratory, Los Alamos
In a distributed environment, the utilization of file buffer caches in different clients may greatly vary. Cooperative caching has been proposed to increase cache utilization by coordinating the shared usage of distributed caches. It allows clients that would more greatly benefit from larger caches to forward data objects to peer clients with relatively underutilized caches. To support such coordination, global cache utilization must be dynamically evaluated. This, in turn, requires an effective analysis of application data access patterns. Existing coordination protocols are demonstrably suboptimal in this respect, exhibiting inefficient memory utilization and undue interference among clients. We propose a locality-aware cooperative caching protocol, called LAC, that is based on analysis and manipulation of data block reuse distance to effectively predict cache utilization and the probability of data reuse at each client. Using a dynamically adaptive synchronization technique, we keep local information up to date and consistently comparable across clients. The system is highly scalable in the sense that global coordination is achieved without centralized control. We have conducted thorough trace-driven simulation experiments to assess the performance differences between LAC and various existing protocols representative of the general class. Using a realistic and representative cost model, we show that the LAC protocol significantly and consistently outperforms existing cooperative caching protocols, demonstrating high and balanced utilization of caches across all clients. In our experiments, LAC reduces block access time by up to 36 percent, with an average of 31 percent, over the system without peer cache coordination, and reduces block access time by up to 22 percent, with an average of 13 percent, over the best performer of the existing protocols.
Replacement algorithm, locality, cooperative caching, networked file system.
Song Jiang, Xuechen Zhang, Shuang Liang, Kei Davis, "Improving Networked File System Performance Using a Locality-Aware Cooperative Cache Protocol", IEEE Transactions on Computers, vol.59, no. 11, pp. 1508-1519, November 2010, doi:10.1109/TC.2010.37
[1] S. Annapureddy, M.J. Freedman, and D. Mazières, "Shark: Scaling File Servers via Cooperative Caching," Proc. Second USENIX/ACM Symp. Networked Systems Design and Implementation, 2005.
[2] F.M. Cuenca-Acuna and T.D. Nguyen, "Cooperative Caching Middleware for Cluster-Based Servers," Proc. 10th IEEE Int'l Symp. High Performance Distributed Computing, 2001.
[3] D. Nagle, D. Serenyi, and A. Matthews, "The Panasas ActiveScale Storage Cluster—Delivering Scalable High Bandwidth Storage," Proc. 2004 ACM/IEEE Conf. Supercomputing (SC '04), 2004.
[4] "Lustre File System," Lustre, /, 2010.
[5] P. Corbett, D. Feltelson, J. Prost, G. Almasi, S. Baylor, A. Bolmarcich, Y. Hsu, J. Satran, M. Snir, R. Colao, B. Herr, J. Kavaky, T. Morgan, and A. Ziotek, "Parallel File Systems for the IBM SP Computers," IBM Systems J., vol. 34, no. 2, pp. 222-248, 1995.
[6] F. Schmuck and R. Haskin, "GPFS: A Shared-Disk File System for Large Computing Clusters," Proc. Conf. File and Storage Technologies (FAST '02), 2002.
[7] W. Gropp, S.V. Lederman, A. Lumsdaine, E. Lusk, B. Nitzberg, W. Saphir, and M. Snir, MPI: The Complete Reference, the MPI-2 Extensions, vol. 2. The MIT Press, 1998.
[8] M. Dahlin, R. Wang, T.E. Anderson, and D.A. Patterson, "Cooperative Caching: Using Remote Client Memory to Improve File System Performance," Proc. Symp. Operating Systems Design and Implementation (OSDI '94), 1994.
[9] "OLTP Application and Search Engine I/O, UMASS Trace Repository," Storage, 2010.
[10] M.J. Feeley, W.E. Morgan, F.H. Pighin, A.R. Karlin, H.M. Levy, and C.A. Thekkath, "Implementing Global Memory Management in a Workstation Cluster," Proc. Symp. Operating Systems Principles (SOSP '95), 1995.
[11] G.M. Voelker, E.J. Anderson, T. Kimbrel, M.J. Feeley, J.S. Chase, A.R. Karlin, and H.M. Levy, "Implementing Cooperative Prefetching and Caching in a Globally-Managed Memory System," Proc. SIGMETRICS, 1998.
[12] P. Sarkar and J. Hartman, "Hint-Based Cooperative Caching," ACM Trans. Computer Systems, vol. 18, no. 4, pp. 387-419 , 2000.
[13] T. Anderson, M. Dahlin, J. Neefe, D. Patterson, D. Roselli, and R. Wang, "Serverless Network File Systems," Proc. Symp. Operating Systems Principles (SOSP '95), 1995.
[14] M. Dahlin, C. Mather, R. Wang, T.E. Anderson, and D.A. Patterson, "A Quantitative Analysis of Cache Policies for Scalable Network File Systems," Proc. SIGMETRICS, 1994.
[15] C. Ding and Y. Zhong, "Predicting Whole-Program Locality through Reuse Distance Analysis," Proc. Conf. Programming Language Design and Implementation (PLDI '03), 2003.
[16] Y. Zhong, M. Orlovich, X. Shen, and C. Ding, "Array Regrouping and Structure Splitting Using Whole-Program Reference Affinity," Proc. Conf. Programming Language Design and Implementation (PLDI '04), 2004.
[17] C. Fang, S. Carr, S. Onder, and Z. Wang, "Reuse-Distance-Based Miss-Rate Prediction on a per Instruction Basis," Proc. ACM SIGPLAN Workshop Memory System Performance, 2004.
[18] S. Jiang, K. Davis, F. Petrini, X. Ding, and X. Zhang, "A Locality-Aware Cooperative Cache Management Protocol to Improve Network File System Performance," Proc. Int'l Conf. Distributed Computing Systems (ICDCS '06), 2006.
[19] T.M. Wong and J. Wilkes, "My Cache or Yours? Making Storage More Exclusive," Proc. USENIX Ann. Technical Conf., 2002.
[20] M. Uysal, A. Acharya, and J. Saltz, "Requirements of I/O Systems for Parallel Machines: An Application-Driven Study," Technical Report UMIACS-TR-97-49, Univ. of Maryland at College Park, 1997.
[21] S. Jiang and X. Zhang, "LIRS: An Efficient Low Inter-Reference Recency Set Replacement Policy to Improve Buffer Cache Performance," Proc. SIGMETRICS, 2002.
[22] F. Petrini, S. Coll, E. Frachtenberg, and A. Hoisie, "Hardware- and Software-Based Collective Communication on the Quadrics Network," Proc. IEEE Int'l Symp. Network Computing and Application (NCA '01), 2001.
[23] S. Jiang, F. Chen, and X. Zhang, "CLOCK-Pro: An Effective Improvement of the CLOCK Replacement," Proc. USENIX Ann. Technical Conf., 2005.
[24] S. Jiang and X. Zhang, "ULC: A File Block Placement and Replacement Protocol to Effectively Exploit Hierarchical Locality in Multi-Level Buffer Caches," Proc. Int'l Conf. Distributed Computing Systems (ICDCS '04), 2004.
[25] Y. Zhou, J.F. Philbin, and K. Li, "The Multi-Queue Replacement Algorithm for Second-Level Buffer Caches," Proc. USENIX Ann. Technical Conf., June 2001.
[26] R. Chu, J. Xie, N. Xiao, and X. Lu, "RAM Grid Middleware for Autonomic Cooperative Caching," Proc. Sixth Int'l Conf. Grid and Cooperative Computing, 2007.
[27] I. Hwang, S. Maeng, and J. Cho, "Home-Based Cooperative Cache for Parallel I/O Applications," Future Generation Computer Systems, vol. 22, no. 5, pp. 633-642, 2006.
17 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool