This Article 
 Bibliographic References 
 Add to: 
An Adaptive Cache Coherence Protocol Specification for Parallel Input/Output Systems
June 2004 (vol. 15 no. 6)
pp. 533-545

Abstract—Caching has been intensively used in memory and traditional file systems to improve system performance. However, the use of caching in parallel file systems and I/O libraries has been limited to I/O nodes to avoid cache coherence problems. In this paper, we specify an adaptive cache coherence protocol very suitable for parallel file systems and parallel I/O libraries. This model exploits the use of caching, both at processing and I/O nodes, providing performance increase mechanisms as aggressive prefetching and delayed-write techniques. The cache coherence problem is solved by using a dynamic scheme of cache coherence protocols with different sizes and shapes of granularity. The proposed model is very appropriate for parallel I/O interfaces, as MPI-IO. Performance results, obtained on an IBM SP2, are presented to demonstrate the advantages offered by the cache management methods proposed.

[1] A. Calderon, F. Garcia, J. Carretero, F. Fernandez, and O. Perez, New Techniques for Collective Communications in Clusters: A Case Study with MPI Proc. 30th Int'l Conf. Parallel Processing, pp. 185-192, Sept. 2001.
[2] P.H. Carns, W.B. Ligon III, R.B. Ross, and R. Takhur, PVFS: A Parallel File System for Linux Clusters Technical Report ANL/MCS-P804-0400, 2000.
[3] J. Carretero, F. Perez, P. De Miguel, F. Garcia, and L. Alonso, Performance Increase Mechanisms for Parallel and Distributed File Systems Parallel Computing, vol. 23, pp. 525-542, Aug. 1997.
[4] A. Choudhary, R. Bordawekar, S. More, K. Sivaram, and R. Thakur, PASSION Runtime Library for the Intel Paragon Proc. Intel Supercomputer User's Group Conf., June 1995.
[5] P. Corbett and D. Feitelson, The Vesta Parallel File System ACM Trans. Computer Systems, vol. 14, no. 3, pp. 225-264, Aug. 1996.
[6] T. Cortes, S. Girona, and J. Labarta, Dessign Issues of a Cooperative Cache with no Coherence Problems Proc. Fifth Workshop I/O in Parallel and Distributed Systems, Nov. 1997.
[7] P.E. Crandall, R.A. Aydt, A.A. Chien, and D.A. Reed, Input/Output Characteristics of Scalable Parallel Applications Proc. Supercomputing Conf., 1995.
[8] Proc. High Performance Fortran Forum: High Performance Fortran Language Specification. May 1993.
[9] T. Fuerle, O. Jorns, E. Schikuta, and H. Wanek, Meta-ViPIOS: Harness Distributed I/O Resources with ViPIOS J. Research Computing and Systems, special issue on parallel computing, 1999.
[10] F. Garcia, A. Calderon, and J. Carretero, MiMPI: A Multithread-Safe Implementation of MPI Recent Advances in PVM and MPI, Sixth PVM/MPI European Users Group Meeting, J. Dongarra, E. Luque, and T. Margalef, eds., pp. 207-214, 1999.
[11] F. Garcia, A. Calderon, J. Carretero, J.M. Perez, and J. Fernandez, The Design of the Expand Parallel File System J. High Performance Computing Applications, pp. 21-37, Spring 2003.
[12] G. Gibson, The Scotch Paralell Storage Systems Technical Report CMU-CS-95-107, School of Computer Science, Carnegie Mellon Univ., Pittsburbh, Penn., 1995.
[13] J. Howard et al., Scale and Performance in a Distributed File System ACM Trans. Computer Systems, vol. 6, no. 1, pp. 51-81, Feb. 1988.
[14] J. Huber et al., PPFS: A High Performance Portable Parallel File System Proc. Ninth ACM Int'l Conf. Supercomputing, pp. 385-394, July 1995.
[15] M. Kallahalla and P.J. Varman, PC-OPT: Optimal Offline Prefetching and Caching for Parallel I/O Systems IEEE Trans. Computers, vol. 51, no. 11, Nov. 2002.
[16] D. Kotz, Prefetching and Caching Techniques in File Systems for MIMD Multiprocessors PhD thesis, Duke Univ., Apr. 1991.
[17] T.M. Madhyastha, Automatic Classification of Input/Output Access Patterns PhD thesis, Univ. of Illinois, Urbana-Champaign, 1997.
[18] T.M. Madhyastha and D.A. Reed, Learning to Classity Parallel Input/Ouput Access Patterns IEEE Trans. Parallel and Distributed Systems, vol. 13, no. 8, Aug. 2002.
[19] M. Nelson, B. Welch, and J. Ousterhout, Caching in the Sprite Network File System ACM Trans. Computer Systems, vol. 6, no. 1, pp. 134-154, Feb. 1988.
[20] N. Nieuwejaar and D. Kotz, The Galley Parallel File System Technical Report PCS-TR96-286, Dartmouth College Computer Science, 1996.
[21] R. Olfield and D. Kotz, The Armada Parallel File System, , 1998.
[22] D. Patterson, G. Gibson, and R. Katz, A Case for Redundant Arrays of Inexpensive Disks (RAID) Proc. ACM SIGMOD, pp. 109-116, June 1988.
[23] A. Purakayastha, C.S. Ellis, D. Kotz, N. Nieuwejaar, and M. Best, Characterizing Parallel File-Access Patterns on a Large-Scale Multiprocessor Proc. Ninth Int'l Parallel Processing Symp., pp. 165-172, Apr. 1995.
[24] A. Purakayastha, C.S. Ellis, and D. Kotz, ENWRICH: A Computer-Processor Write Caching Scheme for Parallel File Systems Technical Report TRCS-1995-22, Dept. of Computer Science, Duke Univ., Durham, N.C., Oct. 1995.
[25] J. del Rosario, R. Bordawekar, and A. Choundary, Improved Parallel I/O via a Two-Phase Run-Time Access Strategy ACM Computer Architecture News, vol. 21, no. 5, pp. 31-39, Dec. 1993.
[26] K. Salem and H. Garcia-Molina, Disk Striping Proc. Second Int'l Conf. Data Eng., pp. 336-342, Feb. 1986.
[27] R. Sandberg et al., Design and Implementation of the SUN Network Filesystem Proc. 1985 USENIX Conf., 1985.
[28] H. Wanek and E. Schikuta, Parallel I/O Cluster Computing White Paper, Dec. 2000.
[29] E. Smirni and D.A. Reed, Input/Output Intensive Parallel Applications J. High Performance Computing Applications, vol. 12, no. 3, pp. 364-380, 1998.
[30] D.J. Sorin, M. Plakal, A.E. Condon, M.D. Hill, M.M.K. Martin, and D.A. Wood, Specifying and Verifying a Broadcast and a Multicast Snooping Cache Coherence Protocol IEEE Trans. Parallel and Distributed Systems, vol. 13, no. 6, June 2002.
[31] W. Gropp, R. Takhur, and E. Lusk, An Abstract-Device Interface for Implementing Portable Parallel-I/O Interfaces Proc. Sixth Symp. Frontiers of Massively Parallel Computation, pp. 180-187, Oct. 1996.
[32] M. Vilayannur, A. Sivasubramaniam, M. Kandemir, and R. Thakur, Discretionary Caching for I/O on Clusters Proc. Third IEEE/ACM Int'l Symp. Cluster Computing and the Grid, pp. 96-103, May 2003.
[33] GPFS: General Parallel File Systems, softwaregpfs.html, 2004.

Index Terms:
Parallel file system, caching, cache coherence, protocol specification.
Felix Garcia-Carballeira, Jesus Carretero, Alejandro Calderon, Jose M. Perez, Jose D. Garcia, "An Adaptive Cache Coherence Protocol Specification for Parallel Input/Output Systems," IEEE Transactions on Parallel and Distributed Systems, vol. 15, no. 6, pp. 533-545, June 2004, doi:10.1109/TPDS.2004.1
Usage of this product signifies your acceptance of the Terms of Use.