This Article 
 Bibliographic References 
 Add to: 
Achieving Strong Consistency in a Distributed File System
January 1997 (vol. 23 no. 1)
pp. 35-55

Abstract—Distributed file systems nowadays need to provide for fault tolerance. This is typically achieved with the replication of files. Existing approaches to the construction of replicated file systems sacrifice strong semantics (i.e., the guarantees the systems make to running computations when failures occur and/or files are accessed concurrently). This is done mainly for efficiency reasons. This paper puts forward a replicated file system protocol that enforces strong consistency semantics. Enforcing strong semantics allows for distributed systems to behave more like their centralized counterparts—an essential feature in order to provide the transparency that is so strived for in distributed computing systems. One fundamental characteristic of our protocol is its distributed nature. Because of it, the extra cost needed to ensure the stronger consistency is kept low since the bottleneck problem noticed in primary-copy systems is avoided, load balancing is facilitated, clients can choose physically close servers, and the work required during failure handling and recovery is reduced. Another characteristic is that instead of optimizing each operation type on its own, file system activity was viewed at the level of a file session and the costs of individual operations were able to be spread over the life of a file session. We have developed a prototype and compared the performance of the prototype to both NFS and a nonreplicated version of the prototype that also achieves strong consistency semantics. Through these comparisons the cost of replication and the cost of enforcing the strong consistency semantics are shown.

[1] M. Baker, J.H. Hartman, M.D. Kupfer, K.W. Shirriff, and J. Ousterhout, "Measurements of a Distributed File System," Proc. 13th ACM Symp. Operating Systems Principles, pp. 198-211, Oct. 1991.
[2] M. Baker and J. Ousterhout, "Availability in the Sprite Distributed File System," Operating Systems Review, pp. 198-212, Apr. 1991.
[3] P. Bernstein, V. Hadzilacos, and N. Goodman, Concurrency Control and Recovery in Database Systems. Addison-Wesley, 1987.
[4] R. Floyd, "Short-Term File Reference Patterns in a UNIX Environment," Technical Report 177, Computer Science Dept., The Univ. of Rochester, New York, Mar. 1986.
[5] D.K. Gifford, “Weighted Voting for Replicated Data,” Proc. Seventh ACM SIGOPS Symp. Operating Systems Principles, pp. 150-159, Dec. 1979.
[6] R. Guy et al. "Implementation of the Ficus Replicated File System," Proc. USENIX Conf.,Anaheim Calif., pp. 63-71, June 1990.
[7] A. Hisgen et al., "Availability and Consistency Tradeoffs in the Echo Distributed File System," Proc. Second Workshop Workstation Operating Systems, pp. 49-54. IEEE CS Press, Sept. 1989.
[8] J. Kistler and M. Satyanarayanan, "Disconnected Operation in the Code File System," ACM 13th Symp. Operating Systems Principles, pp. 226-238, Oct. 1991.
[9] E. Levy and A. Silbershatz, "Distributed File Systems: Concepts and Examples," ACM Computing Surveys, vol. 22, no. 4, Dec. 1990.
[10] B. Liskov, S. Ghemawat, R. Gruber, P. Johnson, L. Shrira, and M. Williams, "Replication in the Harp file System," Proc. 13th SOSP, pp. 226-238, Oct. 1991.
[11] T. Mann, A. Hisgen, and G. Swart, "An Algorithm for Data Replication," Report 46, DEC System Research Center, Palo Alto, Calif., 1989.
[12] T. Mann et al., "A Coherent Distributed File Cache with Directory Write-Behind," ACM Trans. Computer Systems, vol. 12, no. 2, pp. 123-164, May 194.
[13] K. Marzullo and F. Schmuck, "Supplying High Availability with a Standard Network File System," Proc. Eighth Int'l Conf. Distributed Computing Systems,San Jose, Calif., pp. 447-453, 1988.
[14] L. Mummert, "Efficient Long-Term File Reference Tracing," Carnegie Mellon Univ., 1993, manuscript in preparation.
[15] J.-F. Paris, "Voting with Witnesses: A Consistency Scheme for Replicated Files," Proc. Sixth Int'l Conf. Distributed Computing Systems, pp. 606-612, May 1986.
[16] M. Satyanarayanan, J. Kistler, P. Kumar, M. Okasaki, E. Siegel, and D. Steere, "Coda: A Highly Available File System for a Distributed Workstation Environment," IEEE Trans. Computers, vol. 39, no. 4, Apr. 1990.
[17] A. Siegel, K. Birman, and K. Marzullo, "Deceit: A Flexible Distributed File System," Technical Report No. 89-1042, Dept.of Computer Science, Cornell Univ., Nov. 1989 (also in USENIX Conf. Proc., Anaheim Calif., p. 5,161, June 1990).
[18] C. Tait and D. Duchamp, "Service Interface and Replica Management Algorithm for Mobile File System Clients," First Int'l Conf. Parallel and Distributed Information Systems,Miami Beach Fla., pp. 190-197, Dec. 1991.
[19] C. Tait and D. Duchamp, "An Efficient Variable-Consistency Replicated File Service," Proc. USENIX File Systems Workshop,Ann Arbor Mich., pp. 111-126, May 1992.
[20] J. Thompson, "Efficient Analysis Of Caching Systems," Technical Report No. UCB/CSD 87/374, Computer Science Division, Univ. of California, Berkeley, Calif., Oct. 1987.
[21] P. Triantafillou and D.J. Taylor, "Multi-Class Replicated Data Management: Exploiting Replication to Improve Efficiency," IEEE Trans. Parallel and Distributed Systems, pp. 121-139, Feb. 1994.
[22] P. Triantafillou and D.J. Taylor, "The Location-Based Paradigm for Replication: Achieving Efficiency and Availability in Distributed Systems," IEEE Trans. Software Eng., vol. 21, no. 1, pp. 1-8, Jan. 1995.
[23] P. Triantafillou and D.J. Taylor, "VELOS: A New Approach for Efficiently Achieving High Availability in Partitioned Distributed Systems," IEEE Trans. Knowledge and Data Engineering, pp. 305-21, Apr. 1996.
[24] P. Triantafillou, "Availability and Performance Limitations in Multidatabases," Information Systems: An International Journal, vol. 21, no. 7, pp. 577-93, 1996.
[25] P. Triantafillou, "Independent Recovery in Large-Scale Distributed Systems," IEEE Trans. Software Eng., vol. 22, no. 11, Nov. 1996.

Index Terms:
Availability, caching, concurrency, consistency semantics, distributed file systems, recovery, replication.
Peter Triantafillou, Carl Neilson, "Achieving Strong Consistency in a Distributed File System," IEEE Transactions on Software Engineering, vol. 23, no. 1, pp. 35-55, Jan. 1997, doi:10.1109/32.581328
Usage of this product signifies your acceptance of the Terms of Use.