This Article 
 Bibliographic References 
 Add to: 
Cyclic Storage for Fault-Tolerant Distributed Executions
September 2006 (vol. 17 no. 9)
pp. 1028-1036

Abstract—Given a set V of active components in charge of a distributed execution, a storage scheme is a sequence B_{0}, B_{1}, \ldots, B_{b-1} of subsets of V, where successive global states are recorded. The subsets, also called blocks, have the same size and are scheduled according to some fixed and cyclic calendar of b steps. During the i\rm th step, block B_{i} is selected. Each component takes a copy of its local state and sends it to one of the components in B_i, in such a way that each component stores (approximately) the same number of local states. Afterward, if a component of B_{i} crashes, all of its stored data is lost and the computation cannot continue. If there exists a block with no failed components in it, then a recent global state can be retrieved and the computation does not need to start over from the very beginning. The goal is to design storage schemes that tolerate as many crashes as possible, while trying to have each component participating in as few blocks as possible and, at the same time, working with large blocks (so that a component in a block stores a small number of local states). In this paper, several such schemes are described and compared in terms of these measures.

[1] A. Azagury, M.E. Factor, and J. Satran, “Point-in-Time Copy: Yesterday, Today and Tomorrow,” Proc. 19th IEEE Symp. Mass Storage Systems, pp. 259-270, 2002.
[2] O. Babaoglu and K. Marzullo, “Consistent Global States of Distributed Systems: Fundamental Concepts and Mechanisms,” Distributed Systems, pp. 55-96, 1993.
[3] L.M. Batten, Combinatorics of Finite Geometries. Cambridge Univ. Press, 1997.
[4] P. Berenbrink, A. Brinkmann, and C. Scheideler, “Design of the PRESTO Multimedia Storage Network,” Proc. Int'l Workshop Comm. and Data Management in Large Networks, pp. 2-12, 1999.
[5] R. Bhagwan, D. Moore, S. Savage, and G. Voelker, “Replication Strategies for Highly Available Peer-to-Peer Storage,” Proc. Int'l Workshop Future Directions in Distributed Computing, 2002.
[6] A.E. Brouwer and A. Schrijver, “The Blocking Number of an Affine Space,” J. Combinatorial Theory Series A, vol. 24, no. 2, pp. 251-253, 1978.
[7] A.A. Bruen and R. Silverman, “Arcs and Blocking Sets II,” European J. Combinatorics, vol. 8, no. 4, pp. 351-356, 1987.
[8] J.N. Cederberg, A Course in Modern Geometries. Springer-Verlag, 1989.
[9] M. Chandy and L. Lamport, “Distributed Snapshots: Determining Global States in Distributed Systems,” ACM Trans. Computer Science, vol. 3, no. 1, pp. 63-75, 1985.
[10] T. Chiueh and P. Deng, “Evaluation of Checkpoint Mechanisms for Massively Parallel Machines,” Proc. 26th Ann. Int'l Symp. Fault-Tolerant Computing (FTCS '96), pp. 370-381, 1996.
[11] M.B. Cohen and C.J. Colbourn, “Steiner Triple Systems as Multiple Erasure Correcting Codes in Disk Arrays,” Proc. 19th IEEE Int'l Conf. Performance, Computing, and Comm., pp. 288-294, 2000.
[12] M.B. Cohen and C.J. Colbourn, “Ordering Disks for Double Erasure Codes,” Proc. Symp. Parallel Algorithms and Architectures (SPAA '01), pp. 229-236, 2001.
[13] M.B. Cohen and C.J. Colbourn, “Optimal and Pessimal Orderings,” Discrete Applied Math., 2002.
[14] M.B. Cohen and C.J. Colbourn, “Optimal and Pessimal Orderings of Steiner Triple Systems in Disk Arrays,” Theoretical Computer Science, vol. 297, pp. 103-117, 2003.
[15] C.J. Colbourn et al., “Applications of Combinatorial Designs to Communications, Cryptography, and Networking,” technical report, Univ. of Vermont, 2000.
[16] P. Dembowski, Finite Geometries. Springer-Verlag, 1968.
[17] Y. Deswarte, “Tolérance aux Fautes, Sécurité et Protection,” Construction des Systèmes d'Exploitation Répartis, INRIA, 1991.
[18] E.N. Elnozahy, D. Johnson, and W. Zwanaepoel, “The Performance of Consistent Checkpoints,” Proc. 11th IEEE Symp. Reliable Distributed Systems, pp. 86-95, 1992.
[19] J.B. Fraleigh, Álgebra Abstracta. Addison-Wesley Iberoamericana, 1987.
[20] V.K. Garg and J. Ghosh, “Repeated Computation of Global Functions in a Distributed Environment,” IEEE Trans. Parallel and Distributed Systems, vol. 5, no. 8, pp. 823-834, 1994.
[21] R. Koo and S. Toueg, “Checkpointing and Roll-Back Recovery for Distributed Systems,” IEEE Trans. Software Eng., vol. 13, no. 1, pp. 23-31, 1987.
[22] D. Kotz, “Introduction to Multiprocessor I/O Architecture,” Input/Output in Parallel and Distributed Computer Systems, Kluwer Academic, 1996.
[23] D.L. Kreher and D.R. Stinson, Combinatorial Algorithms. CRC Press. 1998.
[24] K. Li, J.F. Naughton, and J.S. Plank, “Real Time, Concurrent Checkpoint for Parallel Programs,” Proc. Second ACM SIGPLAN Symp. Principles and Practice of Parallel Programming, pp. 79-88, 1990.
[25] M.J. Litzkow, M. Livny, and M.W. Mutka, “Condor— A Hunter of Idle Workstations,” Proc. Eighth Int'l Conf. Distributed Computing Systems, pp. 104-111, 1988.
[26] N. Lynch, Distributed Algorithms. Morgan Kaufman, 1996.
[27] F.J. MacWilliams and N.J. Sloane, The Theory of Error-Correcting Codes. North-Holland, 1993.
[28] R. Marcelín-Jiménez, “Performance Measures for Distributed Storage,” Proc. Design, Analysis, and Simulation of Distributed Systems (DASD), pp. 53-62, 2005.
[29] A.N. Mourad, K.W. Fuchs, and D.G. Saab, “Site Partitioning for Redundant Arrays of Distributed Disks,” J. Parallel and Distributed Computing, vol. 33, pp. 1-11, 1996.
[30] D. Peleg and A. Wool, “The Availability of Quorum Systems,” Information and Computation, vol. 123, no. 2, pp. 210-223, 1995.
[31] M.O. Rabin, “Efficient Dispersal of Information for Security, Load Balancing, and Fault Tolerance,” J. ACM, vol. 36, no. 2, pp. 335-348, 1989.
[32] S. Rhea et al., “Maintenance-Free Global Data Storage,” IEEE Internet Computing, pp. 40-49, Sept. 2001.
[33] A. Russell and A.A. Shvartsman, “Distributed Computation Meets Design Theory: Local Schedulling for Disconnected Operations,” Bull. EATCS, pp. 120-131, June 2002.
[34] D.R. Stinson, “An Introduction to Combinatorial Designs,” technical report, Princeton Univ., 1999.
[35] V. Tonchev, Combinatorial Configurations Designs, Codes, Graphs. Longman Scientific and Technical, 1988.
[36] Z. Yang and T.A. Marsland, “Global Snapshots for Distributed Debugging: An Overview,” technical report, Univ. of Alberta, 1992.

Index Terms:
Load balancing and task assignment, distributed applications, checkpoint/restart, fault-tolerance, storage/repositories, distributed systems, network repositories/data mining/backup.
Ricardo Marcel?n-Jim?nez, Sergio Rajsbaum, Brett Stevens, "Cyclic Storage for Fault-Tolerant Distributed Executions," IEEE Transactions on Parallel and Distributed Systems, vol. 17, no. 9, pp. 1028-1036, Sept. 2006, doi:10.1109/TPDS.2006.120
Usage of this product signifies your acceptance of the Terms of Use.