This Article 
 Bibliographic References 
 Add to: 
Coding for High Availability of a Distributed-Parallel Storage System
December 1998 (vol. 9 no. 12)
pp. 1237-1252

Abstract—We have developed a distributed parallel storage system that employs the aggregate bandwidth of multiple data servers connected by a high-speed wide-area network to achieve scalability and high data throughput. This paper studies different schemes to enhance the reliability and availability of such network-based distributed storage systems. The general approach of this paper employs "erasure" error-correcting codes that can be used to reconstruct missing information caused by hardware, software, or human faults. The paper describes the approach and develops optimized algorithms for the encoding and decoding operations. Moreover, the paper presents techniques for reducing the communication and computation overhead incurred while reconstructing missing data from the redundant information. These techniques include clustering, multidimensional coding, and the full two-dimensional parity schemes. The paper considers trade-offs between redundancy, fault tolerance, and complexity of error recovery.

[1] M. Blaum, J. Brady, J. Bruck, and J. Menon, “EVENODD: An Efficient Scheme for Tolerating Double Disk Failures in RAID Architectures,” IEEE Trans. Computers, vol. 44, no. 2, pp. 192–202, Feb. 1995.
[2] L.T. Chen and D. Rotem, “Declustering Objects for Visualization,” Proc. Int'l Conf. Very Large Data Bases, pp. 85-96, 1993.
[3] G.C. Clark Jr. and J. Bibb Cain, Error-Correction Coding for Digital Communications.New York: Plenum Press, 1981.
[4] Y. Deswarte, L. Blain, and J.C. Fabre, "Intrusion Tolerance in Distributed Computing Systems," Proc. IEEE Symp. Security and Privacy, pp. 110-121,Oakland, Calif., 1991.
[5] S. Ghandeharizadeh and L. Ramos, “Continuous Retrieval of Multimedia Data Using Parallelism,” IEEE Trans. Knowledge and Data Eng., vol. 5, no. 4, pp. 658-669, Aug. 1993.
[6] S. Ghandeharizadeh, L. Ramos, Z. Asad, and W. Qureshi, “Object Placement in Parallel Hypermedia Systems,” Proc. Int'l Conf. Very Large Databases, 1991.
[7] J.H. Hartman and J.K. Ousterhout, "Zebra: A Striped Network File System," Proc. USENIX Workshop File Systems, May 1992.
[8] V. Jacobson, R. Braden, and D. Borman, "TCP Extensions for High Performance," Internet Engineering Task Force, Request for Comments (RFC) 1323, May 1992. (Available at dsdspg1intdoc.html ).
[9] B. Lamparter, O. Bohrer, W. Effelsberg, and V. Turau, "Adaptable Forward Error Correction for Multimedia Data Streams," Technical Report TR-93-009, Int'l Computer Science Inst., Berkeley, Calif., Dec. 1993.
[10] S. Lau and Y. Leclerc, "TerraVision: A Terrain Visualization System," Technical Note 540, SRI Int'l, Menlo Park, Calif., Mar. 1994. (Available at ).
[11] A.J. McAuley, “Reliable Broadband Communications Using Burst Erasure Correcting Code,” Proc. ACM SIGCOMM, pp. 287-306, Sept. 1990.
[12] D.A. Patterson, G. Gibson, and R.H. Katz, “A Case for Redundant Arrays of Inexpensive Disks (RAID),” Proc. ACM SIGMOD Conf., pp. 109–116, 1988.
[13] W.W. Peterson and E.J. Weldon, Error-Correcting Codes, second ed. Cambridge, Mass.: MIT Press, 1972.
[14] Delta-4: A Generic Architecture for Dependable Distributed Computing, D. Powel, ed. Springer-Verlag, 1991.
[15] M.O. Rabin, Efficient Dispersal of Information for Security, Load Balancing and Fault Tolerance J. ACM, vol. 36, no. 2, pp. 335-348, 1989.
[16] T.R.N. Rao and Fujiwara, Error-Coding for Computer Systems.Englewood Cliffs, N.J.: Prentice Hall, 1989.
[17] I. Richer and B.B. Fuller, "An Overview of the MAGIC Project," Technical Report M93B0000173, The MITRE Corp., Bedford, Mass., Dec.1 1993. (Available from
[18] M. Satyanarayanan, J. Kistler, P. Kumar, M. Okasaki, E. Siegel, and D. Steere, "Coda: A Highly Available File System for a Distributed Workstation Environment," IEEE Trans. Computers, vol. 39, no. 4, Apr. 1990.
[19] B. Tierney, W. Johnston, L.T. Chen, H. Herzog, G. Hoo, G. Jin, and J. Lee, "Using High Speed Networks to Enable Distributed Parallel Image Server Systems," Proc. Supercomputing '94, Nov. 1994. (Available at).
[20] B. Tierney, W. Johnston, L.T. Chen, H. Herzog, G. Hoo, G. Jin, J. Lee, and D. Rotem, "Distributed Parallel Data Storage Systems: A Scalable Approach for High Speed Image Servers," Proc. ACM Multimedia '94, Oct. 1994. (Available at).

Index Terms:
Storage systems, availability, scalability, RAID, high performance, distributed systems, error-correcting codes.
Qutaibah M. Malluhi, William E. Johnston, "Coding for High Availability of a Distributed-Parallel Storage System," IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 12, pp. 1237-1252, Dec. 1998, doi:10.1109/71.737699
Usage of this product signifies your acceptance of the Terms of Use.