The Community for Technology Leaders
RSS Icon
Issue No.12 - Dec. (2012 vol.23)
pp: 2303-2314
Yunghsiang S. Han , National Taiwan University of Science and Technology, Taipei
Soji Omiwade , University of Houston, Houston
Rong Zheng , University of Houston, Houston
We propose a decentralized progressive data retrieval (PDR) mechanism for data reconstruction in a network of Byzantine and crash-stop nodes. The scheme progressively retrieves stored data, such that it achieves the minimum communication cost possible. In particular, PDR gracefully adapts the cost of successful data retrieval to the number of Byzantine and crash-stop storage nodes. At the core of PDR is an incremental Reed-Solomon decoding (IRD) procedure that is highly computation efficient for data reconstruction. IRD's computation efficiency arises from its ability to utilize intermediate computation results. In addition, we provide an in-depth analysis of PDR and compare it to decentralized erasure coding and decentralized fountain coding algorithms for distributed storage systems. Moreover, our implementation results show that PDR has up to 35 times lower computation time over the state-of-the-art error-erasure decoding scheme for distributed storage systems. In our analysis, we also show that the code structure of PDR and the number of available storage nodes are independent of each other, and they can be used to control both the data dissemination and retrieval complexity.
Decoding, Reed Solomon codes, Encoding, Distributed databases, Computer crashes, Fault tolerance, byzantine failure, Fault tolerance, error control codes, Reed-Solomon codes
Yunghsiang S. Han, Soji Omiwade, Rong Zheng, "Progressive Data Retrieval for Distributed Networked Storage", IEEE Transactions on Parallel & Distributed Systems, vol.23, no. 12, pp. 2303-2314, Dec. 2012, doi:10.1109/TPDS.2012.67
[1] C. Karlof and D. Wagner, "Secure Routing in Wireless Sensor Networks: Attacks and Countermeasures," Ad Hoc Networks, vol. 1, nos. 2/3, pp. 293-315, 2003.
[2] G.R. Goodson, J.J. Wylie, G.R. Ganger, and M.K. Reiter, "Efficient Byzantine-Tolerant Erasure-Coded Storage," Proc. Int'l Conf. Dependable Systems and Networks, pp. 135-144, July 2004.
[3] H. Krawczyk, "Distributed Fingerprints and Secure Information Dispersal," PODC '93: Proc. 12th Ann. ACM Symp. Principles of Distributed Computing, pp. 207-218, 1993.
[4] T.S.D. Sheet, "Moteiv, San Francisco, CA, 2006," 2004.
[5] R. Ramjee, J. Kurose, D. Towsley, and H. Schulzrinne, "Adaptive Playout Mechanisms for Packetized Audio Applications in Wide-Area Networks," Proc. IEEE INFOCOM '94, vol. 2, pp. 680-688, June 1994.
[6] J.S. Plank, J. Luo, C.D. Schuman, L. Xu, and Z. Wilcox-O$^\prime$ Hearn, "A Performance Evaluation and Examination of Open-Source Erasure Coding Libraries for Storage," FAST '09: Proc. Seventh Conf. File and Storage Technologies, pp. 253-265, 2009.
[7] Y. Lin, B. Liang, and B. Li, "Data Persistence in Large-Scale Sensor Networks with Decentralized Fountain Codes," Proc. 26th IEEE INFOCOM, pp. 6-12, 2007.
[8] A.G. Dimakis, V. Prabhakaran, and K. Ramchandran, "Decentralized Erasure Codes for Distributed Networked Storage," IEEE Trans. Information Theory, vol. 52, no. 6, pp. 2809-2816, June 2006.
[9] T.K. Moon, Error Correction Coding: Mathematical Methods and Algorithms. John Wiley & Sons, 2005.
[10] S. Lin and D.J. CostelloJr., Error Control Coding: Fundamentals and Applications, second ed. Prentice-Hall, 2004.
[11] T.K. Truong, W.L. Eastman, I.S. Reed, and I.S. Hsu, "Simplified Procedure for Correcting both Errors and Erasures of Reed-Solomon Code Using Euclidean Algorithm," Proc. IEE, vol. 135, no. 6, pp. 318-324, Nov. 1988.
[12] S.-L. Shieh, S.-G. Lee, and W.-H. Sheen, "A Low-Latency Decoder for Punctured/Shortened Reed-Solomoncodes," Proc. IEEE Int'l Symp. Personal, Indoor and Mobile Radio Comm., pp. 2547-2551, Sept. 2005.
[13] J. Kurose and K. Ross, Computer Networks: A Top Down Approach Featuring the Internet. Pearson Addison Wesley, 2005.
[14] I.S. Reed and X. Chen, Error-Control Coding for Data Networks. Kluwer Academic, 1999.
[15] Y.S. Han, S. Omiwade, and R. Zheng, "Survivable Distributed Storage with Progressive Decoding," Proc. IEEE INFOCOM, pp. 1-5, Mar. 2010.
[16] K. Araki, M. Takada, and M. Morii, "On the Efficient Decoding of Reed-Solomon Codes Based on GMD Criterion," Proc. Int'l Symp. Multiple-Valued Logic, pp. 138-145, May 1992.
[17] H. William, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing. Cambridge Univ. Press, 1988.
[18] M.N. Krohn, M.J. Freedman, and D. Mazieres, "On-the-Fly Verification of Rateless Erasure Codes for Efficient Content Distribution," Proc. IEEE Symp. Security and Privacy, pp. 226-240, May 2004.
27 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool