The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - June (2010 vol.21)
pp: 827-840
Haiying Shen , Clemson University. Clemson
ABSTRACT
In peer-to-peer file sharing systems, file replication technology is widely used to reduce hot spots and improve file query efficiency. Most current file replication methods replicate files in all nodes or two end points on a client-server query path. However, these methods either have low effectiveness or come at a cost of high overhead. File replication in server side enhances replica hit rate, hence, lookup efficiency but produces overloaded nodes and cannot significantly reduce query path length. File replication in client side could greatly reduce query path length, but cannot guarantee high replica hit rate to fully utilize replicas. Though replication along query path solves these problems, it comes at a high cost of overhead due to more replicas and produces underutilized replicas. This paper presents an Efficient and Adaptive Decentralized (EAD) file replication algorithm that achieves high query efficiency and high replica utilization at a significantly low cost. EAD enhances the utilization of file replicas by selecting query traffic hubs and frequent requesters as replica nodes, and dynamically adapting to nonuniform and time-varying file popularity and node interest. Unlike current methods, EAD creates and deletes replicas in a decentralized self-adaptive manner while guarantees high replica utilization. Theoretical analysis shows the high performance of EAD. Simulation results demonstrate the efficiency and effectiveness of EAD in comparison with other approaches in both static and dynamic environments. It dramatically reduces the overhead of file replication, and yields significant improvements on the efficiency and effectiveness of file replication in terms of query efficiency, replica hit rate, and overloaded nodes reduction.
INDEX TERMS
Peer-to-peer system, distributed hash table, file sharing system, file replication.
CITATION
Haiying Shen, "An Efficient and Adaptive Decentralized File Replication Algorithm in P2P File Sharing Systems", IEEE Transactions on Parallel & Distributed Systems, vol.21, no. 6, pp. 827-840, June 2010, doi:10.1109/TPDS.2009.127
REFERENCES
[1] S. Saroiu, P. Gummadi, and S. Gribble, "A Measurement Study of Peer-to-Peer File Sharing Systems," Proc. Conf. Multimedia Computing and Networking (MMCN), 2002.
[2] A. Rowstron and P. Druschel, "Storage Management and Caching in PAST, a Large-Scale, Persistent Peer-to-Peer Storage Utility," Proc. Symp. Operating Systems Principles (SOSP), 2001.
[3] F. Dabek et al., "Wide Area Cooperative Storage with CFS," Proc. Symp. Operating Systems Principles (SOSP), 2001.
[4] T. Stading et al., "Peer-to-Peer Caching Schemes to Address Flash Crowds," Proc. Int'l Workshop Peer-to-Peer Systems (IPTPS), 2002.
[5] M. Theimer and M. Jones, "Overlook: Scalable Name Service on an Overlay Network," Proc. Int'l Conf. Distributed Computing Systems (ICDCS), 2002.
[6] V. Gopalakrishnan et al., "Adaptive Replication in Peer-to-Peer Systems," Proc. Int'l Conf. Distributed Computing Systems (ICDCS), 2004.
[7] Gnutella, http:/www.gnutella.com, 2008.
[8] M. Roussopoulos and M. Baker, "CUP: Controlled Update Propagation in Peer to Peer Networks," Proc. USENIX, 2003.
[9] L. Yin and G. Cao, "DUP: Dynamic-Tree Based Update Propagation in Peer-to-Peer Networks," Proc. Int'l Conf. Data Eng. (ICDE), 2005.
[10] R. Cox, A. Muthitacharoen, and R.T. Morris, "Serving DNS Using a Peer-to-Peer Lookup Service," Proc. Int'l Workshop Peer-to-Peer Systems (IPTPS), 2002.
[11] P. Gummadi, R. Dunn, S. Saroiu, S. Gribble, H. Levy, and J. Zahorjan, "Measurement, Modeling, and Analysis of a Peer-to-Peer File-Sharing Workload," Proc. Symp. Operating Systems Principles (SOSP), 2003.
[12] C. Plaxton, R. Rajaraman, and A. Richa, "Accessing Nearby Copies of Replicated Objects in a Distributed Environment," Proc. ACM Symp. Parallel Algorithms and Architectures (SPAA), 1997.
[13] P. Godfrey and I. Stoica, "Heterogeneity and Load Balance in Distributed Hash Tables," Proc. IEEE INFOCOM, 2005.
[14] H. Shen and C. Xu, "Elastic Routing Table with Provable Performance for Congestion Control in DHT Networks," Proc. Int'l Conf. Distributed Computing Systems (ICDCS), 2006.
[15] Q. Lv, S. Ratnasamy, and S. Shenker, "Can Heterogeneity Make Gnutella Scalable?" Proc. Int'l Workshop Peer-to-Peer Systems (IPTPS), 2002.
[16] Y.-L. Chou, Statistical Analysis. Holt Int'l, 1975.
[17] I. Stoica et al., "Chord: A Scalable Peer-to-Peer Lookup Protocol for Internet Applications," IEEE/ACM Trans. Networking, vol. 1, no. 1, pp. 17-32, Feb. 2003.
[18] A. Rowstron and P. Druschel, "Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems," Proc. Middleware Conf., 2001.
[19] A. Adya et al., "Farsite: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment," Proc. Symp. Operating Systems Design and Implementation (OSDI), 2002.
[20] J.R. Douceur, A. Adya, W.J. Bolosky, D. Simon, and M. Theimer, "Reclaiming Space from Duplicate Files in a Serverless Distributed File System," Proc. Int'l Conf. Distributed Computing Systems, 2002.
[21] J.R. Douceur and R.P. Wattenhofer, "Optimizing File Availability in a Secure Serverless Distributed File System," Proc. Symp. Reliable Distributed Systems (SRDS), pp. 4-13, 2001.
[22] I. Clarke et al., "Freenet: A Distributed Anonymous Information Storage and Retrieval System," Proc. Int'l Workshop Design Issues in Anonymity and Unobservability, pp. 46-66, 2001.
[23] A. Ghodsi, L. Alima, and S. Haridi, "Symmetric Replication for Structured Peer-to-Peer Systems," Proc. Int'l Workshop Databases, Information Systems and Peer-to-Peer Computing, p. 12, 2005.
[24] T. Pitoura, N. Ntarmos, and P. Triantafillou, "Replication, Load Balancing and Efficient Range Query Processing in DHTs," Proc. Conf. Extending Database Technology (EDBT), 2006.
[25] K. Huang et al., "LessLog: A Logless File Replication Algorithm for Peer-to-Peer Distributed Systems," Proc. Int'l Parallel and Distributed Processing Symp. (IPDPS), 2004.
[26] J. Ni, J. Lin, S.J. Harrington, and N. Sharma, "Designing File Replication Schemes for Peer-to-Peer File Sharing Systems," Proc. IEEE Int'l Conf. Comm. (ICC), pp. 5609-5613, 2008.
[27] P. Knezević, A. Wombacher, and T. Risse, "DHT-Based Self-Adapting Replication Protocol for Achieving High Data Availability," Proc. Conf. Signal-Image Technology and Internet-Based Systems (SITIS), 2008.
[28] W.K. Lin, D.M. Chiu, and Y.B. Lee, "Erasure Code Replication Revisited," Proc. Int'l Conf. Peer-to-Peer Computing, 2004.
[29] R. Rodrigues and B. Liskov, "High Availability in DHTs: Erasure Coding versus Replication," Proc. Int'l Workshop Peer-to-Peer Systems, 2005.
[30] C. Gkantsidis and P. Rodriguez, "Network Coding for Large Scale Content Distribution," Proc. IEEE INFOCOM, 2005.
[31] A. Haeberlen, A. Mislove, and P. Druschel, "Glacier: Highly Durable, Decentralized Storage Despite Massive Correlated Failures," Proc. Symp. Networked Systems Design and Implementation, 2005.
[32] J. Kangasharju, K.W. Ross, and D.A. Turner, "Optimizing File Availability in Peer-to-Peer Content Distribution," Proc. IEEE INFOCOM, 2007.
[33] J. Kangasharju, K.W. Ross, and D.A. Turner, "Adaptive Content Management in Structured P2P Communities," Proc. Int'l Conf. Scalable Information Systems (Infoscale), 2006.
[34] S. Tewari and L. Kleinrock, "Proportional Replication in Peer-to-Peer Networks," Proc. IEEE INFOCOM, 2006.
[35] L. Massoulie and M. Vojnovic, "Coupon Replication Systems," Proc. ACM SIGMETRICS, 2005.
[36] S. Tewari and L. Kleinrock, "On Fairness, Optimal Download Performance and Proportional Replication in Peer-to-Peer Networks," Proc. IFIP Networking Conf., 2005.
[37] E. Cohen and S. Shenker, "Replication Strategies in Unstructured Peer-To-Peer Networks," Proc. ACM SIGCOMM, 2002.
[38] S. Tewari and L. Kleinrock, "Analysis of Search and Replication in Unstructured Peer-to-Peer Networks," Proc. ACM SIGMETRICS, 2005.
[39] D. Rubenstein and S. Sahu, "Can Unstructured P2P Protocols Survive Flash Crowds?" IEEE/ACM Trans. Networking, vol. 3, no. 3, pp. 501-512, 2005.
[40] Q. Lv and K. Li, "Search and Replication in Unstructured Peer-to-Peer Networks," Proc. Int'l Conf. Supercomputing (ICS), 2002.
[41] H. Shen, "EAD: An Efficient and Adaptive Decentralized File Replication Algorithm in P2P File Sharing Systems," Proc. Int'l Conf. Peer-to-Peer Computing (P2P), 2008.
19 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool