This Article 
 Bibliographic References 
 Add to: 
The Design and Evaluation of a Self-Organizing Superpeer Network
March 2010 (vol. 59 no. 3)
pp. 317-331
Paweł Garbacki, Delft University of Technology, Delft
Dick H.J. Epema, Delft University of Technology, Delft
Maarten van Steen, Vrije Universiteit Amsterdam, Amsterdam
Superpeer architectures exploit the heterogeneity of nodes in a peer-to-peer (P2P) network by assigning additional responsibilities to higher capacity nodes. In the design of a superpeer network for file sharing, several issues have to be addressed: how client peers are related to superpeers, how superpeers locate files, how the load is balanced among the superpeers, and how the system deals with node failures. In this paper, we introduce a self-organizing superpeer network architecture (SOSPNet) that solves these issues in a fully decentralized manner. SOSPNet maintains a superpeer network topology that reflects the semantic similarity of peers sharing content interests. Superpeers maintain semantic caches of pointers to files, which are requested by peers with similar interests. Client peers, on the other hand, dynamically select superpeers offering the best search performance. We show how this simple approach can be employed not only to optimize searching, but also to solve generally difficult problems encountered in P2P architectures such as load balancing and fault tolerance. We evaluate SOSPNet using a model of the semantic structure derived from eight-month traces of two large file-sharing communities. The obtained results indicate that SOSPNet achieves close-to-optimal file search performance, quickly adjusts to changes in the environment (node joins and leaves), survives even catastrophic node failures, and efficiently distributes the system load taking into account superpeer capacities.

[1] , 2009.
[2] http:/, 2007.
[3] http:/, 2009.
[4] http:/, 2009.
[5] http:/, 2009.
[6] http:/, 2007.
[7] http:/, 2009.
[8] http:/, 2009.
[9] Y. Breitbart, R. Vingralek, and G. Weikum, “Load Control in Scalable Distributed File Structures,” Distributed and Parallel Databases, vol. 4, no. 4, pp. 319-354, Oct. 1996.
[10] L. Breslau, P. Cao, L. Fan, G. Phillips, and S. Shenker, “Web Caching and Zipf-Like Distributions: Evidence and Implications,” Proc. INFOCOM, pp. 126-134, Mar. 1999.
[11] Y. Busnel and A.-M. Kermarrec, “Proxsem: Interest-Based Proximity Measure to Improve Search Efficiency in p2p Systems,” Proc. European Conf. Universal Multiservice Networks (ECUMN '07), Feb. 2007.
[12] C. Chatfield, A.B. Koehler, J.K. Ord, and R.D. Snyder, “A New Look at Models for Exponential Smoothing,” J. Royal Statistical Soc., Series D, The Statistician, vol. 50, pp. 147-159, 2001.
[13] Y. Chawathe, S. Ratnasamy, L. Breslau, and S. Shenker, “Making Gnutella Like p2p Systems Scalable,” Proc. SIGCOMM, Aug. 2003.
[14] Y. Chen, Z. Xu, and C. Zhai, “A Scalable Semantic Indexing Framework for Peer-to-Peer Information Retrieval,” Proc. Workshop Heterogeneous and Distributed Information Retrieval, Aug. 2005.
[15] V. Cholvi, P. Felber, and E. Biersack, “Efficient Search in Unstructured Peer-to-Peer Networks,” Proc. Symp. Parallelism in Algorithms and Architectures (SPAA '04), June 2004.
[16] I. Clarke, O. Sandberg, B. Wiley, and T.W. Hong, “Freenet: A Distributed Anonymous Information Storage and Retrieval System,” Lecture Notes in Computer Science, pp. 46-66, Springer, 2001.
[17] B. Cohen, “Incentives Build Robustness in Bittorrent,” Proc. First Workshop Economics of Peer-to-Peer Systems, May 2003.
[18] A. Crespo and H. Garcia-Molina, “Semantic Overlay Networks for p2p Systems,” technical report, Stanford Univ., Sept. 2002.
[19] T. Decker, R. Luling, and S. Tschoke, “A Distributed Load Balancing Algorithm for Heterogeneous Parallel Computing System,” Proc. Int'l Conf. Parallel and Distributed Processing Techniques and Applications, Nov. 2000.
[20] F. Le Fessant, S. Handurukande, A.-M. Kermarrec, and L. Massoulie, “Clustering in Peer-to-Peer File Sharing Workloads,” Proc. Int'l Workshop Peer-to-Peer Systems (IPTPS '04), Feb. 2004.
[21] A.J. Ganesh, A.-M. Kermarrec, and L. Massouli, “Peer-to-Peer Membership Management for Gossip-Based Protocols,” IEEE Trans. Computers, vol. 52, no. 2, pp. 139-149, Feb. 2003.
[22] P. Garbacki, D.H.J. Epema, and M. van Steen, “Two-Level Semantic Caching Scheme for Super-Peer Networks,” Proc. IEEE 10th Int'l Workshop Web Content Caching and Distribution, Sept. 2005.
[23] L. Garces-Erice, E.W. Biersack, K.W. Ross, P.A. Felber, and G. Urvoy-Keller, “Hierarchical Peer-to-Peer Systems,” Proc. ACM/IFIP Int'l Conf. Parallel and Distributed Computing (Euro-Par), 2003.
[24] K.P. Gummadi, R.J. Dunn, S. Saroiu, S.D. Gribble, H.M. Levy, and J. Zahorjan, “Measurement, Modeling, and Analysis of a Peer-to-Peer File-Sharing Workload,” Proc. Symp. Operating Systems Principles (SOSP '03), Oct. 2003.
[25] S. Handurukande, A.-M. Kermarrec, F. Le Fessant, and L. Massoulie, “Exploiting Semantic Clustering in the Edonkey p2p Network,” Proc. 11th ACM SIGOPS European Workshop, Sept. 2004.
[26] S.B. Handurukande, A.M. Kermarrec, F. Le Fessant, L. Massoulie, and S. Patarin, “Peer Sharing Behaviour in the Edonkey Network, and Implications for the Design of Server-Less File Sharing Systems,” Proc. EuroSys Conf., Apr. 2006.
[27] M. Hutter, “Self-Optimizing and Pareto-Optimal Policies in General Environments Based on Bayes-Mixtures,” Proc. 15th Ann. Conf. Computational Learning Theory (COLT '02), 2002.
[28] A. Iamnitchi, M. Ripeanu, and I. Foster, “Small-World File-Sharing Communities,” Proc. IEEE INFOCOM, Mar. 2004.
[29] A. Iosup, P. Garbacki, J. Pouwelse, and D. Epema, “Correlating Topology and Path Characteristics of Overlay Networks and the Internet,” Proc. Global and Peer-to-Peer Computing Workshop (GP2PC '06) in Conjunction with the IEEE/ACM Int'l Symp. Cluster Computing and the Grid (CCGrid '06), May 2006.
[30] M. Jelasity, A. Montresor, and O. Babaoglu, “A Modular Paradigm for Building Self-Organizing Peer-to-Peer Applications,” Engineering Self-Organising Systems: Nature Inspired Approaches to Software Engineering, G. Di Marzo Serugendo, A. Karageorgos, O.F. Rana, and F. Zambonelli, eds., pp. 265-282, Springer-Verlag, Apr. 2004.
[31] D.R. Karger and M. Ruhl, “Simple Efficient Load Balancing Algorithms for Peer-to-Peer Systems,” Proc. 16th Ann. ACM Symp. Parallelism in Algorithms and Architectures (SPAA '04), pp. 36-43, 2004.
[32] N. Laoutaris, S. Syntila, and I. Stavrakakis, “Meta Algorithms for Hierarchical Web Caches,” Proc. IEEE Int'l Performance Computing and Comm. Conf. (IEEE IPCCC), Apr. 2004.
[33] S. Le Blond, J.-L. Guillaume, and M. Latapy, “Clustering in p2p Exchanges and Consequences on Performances,” Proc. Int'l Workshop Peer-to-Peer Systems (IPTPS '05), Feb. 2005.
[34] J. Liang, R. Kumar, and K.W. Ross, “The Kazaa Overlay: A Measurement Study,” Computer Networks, special issue on overlays, vol. 49, no. 6, Oct. 2005.
[35] A. Loser, M. Wolpers, W. Siberski, and W. Nejdl, “Semantic Overlay Clusters within Super-Peer Networks,” Proc. Int'l Workshop Databases, Information Systems, and P2P Computing, Collocated with 29th Int'l Conf. Very Large Databases (VLDB '03), Sept. 2003.
[36] Q. Lv, P. Cao, E. Cohen, K. Li, and S. Shenker, “Search and Replication in Unstructured Peer-to-Peer Networks,” Proc. Int'l Conf. Supercomputing (ICS '02), June 2002.
[37] D.S. Milojicic, V. Kalogeraki, R. Lukose, K. Nagaraja, J. Pruyne, B. Richard, S. Rollins, and Z. Xu, “Peer-to-Peer Computing,” Technical Report HPL-2002-57, HP Labs, Mar. 2002.
[38] A.T. Mizrak, Y. Cheng, V. Kumar, and S. Savage, “Structured Superpeers: Leveraging Heterogeneity to Provide Constant-Time Lookup,” Proc. Third IEEE Workshop Internet Applications, June 2003.
[39] A. Montresor, “A Robust Protocol for Building Superpeer Overlay Topologies,” Proc. Fourth Int'l Conf. Peer-to-Peer Computing, Aug. 2004.
[40] W. Nejdl, B. Wolf, C. Qu, S. Decker, M. Sintek, A. Naeve, M. Nilsson, M. Palmer, and T. Risch, “Edutella: A p2p Networking Infrastructure Based on rdf,” Proc. 11th Int'l World Wide Web Conf. (WWW '02), May 2002.
[41] W. Nejdl, M. Wolpers, W. Siberski, C. Schmitz, M. Schlosser, I. Brunkhorst, and A. Loser, “Super-Peer-Based Routing Strategies for Rdf-Based Peer-to-Peer Networks,” Web Semantics, vol. 1, no. 2, pp. 177-186, Feb. 2004.
[42] A. Parker, “P2p in 2005,” http://www.cachelogic.comresearch/, 2005.
[43] J.A. Pouwelse, P. Garbacki, D.H.J. Epema, and H.J. Sips, “The Bittorrent p2p File-Sharing System: Measurements and Analysis,” Proc. Int'l Workshop Peer-to-Peer Systems (IPTPS '05), Feb. 2005.
[44] A. Rao, K. Lakshminarayanan, S. Surana, R. Karp, and I. Stoica, “Load Balancing in Dynamic Structured p2p Systems,” Proc. IEEE INFOCOM, Mar. 2004.
[45] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker, “A Scalable Content Addressable Network,” Proc. SIGCOMM, 2001.
[46] M. Ripeanu, “Peer-to-Peer Architecture Case Study: Gnutella Network,” Proc. First Int'l Conf. Peer-to-Peer Computing (P2P '01 ), Aug. 2001.
[47] J. Risson and T. Moors, “Survey of Research Towards Robust Peer-to-Peer Networks: Search Methods,” Computer Networks, vol. 50, no. 17, pp. 3485-3521, 2006.
[48] M.T. Schlosser, T.E. Condie, and S.D. Kamvar, “Simulating a File-Sharing p2p Network,” Proc. First Workshop Semantics in P2P and Grid Computing, May 2003.
[49] A. Singla and C. Rohrs, “Ultrapeers: Another Step Towards Gnutella Scalability,” , 2001.
[50] K. Sripanidkulchai, B. Maggs, and H. Zhang, “Efficient Content Location Using Interest-Based Locality in Peer-to-Peer Systems,” Proc. INFOCOM, Apr. 2003.
[51] I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, and H. Balakrishnan, “Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications,” Proc. SIGCOMM, pp. 149-160, 2001.
[52] C. Tang, Z. Xu, and S. Dwarkadas, “Peer-to-Peer Information Retrieval Using Self-Organizing Semantic Overlay Networks,” Proc. SIGCOMM, Aug. 2003.
[53] C. Tempich, S. Staab, and A. Wranik, “Remindin': Semantic Query Routing in Peer-to-Peer Networks Based on Social Metaphors,” Proc. 13th Int'l World Wide Web Conf., May 2004.
[54] S. Vanichpun and A.M. Makowski, “The Output of a Cache under the Independent Reference Model—Where Did the Locality of Reference Go?” Proc. SIGMETRICS 2004/PERFORMANCE 2004: Joint Int'l Conf. Measurement and Modeling of Computer Systems, pp.295-306, 2004.
[55] S. Voulgaris, A.-M. Kermarrec, L. Massoulie, and M. van Steen, “Exploiting Semantic Proximity in Peer-to-Peer Content Searching,” Proc. Int'l Workshop Future Trends of Distributed Computing Systems (FTDCS '04), May 2004.
[56] S. Voulgaris and M. van Steen, “Epidemic-Style Management of Semantic Overlays for Content-Based Searching,” Proc. EuroPar 2005, Aug. 2005.
[57] Z. Xu and Y. Hu, “Sbarc: A Supernode Based Peer-to-Peer File Sharing System,” Proc. Eighth IEEE Int'l Symp. Computers and Comm., June/July 2003.
[58] B. Yang and H. Garcia-Molina, “Designing a Super-Peer Network,” Proc. IEEE Int'l Conf. Data Eng., Mar. 2003.
[59] B.Y. Zhao, L. Huang, J. Stribling, S.C. Rhea, A.D. Joseph, and J.D. Kubiatowicz, “Tapestry: A Resilient Global-Scale Overlay for Service Deployment,” IEEE J. Selected Areas Comm., vol. 22, no. 1, pp. 41-53, Jan. 2004.
[60] Y. Zhu and Y. Hu, “Efficient, Proximity-Aware Load Balancing for dht-Based p2p Systems,” IEEE Trans. Parallel and Distributed Systems, vol. 16, no. 4, pp. 349-361, Apr. 2005.

Index Terms:
Peer to peer, superpeer architectures, semantic clustering, self-organizing systems.
Paweł Garbacki, Dick H.J. Epema, Maarten van Steen, "The Design and Evaluation of a Self-Organizing Superpeer Network," IEEE Transactions on Computers, vol. 59, no. 3, pp. 317-331, March 2010, doi:10.1109/TC.2009.157
Usage of this product signifies your acceptance of the Terms of Use.