Issue No.11 - November (2009 vol.58)
pp: 1444-1455
Ymir Vigfusson , Cornell University, New York
Ken Birman , Cornell University, New York
Anne-Marie Kermarrec , INRIA, Rennes Bretagne Atlantique
Robbert van Renesse , Cornell University, New York
Peer-to-peer (P2P) architectures are popular for tasks such as collaborative download, VoIP telephony, and backup. To maximize performance in the face of widely variable storage capacities and bandwidths, such systems typically need to shift work from poor nodes to richer ones. Similar requirements are seen in today's large data centers, where machines may have widely variable configurations, loads, and performance. In this paper, we consider the slicing problem, which involves partitioning the participating nodes into k subsets using a one-dimensional attribute, and updating the partition as the set of nodes and their associated attributes change. The mechanism thus facilitates the development of adaptive systems. We begin by motivating this problem statement and reviewing prior work. Existing algorithms are shown to have problems with convergence, manifesting as inaccurate slice assignments, and to adapt slowly as conditions change. Our protocol, Sliver, has provably rapid convergence, is robust under stress and is simple to implement. We present both theoretical and experimental evaluations of the protocol.
Distributed systems, fault tolerance, performance evaluation of algorithms and systems.
Ymir Vigfusson, Ken Birman, Anne-Marie Kermarrec, Robbert van Renesse, "Slicing Distributed Systems", IEEE Transactions on Computers, vol.58, no. 11, pp. 1444-1455, November 2009, doi:10.1109/TC.2009.111
[1] M. Ajtai, J. Komlós, and E. Szemerédi, “Sorting in c Log n Parallel Steps,” J. Combinatorica, vol. 3, no. 1, pp. 1-19, Mar. 1983.
[2] D. Angluin and L.G. Valiant, “Fast Probabilistic Algorithms for Hamiltonian Circuits and Matchings,” J. Computer and System Sciences, vol. 18, no. 2, pp. 155-193, 1979.
[3] F.M. auf der Heide and A. Wigderson, “The Complexity of Parallel Sorting,” SIAM J. Computing, vol. 16, no. 1, pp. 100-107, 1987.
[4] K.E. Batcher, “Sorting Networks and Their Applications,” Proc. AFIPS '68 Sprint Joint Computer Conf., vol. 32, pp. 307-314, 1968.
[5] V. Chvátal, “Lecture Notes on the New AKS Sorting Network,” technical report, Rutgers Univ., 1992.
[6] B. Cohen, “Abstract Incentives Build Robustness in Bittorrent,” Proc. Workshop Economics of Peer-to-Peer Systems, 2003.
[7] R. Cole, “Parallel Merge Sort,” SIAM J. Computing, vol. 17, no. 4, pp. 770-785, 1988.
[8] A.J. Demers, D.H. Greene, C. Hauser, W. Irish, J. Larson, S. Shenker, H.E. Sturgis, D.C. Swinehart, and D.B. Terry, “Epidemic Algorithms for Replicated Database Maintenance,” Proc. Sixth Ann. ACM Symp. Principles of Distributed Computing (PODC '87), pp. 1-12, 1987.
[9] J.R. Douceur and W.J. Bolosky, “A Large-Scale Study of File-System Contents,” Proc. 1999 ACM SIGMETRICS, pp. 59-70, 1999.
[10] A. Fernández, V. Gramoli, E. Jiménez, A.-M. Kermarrec, and M. Raynal, “Distributed Slicing in Dynamic Systems,” Proc. 27th IEEE Int'l Conf. Distributed Computing Systems (ICDCS '07), 2007.
[11] Gnutella Homepage, http:/, 2007.
[12] M.T. Goodrich, “Communication-Efficient Parallel Sorting,” SIAM J. Computing, vol. 29, no. 2, pp. 416-432, 1999.
[13] V. Gramoli, E. Le Merrer, and A.-M. Kermarrec, GossiPeer, http:/, 2009.
[14] S. Guha, N. Daswani, and R. Jain, “An Experimental Study of the Skype Peer-to-Peer VoIP System,” Proc. Fifth Int'l Workshop Peer-to-Peer Systems, 2006.
[15] M. Jelasity and A.-M. Kermarrec, “Ordered Slicing of Very Large-Scale Overlay Networks,” Proc. Sixth IEEE Int'l Conf. Peer-to-Peer Computing (P2P '06), pp. 117-124, 2006.
[16] M. Jelasity, S. Voulgaris, R. Guerraoui, A.-M. Kermarrec, and M. van Steen, “Gossip-Based Peer Sampling,” ACM Trans. Computer Systems, vol. 25, no. 3, p. 8, 2007.
[17] KaZaA Homepage, http:/, 2009.
[18] F.T. Leighton, Introduction to Parallel Algorithms and Architectures: Array, Trees, Hypercubes. Morgan Kaufmann Publishers, Inc., 1992.
[19] L. Lovász, “Random Walks on Graphs: A Survey,” Combinatorics, Paul Erdös Is Eighty, vol. 2, pp. 353-398, 1996.
[20] M. Ripeanu, “Peer-to-Peer Architecture Case Study: Gnutella Network,” Proc. First Int'l Conf. Peer-to-Peer Computing (P2P '01), pp. 99-100, 2001.
[21] S. Saroiu, K.P. Gummadi, and S.D. Gribble, “A Measurement Study of Peer-to-Peer File Sharing Systems,” Proc. Multimedia Computing and Networking, pp. 156-170, 2002.
[22] N. Shavit, E. Upfal, and A. Zemach, “A Wait-Free Sorting Algorithm,” Proc. 16th Ann. ACM Symp. Principles of Distributed Computing (PODC '97), pp. 121-128, 1997.
[23] Skype Homepage, http:/, 2009.
[24] D. Stutzbach and R. Rejaie, “Understanding Churn in Peer-to-Peer Networks,” Proc. Internet Measurement Conf., pp. 189-202, 2006.
[25] B. White, J. Lepreau, L. Stoller, R. Ricci, S. Guruprasad, M. Newbold, M. Hibler, C. Barb, and A. Joglekar, “An Integrated Experimental Environment for Distributed Systems and Networks,” Proc. Fifth Symp. Operating System Design and Implementation (OSDI '02), pp. 255-270, 2002.