This Article 
 Bibliographic References 
 Add to: 
Coupling-Based Internal Clock Synchronization for Large-Scale Dynamic Distributed Systems
May 2010 (vol. 21 no. 5)
pp. 607-619
Roberto Baldoni, Sapienza University of Rome, Rome
Angelo Corsaro, PrismTech, Marcoussis
Leonardo Querzoni, Sapienza University of Rome, Rome
Sirio Scipioni, Sapienza University of Rome, Rome
Sara Tucci Piergiovanni, Sapienza University of Rome, Rome
This paper studies the problem of realizing a common software clock among a large set of nodes without an external time reference (i.e., internal clock synchronization), any centralized control, and where nodes can join and leave the distributed system at their will. The paper proposes an internal clock synchronization algorithm which combines the gossip-based paradigm with a nature-inspired approach, coming from the coupled oscillators phenomenon, to cope with scale and churn. The algorithm works on the top of an overlay network and uses a uniform peer sampling service to fulfill each node's local view. Therefore, differently from clock synchronization protocols for small scale and static distributed systems, here, each node synchronizes regularly with only the neighbors in its local view and not with the whole system. An evaluation of the convergence speed and the synchronization error of the coupled-based internal clock synchronization algorithm has been carried out, showing how convergence time and the synchronization error depends on the coupling factor and the local view size. Moreover, the variation of the synchronization error with respect to churn and the impact of a sudden variation of the number of nodes have been analyzed to show the stability of the algorithm. In all these contexts, the algorithm shows nice performance and very good self-organizing properties. Finally, we showed how the assumption on the existence of a uniform peer-sampling service is instrumental for the good behavior of the algorithm and how, in system models where network delays are unbounded, a mean-based convergence function reaches a lower synchronization error than median-based convergence functions exploiting the number of averaged clock values.

[1] K. Arvind, "Probabilistic Clock Synchronization in Distributed Systems," IEEE Trans. Parallel and Distributed Systems, vol. 5, no. 5, pp. 474-487, May 1994.
[2] A. Awan, R.A. Ferreira, S. Jagannathan, and A. Grama, "Distributed Uniform Sampling in Unstructured Peer-to-Peer Networks," Proc. 39th Ann. Hawaii Int'l Conf. System Sciences, pp. 223-233, 2006.
[3] R. Baldoni, A. Corsaro, L. Querzoni, S. Scipioni, and S.T. Piergiovanni, "An Adaptive Coupling-Based Algorithm for Internal Clock Synchronization of Large Scale Dynamic Systems," Proc. OnTheMove Federated Conferences & Workshops (OTM), pp. 701-716, 2007.
[4] R. Baldoni, C. Marchetti, and A. Virgillito, "Impact of WAN Channel Behavior on End-to-End Latency of Replication Protocols," Proc. European Dependable Computing Conf., 2006.
[5] R. Baldoni, R. Jimenez-Peris, M. Patino-Martinez, L. Querzoni, and A. Virgillito, "Dynamic Quorums for DHT-Based Enterprise Infrastructures," J. Parallel and Distributed Computing, vol. 68, no. 9, pp. 1235-1249, 2008.
[6] F. Cristian, "A Probabilistic Approach to Distributed Clock Synchronization," Distributed Computing, vol. 3, pp. 146-158, 1989.
[7] F. Cristian and C. Fetzer, "Integrating Internal and External Clock Synchronization," J. Real Time Systems, vol. 12, no. 2, pp. 123-171, 1997.
[8] F. Cristian, H. Aghili, and R. Strong, "Clock Synchronization in the Presence of Omission and Performance Faults, and Processor Joins," Proc. 16th Int'l Symp. Fault-Tolerant Computing Systems, pp. 218-223, 1986.
[9] F. Cristian and C. Fetzer, "Lower Bounds for Convergence Function Based Clock Synchronization," Proc. 14th Ann. ACM Symp. Principles of Distributed Computing, pp. 137-143, 1995.
[10] A. Daliot, D. Dolev, and H. Parnas, "Linear Time Byzantine Self-Stabilizing Clock Synchronization," Technical Report TR2003-89, Schools of Eng. and Computer Science, The Hebrew Univ. of Jerusalem, Dec. 2003.
[11] A. Daliot, D. Dolev, and H. Parnas, "Self-Stabilizing Pulse Synchronization Inspired by Biological Pacemaker Networks," Proc. Sixth Symp. Self-Stabilizing Systems, pp. 32-48, 2003.
[12] S. Dolev, "Possible and Impossible Self-Stabilizing Digital Clock Synchronization in General Graph," J. Real-Time Systems, vol. 12, no. 1, pp. 95-107, 1997.
[13] P. Eugster, S. Handurukande, R. Guerraoui, A. Kermarrec, and P. Kouznetsov, "Lightweight Probabilistic Broadcast," ACM Trans. Computer Systems, vol. 21, no. 4, pp. 341-374, 2003.
[14] R. Friedman, A. Kermarrek, H. Miranda, and L. Rodriguez, "Gossip-Based Dissemination," Middleware for Eccentric and Mobile Applications, Springer, 2009.
[15] J. Halpern, B. Simons, and R. Strong, "Fault-Tolerant Clock Synchronization," Proc. Third Ann. ACM Symp. Principles of Distributed Computing, pp. 89-102, 1984.
[16] T. Herman and S. Ghosh, "Stabilizing Phase-Clock," Information Processing Letters, vol. 5, no. 6, pp. 585-598, 1994.
[17] C. Hewitt, "ORGs for Scalable, Robust, Privacy-Friendly Client Cloud Computing," IEEE Internet Computing, vol. 12, no. 5, pp. 96-99, Sept./Oct. 2008.
[18] K. Ho, J. Wu, and J. Sum, "On the Session Lifetime Distribution of Gnutella," Int'l J. Parallel, Emergent, and Distributed Systems, vol. 23, no. 1, pp. 1-15, 2008.
[19] K. Iwanicki, M. van Steen, and S. Voulgaris, "Gossip-Based Synchronization for Large Scale Decentralized Systems," Proc. Second IEEE Int'l Workshop Self-Managed Networks, Systems and Services, pp. 28-42, 2006.
[20] M. Jelasity, R. Guerraoui, A.-M. Kermarrec, and M. van Steen, "The Peer Sampling Service: Experimental Evaluation of Unstructured Gossip-Based Implementations," Proc. Fifth ACM/IFIP/USENIX Int'l Conf. Middleware, pp. 79-98, 2004.
[21] M. Jelasity, A. Montresor, and O. Babaoglu, "Gossip-Based Aggregation in Large Dynamic Networks," ACM Trans. Computer Systems, vol. 23, no. 3, pp. 219-252, 2005.
[22] V. King, S. Lewis, J. Saia, and M. Young, "Choosing a Random Peer in Chord," Algorithmica, vol. 49, no. 2, pp. 147-169, 2007.
[23] Y. Kuramoto, Chemical Oscillations, Waves and Turbulence, chapter 5. Springer-Verlag, 1984.
[24] L. Lamport, "Time, Clocks and Ordering of Events in a Distributed System," Comm. ACM, vol. 21, no. 7, pp. 558-565, 1978.
[25] L. Lamport and P.M. Melliar-Smith, "Byzantine Clock Synchronization," Proc. Third Ann. ACM Symp. Principles of Distributed Computing, pp. 68-74, 1984.
[26] L. Lamport and P.M. Melliar-Smith, "Synchronizing Clocks in the Presence of Faults," J. ACM, vol. 32, no. 1, pp. 52-78, 1985.
[27] J. Lundelius-Welch and N. Lynch, "A New Fault-Tolerant Algorithm for Clock Synchronization," Proc. Third Ann. ACM Symp. Principles of Distributed Computing, pp. 75-88, 1984.
[28] L. Massouli, E. Le Merrer, A.-M. Kermarrec, and A. Ganesh., "Peer Counting and Sampling in Overlay Networks: Random Walk Methods," Proc. 25th Ann. ACM Symp. Principles of Distributed Computing, pp. 123-132, 2006.
[29] P.C. Matthews, R.E. Mirollo, and S.H. Strogatz, "Dynamics of a Large System of Coupled Nonlinear Oscillators," Physica D, vol. 52, nos. 2/3, pp. 293-331, 1991.
[30] D.L. Mills, "Network Time Protocol (Version 1) Specification and Implementation," Network Working Group Report RFC-1059, Univ. of Delaware, 1988.
[31] D.L. Mills, "Network Time Protocol Version 4 Reference and Implementation Guide," Technical Report 06-06-1, Dept. of Electrical and Computer Eng., Univ. of Delaware, 2006.
[32] Object Management Group, Data Distribution Service for Real-Time Systems Specification v1.2, ptc/2006-04-09, 2006.
[33] A. Rowstron and P. Druschel, "Pastry: Scalable, Decentralized Object Location and Routing for Large-Scale Peer-to-Peer Systems," Proc. IFIP/ACM Int'l Conf. Distributed Systems Platforms (Middleware), pp. 329-350, 2001.
[34] K. Satoh, "Computer Experiment on the Cooperative Behavior of a Network of Interacting Nonlinear Oscillators," J. Physical Soc. of Japan, vol. 58, no. 6, pp. 2010-2021, 1989.
[35] S. Scipioni, L. Querzoni, S.T. Piergiovanni, and R. Baldoni, "A Theoretical Evaluation of Peer-to-Peer Internal Clock Synchronization," Proc. Autonomics Conf., 2008.
[36] I. Stoica, R. Morris, D. Liben-Nowell, D.R. Karger, M.F. Kaashoek, F. Dabek, and H. Balakrishnan, "Chord: A Scalable Peer-to-Peer Lookup Protocol," IEEE/ACM Trans. Networking, vol. 11, no. 1, pp. 17-32, Feb. 2003.
[37] S.H. Strogatz and R.E. Mirollo, "Phase-Locking and Critical Phenomena in Lattices of Coupled Nonlinear Oscillators with Random Intrinsic Frequencies," Physica D, vol. 31, pp. 143-168, 1988.
[38] C. Tang, R.N. Chang, and E. So, "A Distributed Service Management Infrastructure for Enterprise Data Centers Based on Peer-to-Peer Technology," Proc. IEEE Int'l Conf. Services Computing, pp. 52-59, 2006.
[39] C. Tang, M. Steinder, M. Spreitzer, and G. Pacifici, "A Scalable Application Placement Controller for Enterprise Data Centers," Proc. 16th Int'l Conf. World Wide Web, pp. 331-340, 2007.
[40] P. Verissimo, L. Rodrigues, and A. Casimiro, "CesiumSpray: A Precise and Accurate Global Time Service for Large-Scale Systems," J. Real-Time Systems, vol. 12, no. 3, pp. 243-294, 1997.
[41] S. Voulgaris, D. Gavidia, and M. van Steen, "CYCLON: Inexpensive Membership Management for Unstructured P2P Overlays," J. Network and System Management, vol. 13, no. 2, pp. 197-217, 2005.
[42] A.T. Winfree, "Biological Rhythms and the Behaviour of Populations of Coupled Oscillators," J. Theoretical Biology, vol. 28, pp. 327-374, 1967.

Index Terms:
Peer-to-peer, internal clock synchronization, peer sampling, overlay networks.
Roberto Baldoni, Angelo Corsaro, Leonardo Querzoni, Sirio Scipioni, Sara Tucci Piergiovanni, "Coupling-Based Internal Clock Synchronization for Large-Scale Dynamic Distributed Systems," IEEE Transactions on Parallel and Distributed Systems, vol. 21, no. 5, pp. 607-619, May 2010, doi:10.1109/TPDS.2009.111
Usage of this product signifies your acceptance of the Terms of Use.