This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Decentralized Convergence Detection Algorithm for Asynchronous Parallel Iterative Algorithms
January 2005 (vol. 16 no. 1)
pp. 4-13

Abstract—We introduce a theoretical algorithm and its practical version to perform a decentralized detection of the global convergence of parallel asynchronous iterative algorithms. We prove that, even if the algorithm is completely decentralized, the detection of global convergence is achieved on one processor under the classical conditions. The proposed algorithm is very useful in the context of grid computing in which the processors are distributed and in which detecting the convergence on a master processor may be penalizing or even impossible as in Peer to Peer computation frameworks. Finally, the efficiency of the practical algorithm is illustrated in a typical experiment.

[1] Y. Saad, Iterative Methods for Sparse Linear Systems, second ed. SIAM, 2003.
[2] J.M. Bahi, S. Contassot-Vivier, and R. Couturier, “Asynchronism for Iterative Algorithms in a Global Computing Environment,” Proc. 16th Ann. Int'l Symp. High Performance Computing Systems and Applications (HPCS 2002), pp. 90-97, June 2002.
[3] N. Francez, “Distributed Termination,” ACM Trans. Programming Languages and Systems, vol. 2, no. 1, pp. 42-55, Jan. 1980.
[4] E.W. Dijkstra, W.H.J. Feijen, and A.J.M. vanGasteren, “Derivation of a Termination Detection Algorithm for Distributed Computation,” Information Processing Letters, vol. 16, no. 5, pp. 217-219, 1983.
[5] S.P. Rana, “A Distributed Solution to the Distributed Termination Problem,” Information Processing Letters, vol. 17, pp. 43-46, July 1983.
[6] N. Maillard, E.M. Daoudi, P. Manneback, and J.-L. Roch, “Contrôle Amorti des Synchronisations pour le Test d'ArrÁt des Méthodes Itératives,” Renpar 14, pp. 177-182, Apr. 2002.
[7] D.P. Bertsekas and J.N. Tsitsiklis, “Convergence Rate and Termination of Asynchronous Iterative Algorithms,” Proc. 1989 Int'l Conf. Supercomputing, pp. 461-470, June 1989.
[8] S.A. Savari and D.P. Bertsekas, “Finite Termination of Asynchronous Iterative Algorithms,” Parallel Computing, vol. 22, pp. 39-56, 1996.
[9] D. El Baz, Contribution à l'Algorithmique Parallèle, le Concept d'Asynchronisme: Étude Théorique, Mise en Oeuvre, et Application. LAAS CNRS, Institut National Polytechnique de Toulouse, 1998.
[10] A.S. Charão, “Multiprogrammation Parallèle Générique des Méthodes de Décomposition de Domaine,” PhD dissertation, Institut National Polytechnique de Grenoble, http://www-med iatheque.imag.fr/Mediatheque.IMAG/ collections-electroniques/publications index.html, 2001.
[11] “IEEE Standard for a High Performance Serial Bus,” Technical Report Std 1394-1995, IEEE Computer Society, Aug. 1996.
[12] G. Antonoiu and P.K. Srimani, “A Self-Stabilizing Leader Election Algorithm for Tree Graphs,” J. Parallel and Distributed Computing, vol. 34, no. 2, pp. 227-232, May 1996.
[13] M. El-Ruby, J. Kenevan, R. Carison, and K. Khalil, “Leader Election in Distributed Computing Systems,” Proc. Conf. Computing in the 90's, pp. 350-356, Oct. 1991.
[14] H. Garcia-Molina, “Elections in a Distributed Computing System,” IEEE Trans. Computers, vol. 31, no. 1, pp. 47-59, Jan. 1982.
[15] S.D. Stoller, “Leader Election in Asynchronous Distributed Systems,” IEEE Trans. Computers, vol. 49, no. 3, pp. 283-284, , Mar. 2000.
[16] D.P. Bertsekas and J.N. Tsitsiklis, Parallel and Distributed Computation: Numerical Methods. Englewood Cliffs, N.J.: Prentice Hall, 1989.
[17] J.M. Bahi, S. Contassot-Vivier, and R. Couturier, “Coupling Dynamic Load Balancing with Asynchronism in Iterative Algorithms on the Computational Grid,” Proc. 17th Int'l Parallel and Distributed Processing Symp. (IPDPS 2003), p. 40, Apr. 2003.
[18] M. El Tarazi, “Some Convergence Results for Asynchronous Algorithms,” Numerical Math., vol. 39, pp. 325-340, 1982.
[19] D. El Baz, P. Spiteri, J.C. Miellou, and D. Gazen, “Asynchronous Iterative Algorithms with Flexible Communication for Nonlinear Network Flow Problems,” J. Parallel and Distributed Computing, vol. 38, no. 1, pp. 1-15, Oct. 1996.
[20] J.M. Bahi, “Asynchronous Iterative Algorithms for Nonexpansive Linear Systems,” J. Parallel and Distributed Computing, vol. 60, no. 1, pp. 92-112, Jan. 2000.
[21] N. Lynch, Distributed Algorithms. San Francisco, Calif.: Morgan Kaufmann, citeseer.ist.psu.edu/5897.htmlhttp://theory.lcs.mit.edu/ tdsdistalgs.html, 1996.
[22] J.M. Bahi, J.-C. Miellou, and K. Rhofir, “Asynchronous Multisplitting Methods for Nonlinear Fixed Point Problems,” Numerical Algorithms, vol. 15, nos. 3 and 4, pp. 315-345, 1997.
[23] A. Pope, The CORBA Reference Guide: Understanding the Common Object Request Broker Architecture. Reading, Mass.: Addison-Wesley, Dec. 1997.

Index Terms:
Parallel iterative algorithms, asynchronism, convergence detection.
Citation:
Jacques M. Bahi, Sylvain Contassot-Vivier, Raphaël Couturier, Flavien Vernier, "A Decentralized Convergence Detection Algorithm for Asynchronous Parallel Iterative Algorithms," IEEE Transactions on Parallel and Distributed Systems, vol. 16, no. 1, pp. 4-13, Jan. 2005, doi:10.1109/TPDS.2005.2
Usage of this product signifies your acceptance of the Terms of Use.