This Article 
 Bibliographic References 
 Add to: 
Barrier Synchronization on Wormhole-Routed Networks
June 2001 (vol. 12 no. 6)
pp. 583-597

Abstract—In this paper, we propose an efficient barrier synchronization scheme on networks with arbitrary topologies. We first present a distributed method in building a barrier routing tree. The barrier messages can be delivered adaptively according to the hierarchy of the established barrier tree to void congestion and faulty nodes in the network. We then propose a new technique, called bandwidth-preempting technique, for a blocked barrier message to preempt a channel occupied by a data message so that the latency of a barrier message can be controlled without affecting much of the overall system performance. We also propose an analytical performance model and present simulation results for the performance evaluation of the proposed scheme. Performance evaluations show that the proposed scheme outperforms the existing algorithms for barrier synchronization.

[1] J. M. Mellor-Crummey and M. L. Scott,“Algorithms for scalable synchronization on shared-memory multiprocessors,”ACM Trans. Comput. Syst., vol, 9, no. 1, pp. 21–65, Feb. 1991.
[2] M. Kandemir, N. Shenoy, P. Banerjee, J. Ramanujam, and A. Choudhary, “Minimizing Data and Synchronization Costs in One-Way Communication,” Proc. Int'l Conf. Parallel Processing, 1998.
[3] J.-S. Kim, S. Ha, and C.S. Jhon, “Efficient Barrier Synchronization Mechanism for the BSP Model on Message-Passing Architectures,” Proc. 12th IEEE Int'l Parallel Processing Symp., pp. 255-259, 1998.
[4] R. Sivaram, C.B. Stunkel, and D.K. Panda, "A Reliable Hardware Barrier Synchronization Scheme," Proc. 11th IEEE Int'l Parallel Processing Symp., pp. 274-280, Apr. 1997.
[5] J.-S. Yang and C.-T. King, “Designing Tree-Based Barrier Synchronization on 2D Mesh Networks,” IEEE Trans. Parallel and Distributed Systems, vol. 9, no. 6, pp. 526-534, June 1998.
[6] H. Xu, P.K. McKinley, and L.M. Ni, “Efficient Implementation of Barrier Synchronization in Wormhole-Routed Hypercube Multicomputer,” J. Parallel and Distributed Computing, vol. 16, pp. 172-184, Oct. 1992.
[7] K.B. Fan and C.T. King, "Turn Grouping for Efficient Barrier Synchronization in Wormhole Mesh Networks," Proc. 25th Int'l Conf. Parallel Processing, Aug. 1997.
[8] D.K. Panda, “Fast Barrier Synchronization Wormhole$k \hbox {-} {\rm ary} \;n \hbox {-} {\rm cube}$Networks with Multi-Destination Worms,” Proc. Int'l Symp. High Performance Computer Architecture, pp. 200-209, 1995.
[9] R.E. Kessler and J.L. Schwarzmeier, "CRAY T3D: A New Dimension for Cray Research," Proc. COMPCON, pp. 176-182, Feb. 1993.
[10] S.L. Scott, "Synchronization and Communication in the T3E Multiprocess," Proc. ASPLOS-VII, Oct. 1996.
[11] R. Kesavan, K. Bondalapati, and D.K. Panda, “Multicast on Irregular Switch-Based Networks with Wormhole Routing,” Proc. Int'l Symp. High Performance Computer Architecture (HPCA-3), pp. 48-57, Feb. 1997.
[12] J. Protic, M. Tomasevic, and V. Milutinovic, Distributed Shared Memory: Concepts and Systems, IEEE Computer Soc. Press, Los Alamitos, Calif., 1998.
[13] M.D. Schroeder, A.D. Birrell, M. Burrows, H. Murray, R.M. Needham, T.L. Rodeheffe, E.H. Satterthwaite, and C.P. Thacker, "Autonet: A High-Speed, Self-Configuring Local Area Network Using Point-to-Point Links," IEEE J. Selected Areas in Comm., vol. 9, no. 8, pp. 1,318-1,335, Oct. 1991.
[14] X. Lin, P.K. McKinley,, and L.M. Ni,"Deadlock-Free Multicast Wormhole Routing in 2-D Mesh Multicomputers," IEEE Trans. Parallel and Distributed Systems, vol. 5, no. 8, Aug. 1994, pp. 793-804.
[15] J.H. Kim, Z. Liu, and A.A. Chien., "Compressionless Routing: A Framework for Fault-Tolerant Routing," IEEE Trans. Parallel and Distributed Systems, vol. 8, no. 3, pp. 229-244, Mar. 1997.
[16] W.J. Dally, "Virtual-Channel Flow Control," IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 194-205, Mar. 1992.
[17] J. Duato, “A Necessary and Sufficient Condition for Deadlock-Free Adaptive Routing in Wormhole Networks,” IEEE Trans. Parallel and Distributed Systems, vol. 6, no. 10, pp. 1,055–1,067, Oct. 1995.
[18] Y.-J. Tsai, Y. Huang, and P.K. McKinley, “Performance Evaluation of Barrier Synchronization in ATM Network,” Proc. Int'l Conf. Computer Comm. and Networks, Oct. 1996.
[19] F. Silla, M.P. Malumbres, J. Duato, D. Dai, and D.K. Panda, “Impact of Adaptive on the Behavior of Networks of Workstations Under Bursty Traffic,” Proc. Int'l Conf. Parallel Processing, 1998.
[20] Y. Sun, P.Y.S. Cheung, and X. Lin, “Bandwidth-Preempting Barrier Synchronization on Wormhole-Routed 2D Networks,” Technical Report HKU-EEE-99-002, 1999.

Index Terms:
Barrier synchronization, network topology, tree-based routing, data message, barrier message, bandwidth-preempting technique.
Yuzhong Sun, Paul Y.S. Cheung, Xiaola Lin, "Barrier Synchronization on Wormhole-Routed Networks," IEEE Transactions on Parallel and Distributed Systems, vol. 12, no. 6, pp. 583-597, June 2001, doi:10.1109/71.932712
Usage of this product signifies your acceptance of the Terms of Use.