This Article 
 Bibliographic References 
 Add to: 
A Cost-Effective Approach to Deadlock Handling in Wormhole Networks
July 2001 (vol. 12 no. 7)
pp. 716-729

Abstract—Wormhole networks have traditionally used deadlock avoidance strategies. More recently, deadlock recovery strategies have begun to gain acceptance. In particular, progressive deadlock recovery techniques allocate a few dedicated resources to quickly deliver deadlocked packets. Deadlock recovery is based on the assumption that deadlocks are rare; otherwise, recovery techniques are not efficient. Measurements of deadlock occurrence frequency show that deadlocks are highly unlikely when enough routing freedom is provided [36], [32]. However, networks are more prone to deadlocks when the network is close to or beyond saturation, causing some network performance degradation. Similar performance degradation behavior at saturation was also observed in networks using deadlock avoidance strategies [13]. In this paper, we take a different approach to handling deadlocks and performance degradation. We propose the use of an injection limitation mechanism that prevents performance degradation near the saturation point and, at the same time, reduces the probability of deadlock to negligible values. We also propose an improved deadlock detection mechanism that uses only local information, detects all deadlocks, and considerably reduces the probability of false deadlock detection over previous proposals. In the rare case when impending deadlock is detected, our proposal consists of using a simple recovery technique that absorbs the deadlocked message at the current node and later reinjects it for continued routing toward its destination. Performance evaluation results show that our new approach to handling deadlock is more efficient than previously proposed techniques.

[1] A. Agarwal, "Limits on Interconnection Network Performance," IEEE Trans. Parallel and Distributed Systems, vol. 2, no. 4, pp. 398-412, Oct. 1991.
[2] K.V. Anjan and T.M. Pinkston, "DISHA: An Efficient Fully Adaptive Deadlock Recovery Scheme," Proc. Ninth Int'l Parallel Processing Symp., Apr. 1995.
[3] K.V. Anjan and T.M. Pinkston, “An Efficient, Fully Adaptive Deadlock Recovery Scheme: Disha,” Proc. 22nd Ann. Int'l Symp. Computer Architecture, pp. 201-210, June 1995.
[4] K.V. Anjan, T.M. Pinkston, and J. Duato, Generalized Theory for Deadlock-Free Adaptive Routing and Its Application to Disha Concurrent Proc. 10th Int'l Parallel Processing Symp., Apr. 1996.
[5] W.C. Athas and C.L. Seitz, “Multicomputers: Message-Passing Concurrent Computers,” Computer, vol. 21, pp. 9-24, Aug. 1988.
[6] E. Baydal, P. López, and J. Duato, A Simple and Efficient Mechanism to Prevent Saturation in Wormhole Networks Proc. Int'l Parallel and Distributed Processing Symp., May 2000.
[7] N. Boden et al., "Myrinet: A Gigabit-per-Second Local Area Network," IEEE Micro, Feb. 1995, pp. 29-36.
[8] R. Boppana and S. Chalasani, "A Comparison of Adaptive Wormhole Routing Algorithms," Proc. 20th Ann. Int'l Symp. Computer Architecture," pp. 351-360, 1993.
[9] A.A. Chien and J.H. Kim, "Planar-Adaptive Routing: Low-Cost Adaptive Networks for Multiprocessors," Proc. 19th Int'l Symp. Computer Architecture, vol. 20, no. 2, pp. 268-277, May 1992.
[10] A.A. Chien, “A Cost and Speed Model fork-Aryn-Cube Wormhole Routers,” Proc. Hot Interconnects '93, Aug. 1993.
[11] W.J. Dally and C.L. Seitz, “Deadlock-Free Message Routing in Multiprocessor Interconnection Networks,” IEEE Trans. Computers, Vol. C-36, No. 5, May 1987, pp. 547-553.
[12] W.J. Dally, "Virtual-Channel Flow Control," IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 194-205, Mar. 1992.
[13] J. Duato, "A New Theory of Deadlock-Free Adaptive Routing in Wormhole Networks," IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 12, pp. 1,320-1,331, Dec. 1993.
[14] J. Duato and P. López, "Performance Evaluation of Adaptive Routing Algorithms for k-ary n-cubes," Proc. Workshop Parallel Computer Routing and Communication, May. 1994.
[15] J. Duato, “Improving the Efficiency of Virtual Channels with Time-Dependent Selection Functions,” Future Generation Computer Systems, no. 10, pp. 45-58, 1994.
[16] J. Duato, “A Necessary and Sufficient Condition for Deadlock-Free Adaptive Routing in Wormhole Networks,” IEEE Trans. Parallel and Distributed Systems, vol. 6, no. 10, pp. 1,055–1,067, Oct. 1995.
[17] J. Duato, S. Yalamanchili, and L.M. Ni, Interconnection Networks: An Engineering Approach. Los Alamitos, Calif.: IEEE CS Press, 1997.
[18] P.T. Gaughan and S. Yalamanchili, “Adaptive Routing Protocols for Hypercube Interconnection Networks,” Computer, vol. 26, no. 5, pp. 12–23, May 1993.
[19] R. Horst, “ServerNet Deadlock Avoidance and Fractahedral Topologies,” Proc. Int'l Parallel Processing Symp., pp. 274–280, Apr. 1996.
[20] Paragon XP/S Product Overview. Beaverton, OR: Intel Corp., Supercomputer Systems Division, 1991.
[21] L. Widigen, E. Sowadsky, and K. McGrath, "Eliminating Operand Read Latency," Computer Architecture News, Dec. 1996, pp. 18-22.
[22] J.H. Kim, Z. Liu, and A.A. Chien, "Compressionless Routing: A Framework for Adaptive and Fault Tolerant Routing," Proc. 21st Ann. Int'l Symp. Computer Architecture, pp. 289-300, Apr. 1994.
[23] F.T. Leighton,Introduction to Parallel Algorithms and Architectures: Arrays, Trees, Hypercubes.San Mateo, Calif.: Morgan Kaufmann, 1992.
[24] D. Lenoski et al., “The Stanford DASH Multiprocessor,” Computer, pp. 63-79, Mar. 1992.
[25] P. López and J. Duato,“Deadlock-free adaptive routing algorithms for the 3D-torus:Limitations and solutions,” Proc. Parallel Architectures Languages Europe 93, June 1993.
[26] P. López, J.M. Martínez, J. Duato, and F. Petrini, “On the Reduction of Deadlock Frequency by Limiting Message Injection in Wormhole Networks,” Proc. Parallel Computer Routing and Comm. Workshop, June 1997.
[27] P. López, J.M. Martínez, and J. Duato, A Very Efficient Distributed Deadlock Detection Mechanism for Wormhole Networks Proc. High Performance Computer Architecture Symp., Feb. 1998.
[28] P. López, J.M. Martínez, and J. Duato, DRIL: Dynamically Reduced Message Injection Limitation Mechanism for Wormhole Networks Proc. Int'l Conf. Parallel Processing, pp. 535-542, Aug. 1998.
[29] P.K. McKinley, H. Xu, A. Esfahanian, and L.M. Ni, “Unicast-Based Multicast Communication in Wormhole-Routed Networks,” Proc. 1992 Int'l Conf. Parallel Processing, Aug. 1992.
[30] P.R. Miller, “Efficient Communications for Fine-Grain Distributed Computers,” PhD thesis, Southampton Univ., 1991.
[31] Myrinet Product List, org/sc96/proceedings/. ncwn/pvmsystem.ps myrinetproduct_list.html, 2001.
[32] T.M. Pinkston and S. Warnakulasuriya, “On Deadlocks in Interconnection Networks,” Proc. 24th Int'l Symp. Computer Architecture, June 1997.
[33] D.S. Reeves, E.F. Gehringer, and A. Chandiramani, “Adaptive Routing and Deadlock Recovery: A Simulation Study,” Proc. Fourth Conf. Hypercube, Concurrent Computers and Applications, Mar. 1989.
[34] S.L. Scott and G. Thorson, “The Cray T3E Networks: Adaptive Routing in a High Performance 3-D Torus,” Proc. Hot Interconnects IV, Aug. 1996.
[35] Y. Suh, B.V. Dao, J. Duato, and S. Yalamanchili, “Software Based Fault-Tolerant Oblivious Routing in Pipelined Networks,” Proc. 1995 Int'l Conf. Parallel Processing, Aug. 1995.
[36] S. Warnakulasuriya and T.M. Pinkston, “Characterization of Deadlocks in Interconnection Networks,” Proc. 11th Int'l Parallel Processing Symp., Apr. 1997.

Index Terms:
Wormhole switching, adaptive routing, deadlock recovery, deadlock detection, injection limitation.
Juan M. Martínez-Rubio, Pedro López, Jose Duato, "A Cost-Effective Approach to Deadlock Handling in Wormhole Networks," IEEE Transactions on Parallel and Distributed Systems, vol. 12, no. 7, pp. 716-729, July 2001, doi:10.1109/71.940746
Usage of this product signifies your acceptance of the Terms of Use.