This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Flexible and Efficient Routing Based on Progressive Deadlock Recovery
July 1999 (vol. 48 no. 7)
pp. 649-669

Abstract—The development of fully adaptive, cut-through (wormhole) networks is important for achieving high performance in communication-critical parallel processor systems. Increased flexibility in routing allows network bandwidth to be used efficiently, but also creates more opportunity for cyclic resource dependencies to form which can cause deadlock. If not guarded against, deadlocks in routing make packets block in the network indefinitely and, eventually, could result in the entire network coming to a complete standstill. This paper presents a simple, flexible, and efficient routing approach for multicomputer interconnection networks which is based on progressive deadlock recovery as opposed to deadlock avoidance or regressive deadlock recovery. Performance is optimized by allowing the maximum routing freedom provided by network resources to be exploited. True fully adaptive routing is supported in which all physical and virtual channels at each node in the network are available to packets without regard for deadlocks. Deadlock cycles, upon forming, are efficiently broken in finite time by progressively routing one of the blocked packets through a connected, deadlock-free recovery path. This new routing approach enables the design of high-throughput networks that provide excellent performance. Simulations indicate that progressive deadlock recovery routing can improve throughput by as much as 45 percent and 25 percent over leading deadlock avoidance-based and regressive recovery-based routing schemes, respectively.

[1] L.M. Ni and P.K. McKinley, "A Survey of Wormhole Routing Techniques in Direct Networks," Computer, vol. 26, no. 2, pp. 62-76, Feb. 1993.
[2] W.J. Dally, "Virtual-Channel Flow Control," IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 194-205, Mar. 1992.
[3] J. Duato, S. Yalamanchili, and L.M. Ni, Interconnection Networks: An Engineering Approach. Los Alamitos, Calif.: IEEE CS Press, 1997.
[4] T.M. Pinkston and S. Warnakulasuriya, Characterization of Deadlocks in K-Ary N-Cube Networks IEEE Trans. Parallel and Distributed Systems, vol. 10, no. 9, pp 38-49, Sept. 1999.
[5] T.M. Pinkston and S. Warnakulasuriya, “On Deadlocks in Interconnection Networks,” Proc. 24th Int'l Symp. Computer Architecture, June 1997.
[6] W.J. Dally and C.L. Seitz, “Deadlock-Free Message Routing in Multiprocessor Interconnection Networks,” IEEE Trans. Computers, Vol. C-36, No. 5, May 1987, pp. 547-553.
[7] C.J. Glass and L.M. Ni, "The Turn Model for Adaptive Routing," Proc. 19th Int'l Symp. Computer Architecture, vol. 20, no. 2, pp. 278-287, May 1992.
[8] R. Boppana and S. Chalasani, "A Comparison of Adaptive Wormhole Routing Algorithms," Proc. 20th Ann. Int'l Symp. Computer Architecture," pp. 351-360, 1993.
[9] A.A. Chien and J.H. Kim, "Planar-Adaptive Routing: Low-Cost Adaptive Networks for Multiprocessors," Proc. 19th Int'l Symp. Computer Architecture, vol. 20, no. 2, pp. 268-277, May 1992.
[10] D.H. Linder and J.C. Harden, "An Adaptive and Fault Tolerant Wormhole Routing Strategy for k-Ary n-Cubes," IEEE Trans. Computers, vol. 40, no. 1, pp. 2-12, Jan. 1991.
[11] W.J. Dally and H. Aoki, "Deadlock-Free Adaptive Routing in Multicomputer Networks Using Virtual Channels," IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 4, pp. 466-475, Apr. 1993.
[12] A.A. Chien, “A Cost and Performance Model for$k$-ary$n$-cubes Wormhole Routers,” IEEE Trans. Parallel and Distributed Systems, vol. 9, no. 2, pp. 150-162, Feb. 1998.
[13] J. Duato, "A New Theory of Deadlock-Free Adaptive Routing in Wormhole Networks," IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 12, pp. 1,320-1,331, Dec. 1993.
[14] J. Duato, “A Necessary and Sufficient Condition for Deadlock-Free Adaptive Routing in Wormhole Networks,” IEEE Trans. Parallel and Distributed Systems, vol. 6, no. 10, pp. 1,055–1,067, Oct. 1995.
[15] S. Konstantinidou and L. Snyder, "Chaos Router: Architecture and Performance," Proc. 18th Ann. Int'l Symp. Computer Architecture, 1991.
[16] D.S. Reeves, E.F. Gehringer, and A. Chandiramani, “Adaptive Routing and Deadlock Recovery: A Simulation Study,” Proc. Fourth Conf. Hypercube Concurrent Computers and Applications, Mar. 1989.
[17] J.H. Kim, Z. Liu, and A.A. Chien., "Compressionless Routing: A Framework for Fault-Tolerant Routing," IEEE Trans. Parallel and Distributed Systems, vol. 8, no. 3, pp. 229-244, Mar. 1997.
[18] K.V. Anjan and T.M. Pinkston, “An Efficient, Fully Adaptive Deadlock Recovery Scheme: Disha,” Proc. 22nd Ann. Int'l Symp. Computer Architecture, pp. 201-210, June 1995.
[19] K.V. Anjan, T.M. Pinkston, and J. Duato, Generalized Theory for Deadlock-Free Adaptive Routing and Its Application to Disha Concurrent Proc. 10th Int'l Parallel Processing Symp., Apr. 1996.
[20] F. Petrini and M. Vanneschi, “Performance Analysis of Minimal Adaptive Wormhole Routing with Time-Dependent Deadlock Recovery,” Proc. 11th Int'l Parallel Processing Symp., pp. 589-595, Apr. 1997.
[21] J.M. Martínez, P. López, J. Duato, and T.M. Pinkston, “Software-Based Deadlock Recovery Technique for True Fully Adaptive Routing in Wormhole Networks,” Proc. Int'l Conf. Parallel Processing, pp. 182-189, Aug. 1997.
[22] P. López, J.M. Martínez, and J. Duato, A Very Efficient Distributed Deadlock Detection Mechanism for Wormhole Networks Proc. High Performance Computer Architecture Symp., Feb. 1998.
[23] J.M. Martinez, P. Lopez, and J. Duato, “Impact of Buffer Size on the Efficiency of Deadlock Detection,” Proc. High Performance Computer Architecture Symp., pp. 315-318, Jan. 1999.
[24] A.K. Venkatramani, “Disha: A True Fully Adaptive Routing Scheme,” Master's thesis, Univ. of Southern California, May 1995.
[25] T.M. Pinkston, Y. Choi, and M. Raksapatcharawong, “Architecture and Optoelectronic Implementation of the WARRP Router,” Proc. Fifth Symp. Hot Interconnects, pp. 181-189, Aug. 1997.
[26] M. Coli and P. Palazzari, “An Adaptive Deadlock and Livelock Free Routing Algorithm,” Proc. Third Euromicro Workshop Parallel and Distributed Processing, pp. 288-295 Jan. 1995.
[27] P. Palazzari and M. Coli, “Virtual Cut-Through Implementation of the Hole-Based Packet Switching Routing Algorithm,” Proc. Sixth Euromicro Workshop Parallel and Distributed Processing, pp. 416-421, Jan. 1998.
[28] W. Qiao and L.M. Ni, “Adaptive Routing in Irregular Networks Using Cut-Through Switches,” Proc. 1996 Int'l Conf. Parallel Processing, Aug. 1996.
[29] M.D. Schroeder et al., “Autonet: A High-Speed, Self-Configuring Local Area Network Using Point-to-Point Links,” Technical Report SRC Research Report 59(DEC), Apr. 1990.
[30] K.V. Anjan and T.M. Pinkston, "DISHA: An Efficient Fully Adaptive Deadlock Recovery Scheme," Proc. Ninth Int'l Parallel Processing Symp., Apr. 1995.
[31] F. Silla, A. Robles, and J. Duato, Improving Performance of Networks of Workstations by Using Disha Concurrent Proc. Int'l Conf. Parallel Processing, Aug. 1998.
[32] C. Stunkel, D. Shea, B. Abali, M. Atkins, C. Bender, D. Grice, P. Hochshild, D. Joseph, B. Nathanson, R. Swetz, R. Stucke, M. Tsao, and P. Varker, “The SP2 High-Performance Switch,” IBM Systems J., vol. 34, no. 2,pp. 185–204, 1995.
[33] T.M. Pinkston, M. Raksapatcharawong, and C. Kuznia, “An Asynchronous Optical Token Smart-Pixel Design Based on Hybrid CMOS/SEED Integration,” LEOS 1996 Summer Topical Meeting on Smart Pixels Technical Digest, pp. 40-41, Aug. 1996.
[34] T.M. Pinkston, M. Raksapatcharawong, and Y. Choi, “The WARRP Core: Optoelectronic Implementation of Network Network Router Deadlock Handling Mechanisms,” Applied Optics, vol. 37, no. 2, pp. 276-283, Jan. 1998.
[35] T.M. Pinkston, M. Raksapatcharawong, and Y. Choi, “WARRP II: An Optoelectronic Fully Adaptive Network Router Chip,” Topical Meeting on Optics in Computing, pp. 311-315, June 1998.
[36] S.L. Scott and G.M. Thorson, “The Cray T3E Network: Adaptive Routing in a High Performance 3D Torus,” Proc. Symp. Hot Interconnects IV, pp. 147-156, Aug. 1996.
[37] W.J. Dally, L.R. Dennison, D. Harris, K. Kan, and T. Xanthopoulos, “Architecture and implementation of the Reliable Router,” Proc. Hot Interconnects II, Aug. 1994.
[38] M. Galles, “Spider: A High Speed Network Interconnect,” Proc. Symp. Hot Interconnects IV, pp. 141-146, Aug. 1996.
[39] J.D. Allen, P.T. Gaughan, D.E. Schimmel, and S. Yalamanchili, "Ariadne—An Adaptive Router for Fault-Tolerant Multicomputers," Proc. 21st Int'l Symp. Computer Architecture, pp. 278-288, Apr. 1994.
[40] J. Duato, S. Yalamanchili, and L.M. Ni, Interconnection Networks: An Engineering Approach. Los Alamitos, Calif.: IEEE CS Press, 1997.
[41] P. López and J. Duato,“Deadlock-free adaptive routing algorithms for the 3D-torus:Limitations and solutions,” Proc. Parallel Architectures Languages Europe 93, June 1993.
[42] P. López, J.M. Martínez, and J. Duato, DRIL: Dynamically Reduced Message Injection Limitation Mechanism for Wormhole Networks Proc. Int'l Conf. Parallel Processing, pp. 535-542, Aug. 1998.
[43] S. Borkar, R. Cohn, G. Cox, T. Gross, H.T. Kung, M. Lam, M. Levine, B. Moore, W. Moore, C. Peterson, J. Susman, J. Sutton, J. Urbanski, and J. Webb, "Supporting Systolic and Memory Communication in iWarp," Proc. 17th Int'l Symp. Computer Architecture, pp. 70-81, 1990.

Index Terms:
Fully adaptive routing, deadlock detection, high-performance interconnection networks, progressive deadlock recovery, wormhole switching.
Citation:
Timothy Mark Pinkston, "Flexible and Efficient Routing Based on Progressive Deadlock Recovery," IEEE Transactions on Computers, vol. 48, no. 7, pp. 649-669, July 1999, doi:10.1109/12.780873
Usage of this product signifies your acceptance of the Terms of Use.