This Article 
 Bibliographic References 
 Add to: 
Alleviating Consumption Channel Bottleneck in Wormhole-Routed k-ary n-Cube Systems
May 1998 (vol. 9 no. 5)
pp. 481-496

Abstract—This paper identifies performance degradation in wormhole routed k-ary n-cube networks due to limited number of router-to-processor consumption channels at each node. Many recent research in wormhole routing have advocated the advantages of adaptive routing and virtual channel flow control schemes to deliver better network performance. This paper indicates that the advantages associated with these schemes cannot be realized with limited consumption capacity. To alleviate such performance bottlenecks, a new network interface design using multiple consumption channels is proposed. To match virtual multiplexing on network channels, we also propose each consumption channel to support multiple virtual consumption channels. The impact of message arrival rate at a node on the required number of consumption channels is studied analytically. It is shown that wormhole networks with higher routing adaptivity, dimensionality, degree of hot-spot traffic, and number of virtual lanes have to take advantage of multiple consumption channels to deliver better performance. The interplay between system topology, routing algorithm, number of virtual lanes, messaging overheads, and communication traffic is studied through simulation to derive the effective number of consumption channels required in a system. Using the ongoing technological trend, it is shown that wormhole-routed systems can use up to two-four consumption channels per node to deliver better system performance.

[1] S. Abraham and K. Padmanabhan, "Performance of the Direct Binary n-Cube Network for Multiprocessors," IEEE Trans. Computers, vol. 38, no. 7, pp. 1000-1011, July 1989.
[2] A. Agarwal, "Limits on Interconnection Network Performance," IEEE Trans. Parallel and Distributed Systems, vol. 2, no. 4, pp. 398-412, Oct. 1991.
[3] S. Balakrishnan and D.K. Panda, “Impact of Multiple Consumption Channels on Wormhole Routed k-ary n-Cube Networks,” Proc. Int'l Parallel Processing Symp., pp. 163-167, 1993.
[4] D. Basak and D.K. Panda, "Designing Clustered Multiprocessor Systems Under Packaging and Technological Advancements," IEEE Trans. Parallel and Distributed Systems, vol. 7, no. 9, pp. 962-978, Sept. 1996.
[5] M.A. Blumrich et al., "Virtual Memory Mapped Network Interface for the SHRIMP Multicomputer," Proc. 21st Int'l Symp. Computer Architecture, Apr. 1994, pp. 142-153.
[6] R. Boppana and S. Chalasani, "A Comparison of Adaptive Wormhole Routing Algorithms," Proc. 20th Ann. Int'l Symp. Computer Architecture," pp. 351-360, 1993.
[7] Y.M. Boura and C.R. Das, “Efficient Fully Adaptive Wormhole Routing inn-Dimensional Meshes,” Proc. 14th Int'l Conf. Distributed Computing Systems, pp. 589-596, May 1994.
[8] S. Chittor and R. Enbody, "Performance Evaluation of Mesh-Connected Wormhole-Routed Networks for Interprocessor Communication in Multicomputers," Proc. Supercomputing '90, pp. 647-656,New York, Nov. 1990.
[9] Cray T3D System Architecture Overview. Cray Reasearch Inc., 1993.
[10] W.J. Dally and C.L. Seitz, “Deadlock-Free Message Routing in Multiprocessor Interconnection Networks,” IEEE Trans. Computers, Vol. C-36, No. 5, May 1987, pp. 547-553.
[11] W.J. Dally, "Performance Analysis of k-ary n-Cube Interconnection Networks," IEEE Trans. Computers, vol. 39, no. 6, pp. 775-785, June 1992.
[12] W.J. Dally, "Virtual-Channel Flow Control," IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 194-205, Mar. 1992.
[13] S.P. Dandamudi and D.L. Eager, "Hot Spot Contention in Binary Hypercube Networks," IEEE Trans. Computers, Vol. C-41, No. 2, Feb. 1992, pp. 239-244.
[14] J. Duato, "A New Theory of Deadlock-Free Adaptive Routing in Wormhole Networks," IEEE Trans. Parallel and Distributed Systems, vol. 4, no. 12, pp. 1,320-1,331, Dec. 1993.
[15] V. Karamcheti and A.A. Chien, "Software Overhead in Messaging Layers: Where Does the Time Go?" Proc. Sixth Symp. Architectural Support for Programming Languages and Operating Systems (ASPLOS-VI), ACM Press, New York, 1994, pp. 51-60;
[16] M. Heinrich et al. “The Stanford FLASH Multiprocessor,” Proc. 21th Int'l Symp. Computer Architecture, pp. 302-313, April 1994.
[17] D.H. Linder and J.C. Harden, "An Adaptive and Fault Tolerant Wormhole Routing Strategy for k-Ary n-Cubes," IEEE Trans. Computers, vol. 40, no. 1, pp. 2-12, Jan. 1991.
[18] L.M. Ni and P.K. McKinley, "A Survey of Wormhole Routing Techniques in Direct Networks," Computer, vol. 26, no. 2, pp. 62-76, Feb. 1993.
[19] D.K. Panda, D. Basak, D. Dai, R. Kesavan, R. Sivaram, M. Banikazemi, and V. Moorthy, “Simulation of Modern Parallel Systems: A CSIM-Based Approach,” Proc. 1997 Winter Simulation Conf. (WSC '97), pp. 1013-1020, Dec. 1997.
[20] G.F. Pfister and V.A. Norton, "Hot-Spot Contention and Combining in Multistage Interconnection Networks," IEEE Trans. Computers, vol. 34, pp. 943-948, 1985.
[21] S. Ramany and D. Eager, "The Interaction Between Virtual Channel Flow Control and Adaptive Routing in Wormhole Networks," Proc. Int'l Conf. Supercomputing, pp. 136-145, July 1994.
[22] M. Snir et al., "The Communication Software and Parallel Environment of the IBM SP2," IBM Systems J., Vol. 34, No. 2, 1995, pp. 205-221.
[23] C. Stunkel, D. Shea, B. Abali, M. Atkins, C. Bender, D. Grice, P. Hochshild, D. Joseph, B. Nathanson, R. Swetz, R. Stucke, M. Tsao, and P. Varker, “The SP2 High-Performance Switch,” IBM Systems J., vol. 34, no. 2,pp. 185–204, 1995.
[24] C.B. Stunkel et al., “The SP1 High-Performance Switch,” Proc. Scalable High-Performance Computing Conf., CS Press, May 1994, pp. 150-157.
[25] R. Suaya and G. Birtwistle, VLSI and Parallel Computation, chapter 3.2. Morgan Kauffman, 1990.

Index Terms:
Parallel computer architecture, wormhole routing, k-ary n-cube, consumption channel, virtual channel, deterministic routing, adaptive routing, hot-spot traffic, and interprocessor communication.
Debashis Basak, Dhabaleswar K. Panda, "Alleviating Consumption Channel Bottleneck in Wormhole-Routed k-ary n-Cube Systems," IEEE Transactions on Parallel and Distributed Systems, vol. 9, no. 5, pp. 481-496, May 1998, doi:10.1109/71.679218
Usage of this product signifies your acceptance of the Terms of Use.