This Article 
 Bibliographic References 
 Add to: 
Boosting the Performance of Myrinet Networks
July 2002 (vol. 13 no. 7)
pp. 693-709

Editor's Note: This paper unfortunately contains some errors which led to the paper being reprinted in the November 2002 issue. Please see IEEE Transactions on Parallel and Distributed Systems, vol. 13, no. 11, November 2002, pp. 1166-1182 for the correct paper (available without subscription).

Networks of workstations (NOWs) are becoming increasingly popular as a cost-effective alternative to parallel computers. These networks allow the customer to connect processors using irregular topologies, providing the wiring flexibility, scalability, and incremental expansion capability required in this environment. Some of these networks use source routing and wormhole switching. In particular, we are interested in Myrinet networks because it is a well-known commercial product and its behavior can be controlled by the software running in network interfaces (Myrinet Control Program, MCP). Usually, the Myrinet network uses up*/down* routing for computing the paths for every source-destination pair. In this paper, we propose the In-Transit Buffer (ITB) mechanism to improve network performance. We apply the ITB mechanism to NOWs with up*/down* source routing, like Myrinet, analyzing its behavior on both networks with regular and irregular topologies. The proposed scheme can be implemented on Myrinet networks by only modifying the MCP, without changing the network hardware. We evaluate by simulation several networks with different traffic patterns using timing parameters taken from the Myrinet network. Results show that the current routing schemes used in Myrinet networks can be strongly improved by applying the ITB mechanism. In general, our proposed scheme is able to double the network throughput on medium and large NOWs. Finally, we present a first implementation of the ITB mechanism on a Myrinet network.

[1] N. Boden et al., "Myrinet: A Gigabit-per-Second Local Area Network," IEEE Micro, Feb. 1995, pp. 29-36.
[2] R.V. Bopana and S. Chalasani, A Comparison of Adaptive Wormhole Routing Algorithms Proc. 20th Ann. Int'l Symp. Computer Architecture, May 1993.
[3] H. Chen and P. Wyckoff, “Performance Characterization of a Terabit Switch and Myrinet as Cluster Interconnects,” Proc. HOT Interconnects, Aug. 2000.
[4] L. Cherkasova, V. Kotov, and T. Rokicki, “Fibre Channel Fabrics: Evaluation and Design,” Proc. 29th Hawaii Int'l Conf. System Sciences, Feb. 1995.
[5] W.J. Dally, “Express Cubes: Improving the Performance ofk-Aryn-Cube Interconnection Networks,” IEEE Trans. Computers, vol. 40, pp. 1016-1023, 1991.
[6] W.J. Dally, "Virtual-Channel Flow Control," IEEE Trans. Parallel and Distributed Systems, vol. 3, no. 2, pp. 194-205, Mar. 1992.
[7] J. Duato, S. Yalamanchili, and L.M. Ni, Interconnection Networks: An Engineering Approach. Los Alamitos, Calif.: IEEE CS Press, 1997.
[8] GM Protocol,http://www.myri.comGM. 2001.
[9] R. Horst, “ServerNet Deadlock Avoidance and Fractahedral Topologies,” Proc. Int'l Parallel Processing Symp., pp. 274–280, Apr. 1996.
[10] L. Widigen, E. Sowadsky, and K. McGrath, "Eliminating Operand Read Latency," Computer Architecture News, Dec. 1996, pp. 18-22.
[11] P.R. Miller, “Efficient Communications for Fine-Grain Distributed Computers,” PhD thesis, Southampton Univ., 1991.
[12] Myrinet web page,http:/ 2001.
[13] Myrinet, M2-CB-35 LAN cables, . 2001.
[14] S.S. Owicki and A.R. Karlin, “Factors in the Performance of the AN1 Computer Network,” Digital SRC Research Report 88, June 1992.
[15] R. Riesen et al., “CPLANT,” Proc. Second Extreme Linux Workshop, June 1999.
[16] J.C. Sancho, A. Robles, and J. Duato, New Methodology to Compute Deadlock-Free Routing Tables for Irregular Networks Proc. Workshop Comm. and Architectural Support for Network-Based Parallel Computing, Jan. 2000.
[17] M.D. Schroeder et al., “Autonet: A High-Speed, Self-Configuring Local Area Network Using Point-to-Point Links,” Technical Report SRC research report 59, DEC, Apr. 1990.
[18] S. L. Scott, J.R. Goodman, The Impact of Pipelined Channels on K-Ary N-Cube Networks IEEE Trans. Parallel and Distributed Systems, vol. 5, no. 1, pp. 2-16, Jan. 1994.
[19] F. Silla, M.P. Malumbres, A. Robles, P. López, and J. Duato, Efficient Adaptive Routing in Networks of Workstations with Irregular Topology Proc. Workshop Comm. and Architectural Support for Network-Based Parallel Computing, Feb. 1997.
[20] F. Silla and J. Duato, “Improving the Efficiency of Adaptive Routing in Networks with Irregular Topology,” Proc. 1997 Int'l Conf. High Performance Computing, Dec. 1997.

Index Terms:
Networks of workstations, wormhole switching, minimal routing, source routing, performance evaluation.
José Flich, Pedro López, M.P. Malumbres, Jose Duato, "Boosting the Performance of Myrinet Networks," IEEE Transactions on Parallel and Distributed Systems, vol. 13, no. 7, pp. 693-709, July 2002, doi:10.1109/TPDS.2002.1019859
Usage of this product signifies your acceptance of the Terms of Use.