Issue No. 04 - July/August (2009 vol. 29)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MM.2009.67
Keren Bergman , Columbia University
Ron Brightwell , Sandia National Laboratories
Fabrizio Petrini , IBM TJ Watson Research Laboratory
We are pleased to introduce this special issue of IEEE Micro, featuring articles representing the leading-edge activities in high-performance interconnection networks presented at the 2008 IEEE Symposium on High-Performance Interconnects at Stanford University. The Hot Interconnects symposium is an international forum for academic and industry researchers to describe the latest cutting-edge research or unveil new improvements in product development in areas related to high-performance interconnection networks.
The importance and influence of high-performance interconnection networks continues to grow. In the recent past, research into high-performance networking technologies impacted wide-area internetworking, data centers, and supercomputers. The move of chip multiprocessing systems to the mainstream has increased the need for advancements in on-chip interconnects, fueling the development of technologies in areas such as photonics and requiring new approaches in which power and density are critical constraints. High-performance networking is becoming a fundamental technology in more arenas, including financial services and cloud computing. In turn, this expanded scope creates new challenges requiring innovations in algorithms, software, and hardware for high-performance interconnection networks.
This special issue features six articles that touch on various topics in high-performance interconnects, including silicon photonics, crossbar-scheduling algorithms, memory-coherency hardware design and implementation, and optical networks. These articles span the spectrum from fundamental algorithmic advances to the deployment of new commercially available products.
In "Building Many-core Processor-to-DRAM Networks with Monolithic CMOS Silicon Photonics," Christopher Batten et al. present a new monolithic silicon photonics technology that implements photonic components in a standard CMOS process. Simulation and experimental results show a significant increase in energy efficiency and bandwidth density for this new approach when compared to approaches using all-electrical links. The authors explore how this approach can be leveraged to construct a processor-to-DRAM network for a manycore processor. They propose a hierarchical network topology and use simulated synthetic traffic patterns to illustrate their scheme's advantages. The article emphasizes the need for a vertically integrated approach to exploit photonic devices' system-level benefits.
In the second article, "Practical High-Throughput Crossbar Scheduling," Nikos Chrysos and Giorgos Dimitrakopoulos propose a new deterministic crossbar scheduling algorithm that has a simple structure and can be implemented efficiently in hardware. This new algorithm combines ideas from randomized backlog-aware and round-robin schedulers to distribute the bandwidth of congested links fairly and achieve nearly optimal throughput for nonuniform traffic patterns. Chrysos and Dimitrakopoulos completed a hardware implementation of this algorithm for a 32 × 32 crossbar using a 130-nm process. The switch can support line rates above 100 gigabits per second (Gbps) for 40-byte IP packets, demonstrating its viability as a large, high-speed switch.
In the next article, "Coherency Hub Design for Multisocket Sun Servers with CoolThreads Technology," John Feehrer et al. describe the microarchitecture of a memory coherency hub designed for a four-socket system using Sun's UltraSPARC T2 Plus 8-core processor. The article gives detailed information on the coherency scheme and describes several techniques used to enable near-linear performance and scaling on transaction-processing workloads for a full system. In addition, the article discusses details of the implementation, including the engineering challenges involved in meeting commercial product's speed and processing rate specifications.
In "Express Virtual Channels with Capacitively Driven Global Links," Tushar Krishna et al. explore a visionary approach to dramatically increase the performance of on-chip communication. Network on chip with hybrid interconnect (Nochi) is a hybrid interconnect that leverages multiple types of fabric—conventional wires for short-range datapaths and low-swing, multidrop wires for long-range, low-latency communication. The article describes how Nochi can both reduce power consumption and substantially improve latency under heavy loads compared to a design that used only the conventional interconnect.
Michael Tan et al. explore the design of an optical data bus in "A High-Speed Optical Multidrop Bus for Computer Interconnections." They propose an optical data bus with two sets of optical waveguides, one used as a fan-out and the other as a fan-in, to interconnect modules on the shared bus. With 1 mW of laser power, the authors connect eight different modules at 10 Gbps per channel, with an aggregate bandwidth of more than 25 gigabytes per second (GBps) with 10-bit wide signaling paths.
Finally, in "Photonic NoCs: System-Level Design Exploration," Michele Petracca et al. explore the design of photonic on-chip networks for delivering a scalable solution to future multicore processors with bandwidth-per-watt performance that can surpass that of equivalent electronic NoCs. Photonic NoCs offer two distinct advantages—namely, that the achievable communication bandwidth on a single waveguide (or link) can approach multiple terabits per second (Tbps) with limited power dissipation, and that the power dissipation to first order is independent of the distance. The authors discuss and evaluate several proposed architectures that specifically exploit silicon photonics for on-chip and off-chip communication.
We hope that you enjoy reading these articles and that they stimulate thinking on the challenges and opportunities in the research and development of technologies surrounding high-performance interconnection networks. We welcome your feedback on this special issue.
It has been a pleasure to compile this special IEEE Micro issue on high-performance interconnection networks. We would like to thank Editor-in-Chief David Albonesi, Associate Editor-in-Chief Ruby Lee, and the IEEE Micro staff for their assistance and patience. We would also like to express gratitude to John Kim of Northwestern University, Jeff Kash of IBM, Cyriel Minkenberg of IBM, and Karu Sankaralingam of the University of Wisconsin for their help with the review process. This special issue would not have been possible without these people's help, support, and dedication.
Keren Bergman is a professor of electrical engineering at Columbia University, where she also directs the Lightwave Research Laboratory. She leads multiple research programs on optical interconnection networks for advanced computing systems, terascale optical routers, wavelength-striped optical packet-switched networks, and chip multiprocessor nanophotonic networks on chip. Bergman has a PhD in electrical engineering from the Massachusetts Institute of Technology. She is a Fellow of the IEEE and the Optical Society of America, and currently serves as the co-editor in chief of the recently launched IEEE/OSA Journal of Optical Communications and Networking.
Ron Brightwell is manager of the Scalable System Software Department at Sandia National Laboratories in Albuquerque, New Mexico. His research interests include high-performance scalable communication interfaces and protocols for system-area networks and lightweight operating systems for massively parallel computing systems. Brightwell has an MS in computer science from Mississippi State University. He is a senior member of the IEEE and the ACM.
Fabrizio Petrini is a senior researcher of the Multicore Computing Department of the IBM TJ Watson Research Laboratory. His research interests include various aspects of high-performance interconnects, multicore processors and supercomputers, parallel architectures, operating systems, and parallel programming languages. Petrini has a PhD in computer science from the University of Pisa. He is associate editor of IEEE Transactions on Parallel and Distributed Processing.