IEEE Transactions on Parallel and Distributed Systems

IEEE Transactions on Parallel and Distributed Systems (TPDS) is a scholarly archival journal published monthly. Parallelism and distributed computing are foundational research and technology to rapidly advance computer systems and their applications. Read the full scope of TPDS

IEEE Transactions on Parallel and Distributed Systems (TPDS) has moved to the OnlinePlus publication model.

From the April 2015 Issue

Exploiting Efficient and Scalable Shuffle Transfers in Future Data Center Networks

By Deke Guo, Junjie Xie, Xiaolei Zhou, Xiaomin Zhu, Wei Wei, and Xueshan Luo

Free Featured ArticleDistributed computing systems like MapReduce in data centers transfer massive amount of data across successive processing stages. Such shuffle transfers contribute most of the network traffic and make the network bandwidth become a bottleneck. In many commonly used workloads, data flows in such a transfer are highly correlated and aggregated at the receiver side. To lower down the network traffic and efficiently use the available network bandwidth, we propose to push the aggregation computation into the network and parallelize the shuffle and reduce phases. In this paper, we first examine the gain and feasibility of the in-network aggregation with BCube, a novel server-centric networking structure for future data centers. To exploit such a gain, we model the in-network aggregation problem that is NP-hard in BCube. We propose two approximate methods for building the efficient IRS-based incast aggregation tree and SRS-based shuffle aggregation subgraph, solely based on the labels of their members and the data center topology. We further design scalable forwarding schemes based on Bloom filters to implement in-network aggregation over massive concurrent shuffle transfers. Based on a prototype and large-scale simulations, we demonstrate that our approaches can significantly decrease the amount of network traffic and save the data center resources. Our approaches for BCube can be adapted to other server-centric network structures for future data centers after minimal modifications.

download PDF View the PDF of this article      csdl View this issue in the digital library

Editorials and Announcements


  • EICs Undergoing Reappointment for 2016-2017 Terms: IEEE Computer Society publications have editors in chief who are currently standing for reap­pointment to a second two-year term. The Publications Board invites comments upon the tenures of the individual editors. Please click here for more details.

  • According to Thomson Reuters' 2013 Journal Citation Report, TPDS has an impact factor of 2.173.

  • TPDS celebrates its 25th Anniversary. Editor-in-Chief David A. Bader says, "Congratulations to TPDS on its Silver Jubilee! For 25 years, TPDS has been the parallel and distributed computing community's flagship journal for research breakthroughs!"

  • Get Your Journals as eBooks for Free
  • Print on Demand is Now Available for OnlinePlus Titles


Guest Editorials

Reviewers List

Annual Index

Access recently published TPDS articles

RSS Subscribe to the RSS feed of latest TPDS content added to the digital library.

Mail Sign up for the Transactions Connection newsletter.

Listen to the OnlinePlus Podcast: Computer Society Publishing—two more titles migrate to OnlinePlus™ in 2012.

In this podcast, VP of Publications, David Alan Grier talks about Transactions on Mobile Computing and Transactions on Parallel and Distributed Systems migrating to OnlinePlus™.

TPDS is indexed in ISI