IEEE Transactions on Cloud Computing

Expand your horizons with Colloquium, a monthly survey of abstracts from all CS transactions! Replaces OnlinePlus in January 2017.

From the January-March 2017 issue

Orchestrating Bulk Data Transfers across Geo-Distributed Datacenters

By Yu Wu, Zhizhong Zhang, Chuan Wu, Chuanxiong Guo, Zongpeng Li, and Francis C. M. Lau

Featured article thumbnail imageAs it has become the norm for cloud providers to host multiple datacenters around the globe, significant demands exist for inter-datacenter data transfers in large volumes, e.g., migration of big data. A challenge arises on how to schedule the bulk data transfers at different urgency levels, in order to fully utilize the available inter-datacenter bandwidth. The Software Defined Networking (SDN) paradigm has emerged recently which decouples the control plane from the data paths, enabling potential global optimization of data routing in a network. This paper aims to design a dynamic, highly efficient bulk data transfer service in a geo-distributed datacenter system, and engineer its design and solution algorithms closely within an SDN architecture. We model data transfer demands as delay tolerant migration requests with different finishing deadlines. Thanks to the flexibility provided by SDN, we enable dynamic, optimal routing of distinct chunks within each bulk data transfer (instead of treating each transfer as an infinite flow), which can be temporarily stored at intermediate datacenters to mitigate bandwidth contention with more urgent transfers. An optimal chunk routing optimization model is formulated to solve for the best chunk transfer schedules over time. To derive the optimal schedules in an online fashion, three algorithms are discussed, namely a bandwidth-reserving algorithm, a dynamically-adjusting algorithm, and a future-demand-friendly algorithm, targeting at different levels of optimality and scalability. We build an SDN system based on the Beacon platform and OpenFlow APIs, and carefully engineer our bulk data transfer algorithms in the system. Extensive real-world experiments are carried out to compare the three algorithms as well as those from the existing literature, in terms of routing optimality, computational delay and overhead.

download PDF View article (PDF)     csdl View issue (CSDL)      TCC Facebook Link  TCC on Facebook      TCC LinkedIn Group  TCC on Linkedin

Editorials and Announcements



Guest Editorials

Reviewers List

Author Index

Call for Papers

General Call for Papers

General call for papers. View PDF.

Access recently published TCC articles

RSS Subscribe to the RSS feed of latest TCC content added to the digital library.

Mail Sign up for the Transactions Connection newsletter.

Access TCC Preprints in the Computer Society digital library

TCC is financially cosponsored by:

IEEE Computer SocietyIEEE Communications SocietyIEEE Power & Energy SocietyIEEE Consumer Electronics SocietyIEEE Systems Council

TCC is technically cosponsored by:

IEEE Signal Processing Society