The Community for Technology Leaders
2017 IEEE 42nd Conference on Local Computer Networks (LCN) (2017)
Singapore, Singapore
Oct. 9, 2017 to Oct. 12, 2017
ISSN: 0742-1303
ISBN: 978-1-5090-6523-3
pp: 104-111
ABSTRACT
Bulk data transfer is facing significant challenges in the coming era of big data. There are multiple performance bottlenecks along the end-to-end path from the source to destination storage system. The limitations of current generation data transfer tools themselves can have a significant impact on end-to-end data transfer rates. In this paper, we identify the issues that lead to underperformance of these tools, and present a new data transfer tool with an innovative I/O scheduler called MDTM. The MDTM scheduler exploits underlying multicore layouts to optimize throughput by reducing delay and contention for I/O reading and writing operations. With our evaluations, we show how MDTM successfully avoids NUMA-based congestion and significantly improves end-to-end data transfer rates across high-speed wide area networks.
INDEX TERMS
Big Data, multiprocessing systems, optimisation, parallel processing, processor scheduling, telecommunication congestion control, wide area networks
CITATION

L. Zhang, P. Demar, B. Kim and W. Wu, "MDTM: Optimizing Data Transfer Using Multicore-Aware I/O Scheduling," 2017 IEEE 42nd Conference on Local Computer Networks (LCN), Singapore, Singapore, 2018, pp. 104-111.
doi:10.1109/LCN.2017.64
90 ms
(Ver 3.3 (11022016))