The Community for Technology Leaders
Green Image
Issue No. 02 - April (2017 vol. 25)
ISSN: 1063-6692
pp: 662-675
Qiaomin Xie , Electrical and Computer Engineering Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Mayank Pundir , Computer Science Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Yi Lu , Electrical and Computer Engineering Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA
Cristina L. Abad , Computer Science Department, Escuela Superior Politécnica del Litoral, Guayaquil, Ecuador
Roy H. Campbell , Computer Science Department, University of Illinois at Urbana–Champaign, Urbana, IL, USA
ABSTRACT
Data locality is a fundamental problem to data-parallel applications where data-processing tasks consume different amounts of time and resources at different locations. The problem is especially prominent under stressed conditions such as hot spots. While replication based on data popularity relieves hot spots due to contention for a single file, hot spots caused by skewed node popularity, due to contention for files co-located with each other, are more complex, unpredictable, hence more difficult to deal with. We propose Pandas, a light-weight acceleration engine for data-processing tasks that is robust to changes in load and skewness in node popularity. Pandas is a stochastic delay-optimal algorithm. Trace-driven experiments on Hadoop show that Pandas accelerates the data-processing phase of jobs by 11 times with hot spots and 2.4 times without hot spots over existing schedulers. When the difference in processing times due to location is large, such as applicable to the case of memory-locality, the acceleration by Pandas is 22 times.
INDEX TERMS
Delays, Peer-to-peer computing, Hafnium, Robustness, Stochastic processes, Acceleration, Scheduling
CITATION

Q. Xie, M. Pundir, Y. Lu, C. L. Abad and R. H. Campbell, "Pandas: Robust Locality-Aware Scheduling With Stochastic Delay Optimality," in IEEE/ACM Transactions on Networking, vol. 25, no. 2, pp. 662-675, 2017.
doi:10.1109/TNET.2016.2606900
461 ms
(Ver 3.3 (11022016))