The Community for Technology Leaders
2014 Second International Conference on Advanced Cloud and Big Data (CBD) (2014)
Huangshan, China
Nov. 20, 2014 to Nov. 22, 2014
ISBN: 978-1-4799-8086-4
pp: 208-212
ABSTRACT
The current works about task scheduling with deadline-constraint in homogeneous environment rarely take the differences of Map and Reduce task and data locality into account in the same scheduler. To address this problem, we introduce a scheduling algorithm that Map and Reduce are regarded as two separated stages of scheduling problem in homogeneous environment. For the sake of realizing this algorithm, five aspects that are average execution time of map/reduce tasks, map/reduce stage deadline, remaining time of map/reduce stage, job's priority and data locality must be taken into consider. Compared with other real-time scheduling algorithm, we propose several methods which are one-to-one sampling, estimating requirements of resource and compromised task-data matching strategy to solve above five aspects. The experimental results show the sampling method can get accurate map/reduce task execution time and the proposed scheduling algorithm not only satisfies the job's real-time requirement but also improves the throughput of cluster.
INDEX TERMS
Scheduling algorithms, Scheduling, Real-time systems, Sampling methods, Schedules, Throughput
CITATION

Y. Yang, J. Xu, F. Wang, Z. Ma, J. Wang and L. Li, "A MapReduce Task Scheduling Algorithm for Deadline-Constraint in Homogeneous Environment," 2014 Second International Conference on Advanced Cloud and Big Data (CBD), Huangshan, China, 2014, pp. 208-212.
doi:10.1109/CBD.2014.35
83 ms
(Ver 3.3 (11022016))