2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA) (2017)
March 27, 2017 to March 29, 2017
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/AINA.2017.134
MapReduce model was designed for distributed large volume of data processing. Time constraints are very important for user productivity but, in a shared cluster, the need for job starvation avoidance also arises. In this paper we propose an extension to the well-known FairScheduler algorithm from Hadoop which takes into consideration soft deadlines for jobs in homogeneous clusters, aiming to improve productivity by better satisfying the user time needs. Our model is as follows: when a job is launched, a deadline is provided. When the job starts running, it has a default priority assigned by which FairScheduler splits resources. The job gets allocated resources and at a certain moment in time it has a current execution speed. Based on speed, it is computed how many more resources the job needs to finish in time. Given this, the job priority is dynamically adjusted so the FairScheduler's resource division policy can meet the deadlines. We validated our algorithm by execution in a real Hadoop environment and we obtained increased performance under deadline constraints while also avoiding job starvation.
Clustering algorithms, Algorithm design and analysis, Processor scheduling, Job shop scheduling, Delays, Dynamic scheduling, Heuristic algorithms
A. Farcasanu, F. Pop, M. Nita and C. Dobre, "Starvation Avoidance with Deadline Constraints in Hadoop Environments," 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), Taipei, Taiwan, 2017, pp. 1090-1097.