Novel Scheduling Algorithms for Efficient Deployment of MapReduce Applications in Heterogeneous Computing Environments
Issue No. 04 - Oct.-Dec. (2018 vol. 6)
Sun-Yuan Hsieh , Tainan, Taiwan
Chi-Ting Chen , Tainan, Taiwan
Chi-Hao Chen , Tainan, Taiwan
Tzu-Hsiang Yen , Tainan, Taiwan
Hung-Chang Hsiao , Tainan, Taiwan
Rajkumar Buyya , Australia
Cloud computing has become increasingly popular model for delivering applications hosted in large data centers as subscription oriented services. Hadoop is a popular system supporting the MapReduce function, which plays a crucial role in cloud computing. The resources required for executing jobs in a large data center vary according to the job type. In Hadoop, jobs are scheduled by default on a first-come-first-served basis, which may unbalance resource utilization. This paper proposes a job scheduler called the
job allocation scheduler (JAS), designed to balance resource utilization. For various job workloads, the JAS categorizes jobs and then assigns tasks to a CPU-bound queue or an I/O-bound queue. However, the JAS exhibited a locality problem, which was addressed by developing a modified JAS called the job allocation scheduler with locality (JASL). The JASL improved the use of nodes and the performance of Hadoop in heterogeneous computing environments. Finally, two parameters were added to the JASL to detect inaccurate slot settings and create a dynamic job allocation scheduler with locality (DJASL). The DJASL exhibited superior performance than did the JAS, and data locality similar to that of the JASL.
Resource management, Cloud computing, Dynamic scheduling, Heart beat, Computational modeling, Distributed databases
S. Hsieh, C. Chen, C. Chen, T. Yen, H. Hsiao and R. Buyya, "Novel Scheduling Algorithms for Efficient Deployment of MapReduce Applications in Heterogeneous Computing Environments," in IEEE Transactions on Cloud Computing, vol. 6, no. 4, pp. 1080-1095, 2018.