The Community for Technology Leaders
2015 IEEE International Conference on Autonomic Computing (ICAC) (2015)
Grenoble, France
July 7, 2015 to July 10, 2015
ISBN: 978-1-4673-6970-1
pp: 149-150
There is a trade-off between the number of concurrently running MapReduce jobs and their corresponding map and reduce tasks within a node in a Hadoop cluster. Leaving this trade-off statically configured to a single value can significantly reduce job response times leaving only sub optimal resource usage. To overcome this problem, we propose a feedback control loop based approach that dynamically adjusts the Hadoop resource manager configuration based on the current state of the cluster. The preliminary assessment based on workloads synthesized from real-world traces shows that the system performance can be improved by about 30% compared to default Hadoop setup.
Time factors, Random access memory, Yarn, Feedback control, Analytical models, Containers, Memory management

B. Zhang, F. Krikava, R. Rouvoy and L. Seinturier, "Self-Configuration of the Number of Concurrently Running MapReduce Jobs in a Hadoop Cluster," 2015 IEEE International Conference on Autonomic Computing (ICAC), Grenoble, France, 2015, pp. 149-150.
80 ms
(Ver 3.3 (11022016))