2015 IEEE First International Conference on Big Data Computing Service and Applications (BigDataService) (2015)
Redwood City, CA, USA
March 30, 2015 to April 2, 2015
The configuration of a Hadoop cluster is significantly important to its performance, because an improper configuration can greatly deteriorate the job execution performance. Unfortunately, systematic guidelines on how to configure a Hadoop cluster are still missing. In this paper, we undertake an empirical study on key operations and mechanisms of Hadoop job execution, including the task assignment strategy and speculative execution. Based on the experiments, we provide suggestions on the system configuration, particularly on the matching between the hardware resource partitioning scheme and the job splitting granularity.
Delays, Hardware, Indexes, Cloud computing, Electronic publishing, Time-domain analysis, Resource management
Y. Jiang, Z. Huang and D. H. Tsang, "Do You Feel the Lag of Your Hadoop?," 2015 IEEE First International Conference on Big Data Computing Service and Applications (BigDataService)(BIGDATASERVICE), Redwood City, CA, USA, 2015, pp. 115-119.