2017 IEEE International Conference on Cluster Computing (CLUSTER) (2017)
Honolulu, Hawaii, United States
Sept. 5, 2017 to Sept. 8, 2017
Due to the growing size of compute clusters, large scale parallel applications increasingly have to deal with hardware malfunctions and other failure scenarios during execution. The overall goal of this research is to get good performance of MapReduce applications despite failures. The paper focuses on evaluation of the performance of two representative Hadoop MapReduce applications, 'WordCount' and 'Stack Exchange', with different execution parameters and under different failure scenarios. The paper also presents different options to inject failures into MapReduce applications to simulate real world failures. Some of the preliminary observations are that slowdown due to failure is higher with relatively larger input split sizes, slowdown peaks near the optimal split size, and that the performance and slowdown are sensitive to key MapReduce execution parameters.
Hardware, Software, Upper bound, XML, Standards, Big Data
M. T. Rahman, E. Gabriel and J. Subhlok, "Performance Implications of Failures on MapReduce Applications," 2017 IEEE International Conference on Cluster Computing (CLUSTER), Honolulu, Hawaii, United States, 2017, pp. 741-748.