The Community for Technology Leaders
Green Image
Issue No. 01 - Jan.-March (2016 vol. 4)
ISSN: 2168-7161
pp: 49-62
Weiwei Chen , Information Sciences Institute, University of Southern California, Marina del Rey, Los Angeles, CA
Rafael Ferreira da Silva , Information Sciences Institute, University of Southern California, Marina del Rey, Los Angeles, CA
Ewa Deelman , Information Sciences Institute, University of Southern California, Marina del Rey, Los Angeles, CA
Thomas Fahringer , Institute for Computer Science, University of Innsbruck, Innsbruck, Austria
ABSTRACT
Task clustering has proven to be an effective method to reduce execution overhead and to improve the computational granularity of scientific workflow tasks executing on distributed resources. However, a job composed of multiple tasks may have a higher risk of suffering from failures than a single task job. In this paper, we conduct a theoretical analysis of the impact of transient failures on the runtime performance of scientific workflow executions. We propose a general task failure modeling framework that uses a maximum likelihood estimation-based parameter estimation process to model workflow performance. We further propose three fault-tolerant clustering strategies to improve the runtime performance of workflow executions in faulty execution environments. Experimental results show that failures can have significant impact on executions where task clustering policies are not fault-tolerant, and that our solutions yield makespan improvements in such scenarios. In addition, we propose a dynamic task clustering strategy to optimize the workflow's makespan by dynamically adjusting the clustering granularity when failures arise. A trace-based simulation of five real workflows shows that our dynamic method is able to adapt to unexpected behaviors, and yields better makespans when compared to static methods.
INDEX TERMS
Runtime, Maximum likelihood estimation, Shape, Delays, Cloud computing, Fault tolerance, Fault tolerant systems,job grouping, scientific workflows, fault tolerance, parameter estimation, failure, machine learning, task clustering,job grouping, Scientific workflows, fault tolerance, parameter estimation, failure, machine learning, task clustering
CITATION
Weiwei Chen, Rafael Ferreira da Silva, Ewa Deelman, Thomas Fahringer, "Dynamic and Fault-Tolerant Clustering for Scientific Workflows", IEEE Transactions on Cloud Computing, vol. 4, no. , pp. 49-62, Jan.-March 2016, doi:10.1109/TCC.2015.2427200
264 ms
(Ver 3.3 (11022016))