The Community for Technology Leaders
2013 IEEE 9th International Conference on eScience (eScience) (2013)
Beijing, China
Oct. 22, 2013 to Oct. 25, 2013
ISBN: 978-0-7695-5083-1
pp: 188-195
Weiwei Chen , Inf. Sci. Inst., Univ. of Southern California, Marina, CA, USA
Rafael Ferreira Da Silva , CREATIS, Univ. of Lyon, Villeurbanne, France
Ewa Deelman , Inf. Sci. Inst., Univ. of Southern California, Marina, CA, USA
Rizos Sakellariou , Sch. of Comput. Sci., Univ. of Manchester, Manchester, UK
Scientific workflows can be composed of many fine computational granularity tasks. The runtime of these tasks may be shorter than the duration of system overheads, for example, when using multiple resources of a cloud infrastructure. Task clustering is a runtime optimization technique that merges multiple short tasks into a single job such that the scheduling overhead is reduced and the overall runtime performance is improved. However, existing task clustering strategies only provide a coarse-grained approach that relies on an over-simplified workflow model. In our work, we examine the reasons that cause Runtime Imbalance and Dependency Imbalance in task clustering. Next, we propose quantitative metrics to evaluate the severity of the two imbalance problems respectively. Furthermore, we propose a series of task balancing methods to address these imbalance problems. Finally, we analyze their relationship with the performance of these task balancing methods. A trace-based simulation shows our methods can significantly improve the runtime performance of two widely used workflows compared to the actual implementation of task clustering.
Runtime, Delays, Clustering algorithms, Engines, Educational institutions, Optimization,task clustering, Scientific workflow, data locality, load balance
Weiwei Chen, Rafael Ferreira Da Silva, Ewa Deelman, Rizos Sakellariou, "Balanced Task Clustering in Scientific Workflows", 2013 IEEE 9th International Conference on eScience (eScience), vol. 00, no. , pp. 188-195, 2013, doi:10.1109/eScience.2013.40
182 ms
(Ver 3.3 (11022016))