The Community for Technology Leaders
2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications (2013)
Melbourne, VIC Australia
July 16, 2013 to July 18, 2013
pp: 1049-1056
Long Wang , Inst. of High Performance Comput., A*STAR, Singapore, Singapore
Rubing Duan , Inst. of High Performance Comput., A*STAR, Singapore, Singapore
Xiaorong Li , Inst. of High Performance Comput., A*STAR, Singapore, Singapore
Sifei Lu , Inst. of High Performance Comput., A*STAR, Singapore, Singapore
Terence Hung , Inst. of High Performance Comput., A*STAR, Singapore, Singapore
Rodrigo N. Calheiros , Univ. of Melbourne, Melbourne, VIC, Australia
Rajkumar Buyya , Univ. of Melbourne, Melbourne, VIC, Australia
ABSTRACT
As more and more data can be generated at a faster-than-ever rate nowadays, it becomes a challenge to processing large volumes of data for complex data analysis. In order to address performance and cost issues of big data processing on clouds, we present a novel design of adaptive workflow management system which includes a data mining based prediction model, workflow scheduler, and iteration controls to optimize the data processing via iterative workflow tasks. We proposed a new heuristic algorithm, called Upgrade Fit, which dynamically and continuously reallocates multiple types of cloud resources to fulfill the performance and cost requirements. The iterative workflow tasks can be bursty bags of tasks to be executed repetitively for data processing. A real application of weather forecast workflow has been used to evaluate the capability of our system for large volume image data processing. Experimental system has been set up and the results indicate that the system can effectively handle multiple types of cloud resources and optimize the performance iteratively.
INDEX TERMS
Clouds, Optimization, Weather forecasting, Economics, Iterative methods, Support vector machines
CITATION

Long Wang et al., "An Iterative Optimization Framework for Adaptive Workflow Management in Computational Clouds," 2013 12th IEEE International Conference on Trust, Security and Privacy in Computing and Communications(TRUSTCOM), Melbourne, VIC Australia, 2014, pp. 1049-1056.
doi:10.1109/TrustCom.2013.128
416 ms
(Ver 3.3 (11022016))