The Community for Technology Leaders
High-Performance Distributed Computing, International Symposium on (2006)
Paris
June 19, 2006 to June 23, 2006
ISBN: 1-4244-0307-3
pp: 291-302
A. Schuster , Technion, Haifa
D. Geiger , Technion, Haifa
M. Silberstein , Technion, Haifa
ABSTRACT
Consider a workload in which massively parallel tasks that require large resource pools are interleaved with short tasks that require fast response but consume fewer resources. We aim at achieving high throughput and short response time when scheduling such a workload over a set of uncoordinated grids of varying sizes and performance characteristics. We propose the concept of a grid execution hierarchy, where available grids are sorted according to their size, and the execution overheads increase with the size of the grids. We devise a scheduling algorithm for this execution hierarchy of grids by adapting the multilevel feedback queue approach to a multi-grid environment. The algorithm finds a grid of the size, availability, and overhead that best matches a task's resource requirements and expected turnaround time. Our approach is inspired by the shortest processing time first policy (SPTF), in the sense that the task's processing demands are constantly reevaluated during its run, so that a task is migrated to a more suitable level of the execution hierarchy when appropriate. We evaluate our approach in the context of the superlink-online system for processing genetic linkage analysis tasks - a production system consisting of several grids and utilizing tens of thousands of CPU hours a month. With our approach the system provides nearly interactive response time for shorter tasks, while simultaneously serving throughput-oriented massively parallel tasks in an efficient manner
INDEX TERMS
throughput-oriented parallel task, mixed workload scheduling algorithm, multigrid execution hierarchy, multilevel feedback queue approach, task resource requirement, shortest processing time first policy, SPTF, Superlink-online production system, genetic linkage analysis
CITATION
A. Schuster, M. Livny, D. Geiger, M. Silberstein, "Scheduling Mixed Workloads in Multi-grids: The Grid Execution Hierarchy", High-Performance Distributed Computing, International Symposium on, vol. 00, no. , pp. 291-302, 2006, doi:10.1109/HPDC.2006.1652160
111 ms
(Ver )