The Community for Technology Leaders
2012 41st International Conference on Parallel Processing Workshops (2012)
Pittsburgh, PA, USA USA
Sept. 10, 2012 to Sept. 13, 2012
ISSN: 1530-2016
ISBN: 978-1-4673-2509-7
pp: 107-115
ABSTRACT
Job scheduling is a critical and complex task on large-scale supercomputers where a scheduling policy is expected to fulfill amorphous and sometimes conflicting goals from both users and system owners. Moreover, the effectiveness of a scheduling policy is dependent on workload characteristics which vary from time to time. Thus it is challenging to design a versatile scheduling policy that is effective in all circumstances. To address this issue, we propose an adaptive metric-aware job scheduling strategy. First, we propose metric-aware scheduling which enables the scheduler to balance competing scheduling goals represented by different metrics such as job waiting time, fairness, and system utilization. Second, we enhance the scheduler to adaptively adjust scheduling policies based on feedback information of monitored metrics at runtime. We evaluate our design using real workloads from supercomputer centers and demonstrate that our scheduling mechanism can significantly improve system performance in a balanced, sustainable fashion.
INDEX TERMS
Measurement, Scheduling, Resource management, Tuning, Monitoring, Processor scheduling, Schedules, adaptive policy tuning, job scheduling, resource management, supercomputer, metric-aware
CITATION

W. Tang, D. Ren, Z. Lan and N. Desai, "Adaptive Metric-Aware Job Scheduling for Production Supercomputers," 2012 41st International Conference on Parallel Processing Workshops(ICPPW), Pittsburgh, PA, USA USA, 2012, pp. 107-115.
doi:10.1109/ICPPW.2012.17
88 ms
(Ver 3.3 (11022016))