The Community for Technology Leaders
2015 IEEE 35th International Conference on Distributed Computing Systems (ICDCS) (2015)
Columbus, OH, USA
June 29, 2015 to July 2, 2015
ISSN: 1063-6927
ISBN: 978-1-4673-7214-5
pp: 744-745
ABSTRACT
Large-scale interactive services usually divide requests into multiple sub-requests and distribute them to a large number of server components for parallel execution. Hence the tail latency (i.e. The slowest component's latency) of these components determines the overall service latency. On a cloud platform, each component shares and competes node resources such as caches and I/O bandwidths with its co-located jobs, hence inevitably suffering from their performance interference. In this paper, we study the short-running jobs in a 12k-node Google cluster to illustrate the dynamic resource demands of these jobs, resulting in both individual components' latency variability over time and across different nodes and hence posing a major challenge to maintain low tail latency. Given this motivation, this paper introduces a dynamic and interference-aware scheduler for large-scale, parallel cloud services. At each scheduling interval, it collects workload and resource contention information of a running service, and predicts both the component latency on different nodes and the overall service performance. Based on the predicted performance, the scheduler identifies straggling components and conducts near-optimal component-node allocations to adapt to the changing workloads and performance interferences. We demonstrate that, using realistic workloads, the proposed approach achieves significant reductions in tail latency compared to the basic approach without scheduling.
INDEX TERMS
Google, Interference, Monitoring, Bandwidth, Resource management, Servers, Dynamic scheduling
CITATION

R. Han et al., "Interference-Aware Component Scheduling for Reducing Tail Latency in Cloud Interactive Services," 2015 IEEE 35th International Conference on Distributed Computing Systems (ICDCS), Columbus, OH, USA, 2015, pp. 744-745.
doi:10.1109/ICDCS.2015.88
222 ms
(Ver 3.3 (11022016))