The Community for Technology Leaders
2014 IEEE International Conference On Cluster Computing (CLUSTER) (2014)
Madrid, Spain
Sept. 22, 2014 to Sept. 26, 2014
ISBN: 978-1-4799-5548-0
pp: 185-193
Chih-Song Kuo , RWTH Aachen University, Department of Computer Science, Germany
Aamer Shah , RWTH Aachen University, Department of Computer Science, Germany
Akihiro Nomura , Tokyo Institute of Technology, Global Scientific Information and Computing Center, Japan
Satoshi Matsuoka , Tokyo Institute of Technology, Global Scientific Information and Computing Center, Japan
Felix Wolf , RWTH Aachen University, Department of Computer Science, Germany
ABSTRACT
On large-scale clusters, tens to hundreds of applications can simultaneously access a parallel file system, leading to contention and in its wake to degraded application performance. However, the degree of interference depends on the specific file access pattern. On the basis of synchronized time-slice profiles, we compare the interference potential of different file access patterns. We consider both micro-benchmarks, to study the effects of certain patterns in isolation, and realistic applications to gauge the severity of such interference under production conditions. In particular, we found that writing large files simultaneously with small files can slow down the latter at small chunk sizes but the former at larger chunk sizes. We further show that such effects can seriously affect the runtime of real applications—up to a factor of five in one instance. In the future, both our insights and profiling techniques can be used to automatically classify the interference potential between applications and to adjust scheduling decisions accordingly.
INDEX TERMS
Interference, Servers, Throughput, Benchmark testing, Market research, Degradation, Aggregates
CITATION

C. Kuo, A. Shah, A. Nomura, S. Matsuoka and F. Wolf, "How file access patterns influence interference among cluster applications," 2014 IEEE International Conference On Cluster Computing (CLUSTER), Madrid, Spain, 2014, pp. 185-193.
doi:10.1109/CLUSTER.2014.6968743
96 ms
(Ver 3.3 (11022016))