2006 15th IEEE International Conference on High Performance Distributed Computing
When Jobs Play Nice: The Case For Symbiotic Space-Sharing
Paris
June 19-June 23
ISBN: 1-4244-0307-3
W. Weinberg, San Diego Supercomput. Center, California Univ., San Diego, La Jolla, CA
A. Snavely, San Diego Supercomput. Center, California Univ., San Diego, La Jolla, CA
Using a large HPC platform, we investigate the effectiveness of "symbiotic space-sharing", a technique that improves system throughput by executing parallel applications in combinations and configurations that alleviate pressure on shared resources. We demonstrate that relevant benchmarks commonly suffer a 10-60% penalty in runtime efficiency due to memory resource bottlenecks and up to several orders of magnitude for I/O. We show that this penalty can be often mitigated, and sometimes virtually eliminated, by symbiotic space-sharing techniques and deploy a prototype scheduler that leverages these findings to improve system throughput by 20%
Index Terms:
system throughput, symbiotic space-sharing, large HPC platform, parallel application, resources sharing, memory resource bottleneck, I/O bottleneck, prototype scheduler
Citation:
W. Weinberg, A. Snavely, "When Jobs Play Nice: The Case For Symbiotic Space-Sharing," hpdc, pp.361-362, 2006 15th IEEE International Conference on High Performance Distributed Computing, 2006