The Community for Technology Leaders
2013 IEEE International Conference on Cluster Computing (CLUSTER) (2010)
Heraklion, Greece
Sept. 20, 2010 to Sept. 24, 2010
ISBN: 978-0-7695-4220-1
pp: 312-321
I/O is the critical bottleneck for data-intensive scientific applications on HPC systems and leadership-class machines. Applications running on these systems may encounter bottlenecks because the I/O systems cannot handle the overwhelming intensity and volume of I/O requests. Applications and systems use I/O forwarding to aggregate and delegate I/O requests to storage systems. In this paper, we present two optimization techniques at the I/O forwarding layer to further reduce I/O bottlenecks on leadership-class computing systems. The first optimization pipelines data transfers so that I/O requests overlap at the network and file system layer. The second optimization merges I/O requests and schedules I/O request delegation to the back-end parallel file systems. We implemented these optimizations in the I/O Forwarding Scalability Layer and them on the T2K Open Supercomputer at the University of Tokyo and the Surveyor Blue Gene/P system at the Argonne Leadership Computing Facility. On both systems, the optimizations improved application I/O throughput, but highlighted additional areas of I/O contention at the I/O forwarding layer that we plan to address.
I/O Forwarding, Parallel file systems, Leadership-class machines
Robert Ross, Yutaka Ishikawa, Kamil Iskra, Dries Kimpe, Kazuki Ohta, Jason Cope, "Optimization Techniques at the I/O Forwarding Layer", 2013 IEEE International Conference on Cluster Computing (CLUSTER), vol. 00, no. , pp. 312-321, 2010, doi:10.1109/CLUSTER.2010.36
97 ms
(Ver 3.3 (11022016))