Sept. 20, 2010 to Sept. 24, 2010
I/O is the critical bottleneck for data-intensive scientific applications on HPC systems and leadership-class machines. Applications running on these systems may encounter bottlenecks because the I/O systems cannot handle the overwhelming intensity and volume of I/O requests. Applications and systems use I/O forwarding to aggregate and delegate I/O requests to storage systems. In this paper, we present two optimization techniques at the I/O forwarding layer to further reduce I/O bottlenecks on leadership-class computing systems. The first optimization pipelines data transfers so that I/O requests overlap at the network and file system layer. The second optimization merges I/O requests and schedules I/O request delegation to the back-end parallel file systems. We implemented these optimizations in the I/O Forwarding Scalability Layer and them on the T2K Open Supercomputer at the University of Tokyo and the Surveyor Blue Gene/P system at the Argonne Leadership Computing Facility. On both systems, the optimizations improved application I/O throughput, but highlighted additional areas of I/O contention at the I/O forwarding layer that we plan to address.
I/O Forwarding, Parallel file systems, Leadership-class machines
Kazuki Ohta, Dries Kimpe, Jason Cope, Kamil Iskra, Robert Ross, Yutaka Ishikawa, "Optimization Techniques at the I/O Forwarding Layer", CLUSTER, 2010, 2013 IEEE International Conference on Cluster Computing (CLUSTER), 2013 IEEE International Conference on Cluster Computing (CLUSTER) 2010, pp. 312-321, doi:10.1109/CLUSTER.2010.36