The Community for Technology Leaders
SC Conference (1999)
Portland, Oregon, USA
Nov. 13, 1999 to Nov. 18, 1999
ISBN: 1-58113-091-0
pp: 13
Christos Faloutsos , Carnegie Mellon University
Tara M. Madhyastha , University of California, Santa Cruz
Garth A. Gibson , Carnegie Mellon University
Optimizing collective input/output (I/O) is important for improving throughput of parallel scientific applications. Current research suggests that a specialized collective application programming interface, coupled with system-level optimizations, is necessary to obtain good I/O performance. Unfortunately, collective interfaces require an application to disclose its entire access pattern to fully reorder I/O requests, and cannot flexibly utilize additional memory to improve performance. In this paper we propose and analyze a method of optimizing collective access patterns using informed prefetching that is capable of exploiting any amount of available memory to overlap I/O with computation. We compare this approach to disk-directed I/O, an efficient implementation of a collective I/O interface. Moreover, we prove that under certain conditions, a per-processor prefetch depth equal to the number of drives can guarantee sequential disk accesses for any collectively accessed file. In empirical studies, a prefetch horizon of one to two times the number of disks per processor is sufficient to match the performance of disk-directed I/O for sequentially allocated files. Finally, we develop accurate analytical models to predict the throughput of informed prefetching for collective reads as a function of the per-processor prefetch depth.
Christos Faloutsos, Tara M. Madhyastha, Garth A. Gibson, "Informed Prefetching of Collective Input/Output Requests", SC Conference, vol. 00, no. , pp. 13, 1999, doi:10.1109/SC.1999.10051
511 ms
(Ver 3.3 (11022016))