The Community for Technology Leaders
Green Image
<p><b>Abstract</b>—We investigate the Message Passing Interface Input/Output (MPI I/O) implementation issues for two overlapping access patterns: the overlaps among processes within a single I/O operation and the overlaps across a sequence of I/O operations. The former case considers whether I/O atomicity can be obtained in the overlapping regions. The latter focuses on the file consistency problem on parallel machines with client-side file caching enabled. Traditional solutions for both overlapping I/O problems use whole file or byte-range file locking to ensure exclusive access to the overlapping regions and bypass the file system cache. Unfortunately, not only can file locking serialize I/O, but it can also increase the aggregate communication overhead between clients and I/O servers. For atomicity, we first differentiate MPI's requirements from the Portable Operating System Interface (POSIX) standard and propose two scalable approaches, graph coloring and process-rank ordering, which can resolve access conflicts and maintain I/O parallelism. For solving the file consistency problem across multiple I/O operations, we propose a method called Persistent File Domains, which tackles cache coherency with additional information and coordination to guarantee safe cache access without using file locks. </p>
MPI, MPI I/O, atomic I/O, file atomicity, file consistency, cache coherence, overlapping I/O.
Neil Pundit, Alok Choudhary, Kenin Coloma, Lee Ward, Wei-keng Liao, Eric Russell, "Scalable Design and Implementations for MPI Parallel Overlapping I/O", IEEE Transactions on Parallel & Distributed Systems, vol. 17, no. , pp. 1264-1276, November 2006, doi:10.1109/TPDS.2006.163
84 ms
(Ver )