loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
International Parallel and Distributed Processing Symposium (IPDPS'03)
Improving MPI-IO Output Performance with Active Buffering Plus Threads
Nice, France
April 22-April 26
ISBN: 0-7695-1926-1
Xiaosong Ma, University of Illinois at Urbana-Champaign
Marianne Winslett, University of Illinois at Urbana-Champaign
Jonghyun Lee, University of Illinois at Urbana-Champaign
Shengke Yu, University of Illinois at Urbana-Champaign
Efficient collective output of intermediate results to secondary storage becomes more and more important for scientific simulations as the gap between processing power/interconnection bandwidth and the I/O system bandwidth enlarges. Dedicated servers can offload I/O from compute processors and shorten the execution time, but it is not always possible or easy for an application to use them. We propose the use of active buffering with threads (ABT) for overlapping I/O with computation efficiently and flexibly without dedicated I/O servers. We show that the implementation of ABT in ROMIO, a popular implementation of MPI-IO, greatly reduces the application-visible cost of ROMIO?s collective write calls, and improves an application?s overall performance by hiding I/O cost and saving implicit synchronization overhead from collective write operations. Further, ABT is high-level, platform-independent, and transparent to users, giving users the benefit of overlapping I/O with other processing tasks even when the file system or parallel I/O library does not support asynchronous I/O.
Index Terms:
Collective I/O, MPI-IO, Parallel Simulation, Clusters
Citation:
Xiaosong Ma, Marianne Winslett, Jonghyun Lee, Shengke Yu, "Improving MPI-IO Output Performance with Active Buffering Plus Threads," ipdps, pp.68b, International Parallel and Distributed Processing Symposium (IPDPS'03), 2003
Usage of this product signifies your acceptance of the Terms of Use.