loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
8th International Symposium on Parallel Architectures,Algorithms and Networks (ISPAN'05)
An Efficient MPI-IO for Noncontiguous Data Access over InfiniBand
Las Vegas, Nevada, USA
December 07-December 09
ISBN: 0-7695-2509-1
Ding-Yong Hong, National Tsing Hua University, Taiwan
Ching-Wen You, National Tsing Hua University, Taiwan
Yeh-Ching Chung, National Tsing Hua University, Taiwan
Noncontiguous data access is a very common access pattern in many scientific applications. Using POSIX I/O to access many pieces of noncontiguous data segments will generate a lot of I/O requests that cause the I/O system perform poorly. Tow-phase I/O, also called collective I/O, applied by MPI-IO provides a good method to optimize noncontiguous I/O operations. However, this method requires many data segments being redistributed. This redistribution needs some information been calculated in advance and lots of data segments are transmitted via network more than once. With I/O-intensive applications, the aggregate size of these data segments being redistributed becomes significant large. This additional overhead will degrade the system performance. In this thesis, we extend the collective I/O method and propose a new I/O scheme to avoid retransmission of data segments by applying RDMA gather/scatter operations supported by InfiniBand hardware. We also extend the "view" and "datatype" concepts of MPI into the file system to help complete our design. The experiments show that the method we design improves the performance and is more efficient than the collective I/O approach.
Citation:
Ding-Yong Hong, Ching-Wen You, Yeh-Ching Chung, "An Efficient MPI-IO for Noncontiguous Data Access over InfiniBand," ispan, pp.140-147, 8th International Symposium on Parallel Architectures,Algorithms and Networks (ISPAN'05), 2005
Usage of this product signifies your acceptance of the Terms of Use.