loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
2009 International Conference on Parallel Processing
Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand
Vienna, Austria
September 22-September 25
ISBN: 978-0-7695-3802-0
The rapid growth of InfiniBand, 10 Gigabit Ethernet/iWARP and IB WAN extensions is increasingly gaining momentum for designing high end computing clusters and data-centers. For typical applications such as data staging, content replication and remote site backup, FTP has been the most popular method to transfer data within and across these clusters. Although the existing sockets based FTP approaches can be transparently used in these systems through the protocols like IPoIB or SDP, their performance and scalability are limited due to the additional interaction overhead and unoptimized protocol processing. This leads to a challenge how to design more efficient FTP mechanisms by leveraging the advanced features of modern interconnects. In this paper we design a new Advanced Data Transfer Service (ADTS) with the capabilities such as zero-copy data-transfer, memory registration cache, persistent data sessions and pipelined data transfer etc. to enable efficient zero-copy data transfers over IB and iWARP equipped LAN and WAN. We then utilize ADTS to design a high performance FTP library (FTP-ADTS). From our experimental results, we observe that our design outperforms existing sockets based approaches by more that 95 in transferring large volumes of data over LAN. It also provides significantly better performance at much lower (by up to a factor of 6) CPU utilization in various IB WAN scenarios. These results present the promising future for designing high performance communication protocols to power the efficiency and scalability of next-generation parallel and distributed environments.
Index Terms:
FTP, RDMA, Zero-copy, InfiniBand, IB WAN
Citation:
Ping Lai, Hari Subramoni, Sundeep Narravula, Amit Mamidala, Dhabaleswar K. Panda, "Designing Efficient FTP Mechanisms for High Performance Data-Transfer over InfiniBand," icpp, pp.156-163, 2009 International Conference on Parallel Processing, 2009
Usage of this product signifies your acceptance of the Terms of Use.