loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2
Improving parallel data transfer times using predicted variances in shared networks
Cardiff, Wales, UK
May 09-May 12
ISBN: 0-7803-9074-1
Lingyun Yang, Dept. of Comput. Sci., Chicago Univ., IL, USA
J.M. Schopf, COPPE, Univ. Fed. do Rio de Janeiro, Brazil
I. Foster, COPPE, Univ. Fed. do Rio de Janeiro, Brazil
It is increasingly common to use multiple distributed storage systems as a single data store within which large datasets may be replicated. Thus, we face the problem of how to access replicated data efficiently. Multiple-source parallel transfers can reduce access times by transferring data from several replicas in parallel. However, we then face the problem of deciding which data to fetch from which replicas. We propose a Tuned Conservative scheduling technique that uses predicted means and variances for network performance to make data selection decisions. This stochastic scheduling technique adjusts the amount of data fetched on a link according to not only the link performance but the expected variance in that performance. We incorporate our technique into the striped GridFTP server from the Globus Toolkit, and demonstrate that the technique can produce data transfer times that are significantly faster and less variable than those of other techniques.
Citation:
Lingyun Yang, J.M. Schopf, I. Foster, "Improving parallel data transfer times using predicted variances in shared networks," ccgrid, vol. 2, pp.734-742, Fifth IEEE International Symposium on Cluster Computing and the Grid (CCGrid'05) - Volume 2, 2005
Usage of this product signifies your acceptance of the Terms of Use.