loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Prediction of Optimal Parallelism Level in Wide Area Data Transfers
December 2011 (vol. 22 no. 12)
pp. 2033-2045
Esma Yildirim, The State University of New York at Buffalo, Buffalo
Dengpan Yin, Louisiana State University, Baton Rouge
Tevfik Kosar, The State University of New York at Buffalo, Buffalo
Wide area data transfer may be a major bottleneck for the end-to-end performance of distributed applications. A practical way of increasing the wide area throughput at the application layer is using multiple parallel streams. Although increased number of parallel streams may yield much better performance than using a single stream, overwhelming the network by opening too many streams may have an inverse effect. The congestion created by excess number of streams may cause a drop down in the throughput achieved. Hence, it is important to decide on the optimal number of streams without congesting the network. Predicting this "optimum” number is not straightforward, since it depends on many parameters specific to each individual transfer. Generic models that try to predict this number either rely too much on historical information or fail to achieve accurate predictions. In this paper, we present a set of new models which aim to approximate the optimal number with least history information and lowest prediction overhead. An algorithm is introduced to select the best combination of historic information to do the prediction for evaluation purposes as well as optimizing prediction by reducing error rate. We measure the feasibility and accuracy of the proposed prediction models by comparing to actual GridFTP data transfer by using little historical information and have seen that we could predict the throughput of parallel streams accurately and find a very close approximation of the optimal stream number.
Index Terms:
Distributed applications, modeling and prediction, parallelism and concurrency, network protocols.
Citation:
Esma Yildirim, Dengpan Yin, Tevfik Kosar, "Prediction of Optimal Parallelism Level in Wide Area Data Transfers," IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 12, pp. 2033-2045, Dec. 2011, doi:10.1109/TPDS.2011.228
Usage of this product signifies your acceptance of the Terms of Use.