This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing
June 2011 (vol. 22 no. 6)
pp. 899-909
ASCII Text
x 
 
Dengpan Yin, Esma Yildirim, Sivakumar Kulasekaran, Brandon Ross, Tevfik Kosar, "A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing," IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 6, pp. 899-909, June, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TPDS.2010.187,
author = {Dengpan Yin and Esma Yildirim and Sivakumar Kulasekaran and Brandon Ross and Tevfik Kosar},
title = {A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing},
journal ={IEEE Transactions on Parallel and Distributed Systems},
volume = {22},
number = {6},
issn = {1045-9219},
year = {2011},
pages = {899-909},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPDS.2010.187},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Parallel and Distributed Systems
TI - A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing
IS - 6
SN - 1045-9219
SP899
EP909
EPD - 899-909
A1 - Dengpan Yin,
A1 - Esma Yildirim,
A1 - Sivakumar Kulasekaran,
A1 - Brandon Ross,
A1 - Tevfik Kosar,
PY - 2011
KW - Many-task computing
KW - modeling
KW - scheduling
KW - parallel TCP streams
KW - optimization
KW - prediction
KW - stork.
VL - 22
JA - IEEE Transactions on Parallel and Distributed Systems
ER -
 
Dengpan Yin, Louisiana State University, Baton Rouge
Esma Yildirim, Louisiana State University, Baton Rouge
Sivakumar Kulasekaran, Louisiana State University, Baton Rouge
Brandon Ross, Louisiana State University, Baton Rouge
Tevfik Kosar, Louisiana State University, Baton Rouge
In this paper, we present the design and implementation of an application-layer data throughput prediction and optimization service for many-task computing in widely distributed environments. This service uses multiple parallel TCP streams to improve the end-to-end throughput of data transfers. A novel mathematical model is developed to determine the number of parallel streams, required to achieve the best network performance. This model can predict the optimal number of parallel streams with as few as three prediction points. We implement this new service in the Stork Data Scheduler, where the prediction points can be obtained using Iperf and GridFTP samplings. Our results show that the prediction cost plus the optimized transfer time is much less than the nonoptimized transfer time in most cases. As a result, Stork data transfer jobs with optimization service can be completed much earlier, compared to nonoptimized data transfer jobs.

[1] I. Raicu, I. Foster, and Y. Zhao, “Many-Task Computing for Grids and Supercomputers,” Proc. IEEE Workshop Many-Task Computing on Grids and Supercomputers (MTAGS), 2008.
[2] Louisiana Optical Network Initiative (LONI), http:/www.loni.org/, 2010.
[3] Energy Sciences Network (ESNet), http:/www.es.net/, 2010.
[4] TeraGrid, http:/www.teragrid.org/, 2010.
[5] S. Floyd, “Rfc 3649: Highspeed TCP for Large Congestion Windows,” 2003.
[6] R. Kelly, “Scalable TCP: Improving Performance in Highspeed Wide Area Networks,” Computer Comm. Rev., vol. 32, no. 2, pp. 83-91, 2003.
[7] C. Jin, D.X. Wei, S.H. Low, G. Buhrmaster, J. Bunn, D.H. Choe, R.L.A. Cottrell, J.C. Doyle, W. Feng, O. Martin, H. Newman, F. Paganini, S. Ravot, and S. Singh, “Fast TCP: From Theory to Experiments,” IEEE Network, vol. 19, no. 1, pp. 4-11, Feb. 2005.
[8] H. Sivakumar, S. Bailey, and R.L. Grossman, “Psockets: The Case for Application-Level Network Striping for Data Intensive Applications Using High Speed Wide Area Networks,” Proc. IEEE Super Computing Conf. (SC '00), p. 63, Nov. 2000.
[9] J. Lee, D. Gunter, B. Tierney, B. Allcock, J. Bester, J. Bresnahan, and S. Tuecke, “Applied Techniques for High Bandwidth Data Transfers across Wide Area Networks,” Proc. Int'l Conf. Computing in High Energy and Nuclear Physics (CHEP '01), Sept. 2001.
[10] H. Balakrishman, V.N. Padmanabhan, S. Seshan, and R.H.K.M. Stemm, “TCP Behavior of a Busy Internet Server: Analysis and Improvements,” Proc. IEEE INFOCOM '98, pp. 252-262, Mar. 1998.
[11] T.J. Hacker, B.D. Noble, and B.D. Atley, “Adaptive Data Block Scheduling for Parallel Streams,” Proc. IEEE Int'l Symp. High Performance Distributed Computing (HPDC '05), pp. 265-275, July 2005.
[12] L. Eggert, J. Heideman, and J. Touch, “Effects of Ensemble TCP,” ACM Computer Comm. Rev., vol. 30, no. 1, pp. 15-29, 2000.
[13] R.P. Karrer, J. Park, and J. Kim, “TCP-ROME: Performance and Fairness in Parallel Downloads for Web and Real Time Multimedia Streaming Applications,” technical report, Deutsche Telekom Labs, 2006.
[14] D. Lu, Y. Qiao, and P.A. Dinda, “Characterizing and Predicting TCP Throughput on the Wide Area Network,” Proc. IEEE Int'l Conf. Distributed Computing Systems (ICDCS '05), pp. 414-424, 2005.
[15] E. Yildirim, D. Yin, and T. Kosar, “Prediction of Optimal Parallelism Level in Wide Area Data Transfers,” to be published in IEEE Trans. Parallel and Distributed Systems, 2010.
[16] A. Tirumala, L. Cottrell, and T. Dunnigan, “Measuring End-to-End Bandwidth with Iperf Using Web100,” Proc. Passive and Active Measurement Workshop (PAM '03), 2003.
[17] W. Allcock, J. Bresnahan, R. Kettimuthu, M. Link, C. Dumitrescu, I. Raicu, and I. Foster, “The Globus Striped Gridftp Framework and Server,” Proc. 2005 ACM/IEEE Conf. Supercomputing (SC '05), p. 54, 2005.
[18] T. Kosar and M. Livny, “Stork: Making Data Placement a First Class Citizen in the Grid,” Proc. IEEE Int'l Conf. Distributed Computing Systems (ICDCS '04), pp. 342-349, 2004.
[19] T.J. Hacker, B.D. Noble, and B.D. Atley, “The End-to-End Performance Effects of Parallel TCP Sockets on a Lossy Wide Area Network,” Proc. IEEE Int'l Symp. Parallel and Distributed Processing (IPDPS '02), pp. 434-443, 2002.
[20] D. Lu, Y. Qiao, P.A. Dinda, and F.E. Bustamante, “Modeling and Taming Parallel TCP on the Wide Area Network,” Proc. IEEE Int'l Symp. Parallel and Distributed Processing (IPDPS '05), p. 68b, Apr. 2005.
[21] E. Altman, D. Barman, B. Tuffin, and M. Vojnovic, “Parallel TCP Sockets: Simple Model, Throughput and Validation,” Proc. IEEE INFOCOM '06, pp. 1-12, Apr. 2006.
[22] J. Crowcroft and P. Oechslin, “Differentiated End-to-End Internet Services Using a Weighted Proportional Fair Sharing TCP,” ACM SIGCOMM Computer Comm. Rev., vol. 28, no. 3, pp. 53-69, July 1998.
[23] G. Kola and M.K. Vernon, “Target Bandwidth Sharing Using Endhost Measures,” Performance Evaluation, vol. 64, nos. 9-12, pp. 948-964, Oct. 2007.
[24] M. Jain and C. Dovrolis, “Pathload: A Measurement Tool for End-to-End Available Bandwidth,” Proc. Passive and Active Measurements Workshop (PAM), pp. 14-25, 2002.
[25] V.J. Ribeiro, R.H. Riedi, R.G. Baraniuk, J. Navratil, and L. Cottrell, “Pathchirp: Efficient Available Bandwidth Estimation for Network,” Proc. Passive and Active Measurement Workshop, 2003.
[26] P. Primet, R. Harakaly, and F. Bonnassieux, “Experiments of Network Throughput Measurement and Forecasting Using the Network Weather,” Proc. Second IEEE/ACM Int'l Symp. Cluster Computing and the Grid (CCGRID '02), p. 413, 2002.
[27] E. Yildirim, I.H. Suslu, and T. Kosar, “Which Network Measurement Tool is Right for You? A Multidimensional Comparison Study,” Proc. Ninth IEEE/ACM Int'l Conf. Grid Computing (GRID '08), pp. 266-275, 2008.
[28] J. Strauss, D. Katabi, and M.F. Kaashoek, “A Measurement Study of Available Bandwidth Estimation Tools,” Proc. Internet Measurement Conf., pp. 39-44, 2003.
[29] T. Dunigan, M. Mathis, and B. Tierney, “A TCP Tuning Daemon,” Proc. IEEE Int'l Conf. Supercomputing (SC '02), pp. 1-16, Nov. 2002.
[30] G. Jin, G. Yang, B.R. Crowley, and D.A. Agarwal, “Network Characterization Service (NCS),” Proc. IEEE Int'l Symp. High Performance Distributed Computing (HPDC '01), pp. 289-299, 2001.
[31] LBNL “Netest,” http://www-didc.lbl.gov/NCSnetest/, 2010.
[32] D. Thain, T. Tannenbaum, and M. Livny, “Distributed Computing in Practice: The Condor Experience: Research Articles,” Concurrency and Computation: Practice and Experience, vol. 17, nos. 2-4, pp. 323-356, 2005.
[33] D. Yin, E. Yildirim, and T. Kosar, “A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing,” Proc. Second Workshop Many-Task Computing on Grids and Supercomputers (MTAGS '09), Nov. 2009.

Index Terms:
Many-task computing, modeling, scheduling, parallel TCP streams, optimization, prediction, stork.
Citation:
Dengpan Yin, Esma Yildirim, Sivakumar Kulasekaran, Brandon Ross, Tevfik Kosar, "A Data Throughput Prediction and Optimization Service for Widely Distributed Many-Task Computing," IEEE Transactions on Parallel and Distributed Systems, vol. 22, no. 6, pp. 899-909, June 2011, doi:10.1109/TPDS.2010.187
Usage of this product signifies your acceptance of the Terms of Use.