Subscribe

Issue No.01 - January (2012 vol.23)

pp: 134-145

Liang Hu , Coll. of Comput. Sci. & Technol., Jilin Univ., Changchun, China

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPDS.2011.108

ABSTRACT

Resource allocation and job scheduling are the core functions of grid computing. These functions are based on adequate information of available resources. Timely acquiring resource status information is of great importance in ensuring overall performance of grid computing. This work aims at building a distributed system for grid resource monitoring and prediction. In this paper, we present the design and evaluation of a system architecture for grid resource monitoring and prediction. We discuss the key issues for system implementation, including machine learning-based methodologies for modeling and optimization of resource prediction models. Evaluations are performed on a prototype system. Our experimental results indicate that the efficiency and accuracy of our system meet the demand of online system for grid resource monitoring and prediction.

INDEX TERMS

software architecture, grid computing, learning (artificial intelligence), resource allocation, scheduling, system architecture evaluation, online system, grid resource monitoring, machine learning-based prediction, resource allocation, job scheduling, grid computing, distributed system, Monitoring, Predictive models, Information services, Registers, Data models, Computer architecture, Containers, particle swarm optimization., Grid resource, monitoring and prediction, neural network, support vector machine, genetic algorithm

CITATION

Liang Hu, "Online System for Grid Resource Monitoring and Machine Learning-Based Prediction",

*IEEE Transactions on Parallel & Distributed Systems*, vol.23, no. 1, pp. 134-145, January 2012, doi:10.1109/TPDS.2011.108REFERENCES

- [1] L.F. Bittencourt and E.R.M. Madeira, “A Performance-Oriented Adaptive Scheduler for Dependent Tasks on Grids,”
Concurrency and Computation: Practice and Experience, vol. 20, no. 9, pp. 1029-1049, June 2008.- [2] F. Wolf and B. Mohr, “Hardware-Counter Based Automatic Performance Analysis of Parallel Programs,”
Proc. Conf. Parallel Computing (ParCo '03), pp. 753-760, Sept. 2003.- [3] J. Dugan et al., “Iperf Project,” http:/iperf.sourceforge.net/, Mar. 2008.
- [4] M. Livny et al., “Condor Hawkeye Project,”
Univ. of Wisconsin-Madison, http://www.cs.wisc.edu/condorhawkeye/, Sept. 2009.- [5] M.L. Massie, B.N. Chun, and D.E. Culler, “The Ganglia Distributed Monitoring System: Design, Implementation, and Experience,”
Parallel Computing, vol. 30, no. 7, pp. 817-840, July 2004.- [6] A. Waheed et al., “An Infrastructure for Monitoring and Management in Computational Grids,”
Proc. Fifth Int'l Workshop Languages, Compilers and Run-Time Systems for Scalable Computers, vol. 1915, pp. 235-245, Mar. 2000.- [7] J.S. Vetter and D.A. Reed, “Real-Time Performance Monitoring, Adaptive Control, and Interactive Steering of Computational Grids,”
Int'l J. High Performance Computing Applications, vol. 14, no. 4, pp. 357-366, 2000.- [8] D.M. Swany and R. Wolski, “Multivariate Resource Performance Forecasting in the Network Weather Service,”
Proc. ACM/IEEE Conf. Supercomputing, pp. 1-10, Nov. 2002.- [9] P.A. Dinda and D.R. O'Hallaron, “Host Load Prediction Using Linear Models,”
Cluster Computing, vol. 3, no. 4, pp. 265-280, 2000.- [10] E. Caron, A. Chis, F. Desprez, and A. Su, “Design of Plug-in Schedulers for a GRIDRPC Environment,”
Future Generation Computer Systems, vol. 24, no. 1, pp. 46-57, 2008.- [11] P.A. Dinda, “Design, Implementation, and Performance of an Extensible Toolkit for Resource Prediction in Distributed Systems,”
IEEE Trans. Parallel and Distributed Systems, vol. 17, no. 2, pp. 160-173, Feb. 2006.- [12] A.C. Sodan, G. Gupta, L. Han, L. Liu, and B. Lafreniere, “Time and Space Adaptation for Computational Grids with the ATOP-Grid Middleware,”
Future Generation Computer Systems, vol. 24, no. 6, pp. 561-581, 2008.- [13] M. Wu and X.H. Sun, “Grid Harvest Service: A Performance System of Grid Computing,”
J. Parallel and Distributed Computing, vol. 66, no. 10, pp. 1322-1337, 2006.- [14] L.T. Lee, D.F. Tao, and C. Tsao, “An Adaptive Scheme for Predicting the Usage of Grid Resources,”
Computing Computers and Electrical Eng., vol. 33, no. 1, pp. 1-11, 2007.- [15] A. Eswaradass, X.H. Sun, and M. Wu, “A Neural Network Based Predictive Mechanism for Available Bandwidth,”
Proc. 19th IEEE Int'l Parallel and Distributed Processing Symp. (IPDPS '05), p. 33a, 2005.- [16] V.N. Vapnik,
The Nature of Statistical Learning Theory, second ed. Springer-Verlag, 1999.- [17] H. Prem and N.R.S. Raghavan, “A Support Vector Machine Based Approach for Forecasting of Network Weather Services,”
J. Grid Computing, vol. 4, no. 1, pp. 89-114, 2006.- [18] J.H. Holland,
Adaptation in Natural and Artificial Systems. MIT Press, 1975.- [19] J. Kennedy and R.C. Eberhart, “Particle Swarm Optimization,”
Proc. IEEE Int'l Conf. Neural Networks, pp. 1942-1948, 1995.- [20] L. Fausett,
Fundamentals of Neural Networks. Prentice-Hall, 1994.- [21] S. Haykin,
Neural Networks: A Comprehensive Foundation. Macmillan Publishing, 1994.- [22] D. Patterson,
Artificial Neural Networks. Prentice-Hall, 1996.- [23] A.J. Smola and B. Schölkopf, “A Tutorial on Support Vector Regression,”
Statistics and Computing, vol. 14, no. 33, pp. 199-222, 2004.- [24] Bandwidth Data Set, http://www.slac.stanford.edu/comp/net/iepm-bw.slac.stanford.edu combinedata/, Feb. 2009.
- [25] Host Load Data Set, http://people.cs.uchicago.edu/~lyangLoad /, Feb. 2009.
- [26] Y. Shi and R.C. Eberhart, “A Modified Particle Swarm Optimizer,”
Proc. IEEE Int'l Conf. Evolutionary Computation, pp. 69-73, 2000.- [27] C.W. Hsu, C.C. Chang, and C.J. Lin, “A Practical Guide to Support Vector Classification,” http://www.csie.ntu.edu.tw/~cjlin/ papers/ guideguide.pdf Dept. of Computer Science and Information Eng., Nat'l Taiwan Univ., 2003.
- [28] M.W. Browne, “Cross-Validation Methods,”
J. Math. Psychology, vol. 44, no. 1, pp. 108-132, 2000.- [29] J. Kennedy and R.C. Eberhart, “A Discrete Binary Version of the Particle Swarm Optimization,”
Proc. IEEE Int'l Conf. Neural Networks, pp. 4104-4108, 1997.- [30] Globus Alliance “Globus Project,” http://www.globus.org/ toolkit/downloads 4.2.0/, 2008.
- [31] C.C. Chang and C.J. Lin, “LIBSVM: a Library for Support Vector Machines,” http://www.csie.ntu.edu.tw/~cjlinlibsvm/, May 2008.
- [32] S. Borja, “The Globus Toolkit 4 Programmer's Tutorial,” http://gdp.globus.orggt4-tutorial/, Nov. 2005.
- [33] M. Livny et al., “Condor Project,”
Univ. of Wisconsin-Madison, http://www.cs.wisc.educondor/, Sept. 2009. |