This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Numerical Method for the Evaluation of the Distribution of Cumulative Reward till Exit of a Subset of Transient States of a Markov Reward Model
November/December 2011 (vol. 8 no. 6)
pp. 798-809
ASCII Text
x 
 
Juan A. Carrasco, Víctor Suñé, "A Numerical Method for the Evaluation of the Distribution of Cumulative Reward till Exit of a Subset of Transient States of a Markov Reward Model," IEEE Transactions on Dependable and Secure Computing, vol. 8, no. 6, pp. 798-809, November/December, 2011.
 
 
BibTex
x
 
@article{ 10.1109/TDSC.2010.49,
author = {Juan A. Carrasco and Víctor Suñé},
title = {A Numerical Method for the Evaluation of the Distribution of Cumulative Reward till Exit of a Subset of Transient States of a Markov Reward Model},
journal ={IEEE Transactions on Dependable and Secure Computing},
volume = {8},
number = {6},
issn = {1545-5971},
year = {2011},
pages = {798-809},
doi = {http://doi.ieeecomputersociety.org/10.1109/TDSC.2010.49},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Dependable and Secure Computing
TI - A Numerical Method for the Evaluation of the Distribution of Cumulative Reward till Exit of a Subset of Transient States of a Markov Reward Model
IS - 6
SN - 1545-5971
SP798
EP809
EPD - 798-809
A1 - Juan A. Carrasco,
A1 - Víctor Suñé,
PY - 2011
KW - Fault tolerance
KW - modeling techniques
KW - Markov reward models
KW - numerical algorithms.
VL - 8
JA - IEEE Transactions on Dependable and Secure Computing
ER -
 
Juan A. Carrasco, Universitat Politècnica de Catalunya, Barcelona
Víctor Suñé, Universitat Politècnica de Catalunya, Barcelona
Markov reward models have interesting modeling applications, particularly those addressing fault-tolerant hardware/software systems. In this paper, we consider a Markov reward model with a reward structure including only reward rates associated with states, in which both positive and negative reward rates are present and null reward rates are allowed, and develop a numerical method to compute the distribution function of the cumulative reward till exit of a subset of transient states of the model. The method combines a model transformation step with the solution of the transformed model using a randomization construction with two randomization rates. The method introduces a truncation error, but that error is strictly bounded from above by a user-specified error control parameter. Further, the method is numerically stable and takes advantage of the sparsity of the infinitesimal generator of the transformed model. Using a Markov reward model of a fault-tolerant hardware/software system, we illustrate the application of the method and analyze its computational cost. Also, we compare the computational cost of the method with that of the (only) previously available method for the problem. Our numerical experiments seem to indicate that the new method can be efficient and that for medium size and large models can be substantially faster than the previously available method.

[1] M.D. Beaudry, “Performance-Related Reliability Measures for Computing Systems,” IEEE Trans. Computers, vol. 27, no. 6, pp. 540-547, June 1978.
[2] M. Malhotra, “A Computationally Efficient Technique for Transient Analysis of Repairable Markovian Systems,” Performance Evaluation, vol. 24, no. 4, pp. 311-331, 1996.
[3] A. Reibman and K. Trivedi, “Numerical Transient Analysis of Markov Models,” Computers and Operations Research, vol. 15, no. 1, pp. 19-36, 1988.
[4] V.G. Kulkarni, “A New Class of Multivariate Phase Type Distributions,” Operations Research, vol. 37, no. 1, pp. 151-158, 1989.
[5] G. Ciardo, R.A. Marie, B. Sericola, and K.S. Trivedi, “Performability Analysis Using Semi-Markov Reward Processes,” IEEE Trans. Computers, vol. 39, no. 10, pp. 1251-1264, Oct. 1990.
[6] S. Ahn and V. Ramaswami, “Bilateral Phase Type Distributions,” Stochastic Models, vol. 21, no. 2, pp. 239-259, 2005.
[7] J.F. Meyer, “On Evaluating the Performability of Degradable Computing Systems,” IEEE Trans. Computers, vol. 29, no. 8, pp. 720-731, Aug. 1980.
[8] R.M. Smith, K.S. Trivedi, and A.V. Ramesh, “Performability Analysis: Measures, an Algorithm, and a Case Study,” IEEE Trans. Computers, vol. 37, no. 4, pp. 406-417, Apr. 1988.
[9] E. de Souza e Silva and H.R. Gail, “Calculating Availability and Performability Measures of Repairable Computer Systems Using Randomization,” J. ACM, vol. 36, no. 1, pp. 171-193, 1989.
[10] S.M.R. Islam and H.H. Ammar, “Performability of the Hypercube,” IEEE Trans. Reliability, vol. 38, no. 5, pp. 518-526, Dec. 1989.
[11] L. Donatiello and V. Grassi, “On Evaluating the Cumulative Performance Distribution of Fault-Tolerant Computer Systems,” IEEE Trans. Computers, vol. 40, no. 11, pp. 1301-1307, Nov. 1991.
[12] K.R. Pattipati, Y. Li, and H.A.P. Blom, “A Unified Framework for the Performability Evaluation of Fault-Tolerant Computer Systems,” IEEE Trans. Computers, vol. 42, no. 3, pp. 312-326, Mar. 1993.
[13] H. Nabli and B. Sericola, “Performability Analysis: A New Algorithm,” IEEE Trans. Computers, vol. 45, no. 4, pp. 491-494, Apr. 1996.
[14] V. Suñé, J.A. Carrasco, H. Nabli, and B. Sericola, “Comment on ‘Performability Analysis: A New Algorithm’,” IEEE Trans. Computers, vol. 59, no. 1, pp. 137-138, Jan. 2010.
[15] M.A. Qureshi and W.H. Sanders, “A New Methodology for Calculating Distributions of Reward Accumulated during a Finite Interval,” Proc. IEEE 26th Int'l Symp. Fault-Tolerant Computing, pp. 116-125, June 1996.
[16] E. de Souza e Silva and H.R. Gail, “An Algorithm to Calculate Transient Distributions of Cumulative Rate and Impulse Based Reward,” Stochastic Models, vol. 14, no. 3, pp. 509-536, 1998.
[17] S. Rácz, Á. Tari, and M. Telek, “MRMSolve Distribution Estimation of Large Markov Reward Models,” Computer Performance Evaluation: Modelling Techniques and Tools, pp. 141-158, Springer-Verlag, 2002.
[18] J.A. Carrasco, “Two Methods for Computing Bounds for the Distribution of Cumulative Reward for Large Markov Models,” Performance Evaluation, vol. 63, no. 12, pp. 1165-1195, 2006.
[19] W.K. Grassmann, “Transient Solutions in Markovian Queueing Systems,” Computers and Operations Research, vol. 4, no. 1, pp. 47-53, 1977.
[20] N.J. Higham, “The Accuracy of Floating Point Summation,” SIAM J. Scientific Computing, vol. 14, no. 4, pp. 783-799, 1993.
[21] IEEE Std 754-2008 Standard for Floating-Point Arithmetic (Revision of IEEE Std 754-1985), IEEE Computer Society, 2008.
[22] T.A. Davis, Direct Methods for Sparse Linear Systems. SIAM, 2006.
[23] I. Koren and C.M. Krishna, Fault-Tolerant Systems. Morgan Kaufmann, 2007.
[24] M. Galassi, J. Davies, J. Theiler, B. Gough, G. Jungman, P. Alken, M. Booth, and F. Rossi, GNU Scientific Library Reference Manual, third ed. Network Theory Limited, 2009.
[25] R.C. Whaley, A. Petitet, and J.J. Dongarra, “Automated Empirical Optimizations of Software and the ATLAS Project,” Parallel Computing, vol. 27, nos. 1/2, pp. 3-35, 2001.

Index Terms:
Fault tolerance, modeling techniques, Markov reward models, numerical algorithms.
Citation:
Juan A. Carrasco, Víctor Suñé, "A Numerical Method for the Evaluation of the Distribution of Cumulative Reward till Exit of a Subset of Transient States of a Markov Reward Model," IEEE Transactions on Dependable and Secure Computing, vol. 8, no. 6, pp. 798-809, Nov.-Dec. 2011, doi:10.1109/TDSC.2010.49
Usage of this product signifies your acceptance of the Terms of Use.