This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Transient Analysis of Some Rewarded Markov Models Using Randomization with Quasistationarity Detection
September 2004 (vol. 53 no. 9)
pp. 1106-1120
ASCII Text
x 
 
Juan A. Carrasco, "Transient Analysis of Some Rewarded Markov Models Using Randomization with Quasistationarity Detection," IEEE Transactions on Computers, vol. 53, no. 9, pp. 1106-1120, September, 2004.
 
 
BibTex
x
 
@article{ 10.1109/TC.2004.68,
author = {Juan A. Carrasco},
title = {Transient Analysis of Some Rewarded Markov Models Using Randomization with Quasistationarity Detection},
journal ={IEEE Transactions on Computers},
volume = {53},
number = {9},
issn = {0018-9340},
year = {2004},
pages = {1106-1120},
doi = {http://doi.ieeecomputersociety.org/10.1109/TC.2004.68},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Computers
TI - Transient Analysis of Some Rewarded Markov Models Using Randomization with Quasistationarity Detection
IS - 9
SN - 0018-9340
SP1106
EP1120
EPD - 1106-1120
A1 - Juan A. Carrasco,
PY - 2004
KW - Rewarded continuous-time Markov chains
KW - transient analysis
KW - randomization
KW - quasistationary distribution.
VL - 53
JA - IEEE Transactions on Computers
ER -
 
Rewarded homogeneous continuous-time Markov chain (CTMC) models can be used to analyze performance, dependability and performability attributes of computer and telecommunication systems. In this paper, we consider rewarded CTMC models with a reward structure including reward rates associated with states and two measures summarizing the behavior in time of the resulting reward rate random variable: the expected transient reward rate at time t and the expected averaged reward rate in the time interval [0,t]. Computation of those measures can be performed using the randomization method, which is numerically stable and has good error control. However, for large stiff models, the method is very expensive. Exploiting the existence of a quasistationary distribution in the subset of transient states of discrete-time Markov chains with a certain structure, we develop a new variant of the (standard) randomization method, randomization with quasistationarity detection, covering finite CTMC models with state space S\cup\{f_1,f_2,\ldots,f_A\}, A\geq 1, where all states in S are transient and reachable among them and the states f_i are absorbing. The method has the same good properties as the standard randomization method and can be much more efficient. We also compare the performance of the method with that of regenerative randomization.

[1] Handbook of Mathematical Functions, M. Abramowitz and I.A. Stegun, eds. Dover, 1965.
[2] C. Béounes, M. Aguéra, J. Arlat, S. Bachman, C. Bourdeau, J.E. Doucet, K. Kanoun, J.-C. Laprie, S. Metge, J. Moreira de Souza, D. Powell, and P. Spiesser, “SURF2: A Program for Dependability Evaluation of Complex Hardware and Software Systems” Proc. 23rd IEEE Int'l Symp. Fault-Tolerant Computing, pp. 668-673, Toulouse, France, 1993.
[3] P.N. Bowerman, R.G. Nolty, and E.M. Scheuer, “Calculation of the Poisson Cumulative Distribution Function,” IEEE Trans. Reliability, vol. 39, pp. 158-161, 1990.
[4] J.A. Carrasco, Transient Analysis of Large Markov Models with Absorbing States Using Regenerative Randomization Technical Report DMSD_99_2, Universitat Politécnica de Catalunya, Feb. 1999, ftp://ftp-eel.upc.estechreports.
[5] J.A. Carrasco, Computation of Bounds for Transient Measures of Large Rewarded Markov Models Using Regenerative Randomization Computers and Operations Research, vol. 30, pp. 1005-1035, June 2003.
[6] J.A. Carrasco, Computationally Efficient and Numerically Stable Reliability Bounds for Repairable Fault-Tolerant Systems IEEE Trans. Computers, vol. 51, no. 3, pp. 254-268, Mar. 2002.
[7] J.A. Carrasco, Markovian Dependability/Performability Modeling of Fault-Tolerant Systems Reliability Eng. Handbook, H. Pham, ed., pp. 613-642, Springer-Verlag, 2003.
[8] G. Ciardo, J. Muppala, and K. Trivedi, SPNP: Stochastic Petri Net Package Proc. Third Int'l Workshop Petri Nets and Performance Models, pp. 142-151, 1989.
[9] B.L. Fox and P.W. Glynn, Computing Poisson Probabilities Comm. ACM, vol. 31, pp. 440-445, 1988.
[10] A. Goyal, W.C. Carter, E. de Souza e Silva, and S.S. Lavenberg, The System Availability Estimator Proc. 16th IEEE Int'l Symp. Fault-Tolerant Computing (FTCS-16), pp. 84-89, July 1986.
[11] W.K. Grassmann, Transient Solutions in Markovian Queuing Systems Computers and Operations Research, vol. 4, pp. 47-53, 1977.
[12] W.K. Grassmann, Means and Variances of Time Averages in Markovian Environments European J. Operational Research, vol. 31, no. 4, pp. 839-854, Oct. 1984.
[13] D. Gross and D.R. Miller, The Randomization Technique as a Modelling Tool and Solution Procedure for Transient Markov Processes Operations Research, vol. 32, pp. 343-361, 1984.
[14] M. Kijima, Markov Processes for Stochastic Modeling. London: Chapman&Hill, 1997.
[15] L. Knüsel, Computation of the Chi-Square and Poisson Distribution SIAM J. Scientific and Statistical Computing, vol. 7, no. 3, pp. 1022-1036, July 1986.
[16] M. Malhotra, J.K. Muppala, and K.S. Trivedi, Stiffness-Tolerant Methods for Transient Analysis of Stiff Markov Chains Microelectronics and Reliability, vol. 34, no. 11, pp. 1825-1841, 1994.
[17] M. Malhotra, A Computationally Efficient Technique for Transient Analysis of Repairable Markovian Systems Performance Evaluation, nos. 1-2, pp. 311-331, Nov. 1995.
[18] B. Melamed and M. Yadin, Randomization Procedures in the Computation of Cumulative-Time Distributions over Discrete State Markov Processes Operations Research, vol. 32, pp. 926-944, 1984.
[19] B. Melamed and M. Yadin, Numerical Computation of Sojourn-Time Distributions in Queuing Networks J. ACM, vol. 31, no. 4, pp. 839-854, Oct. 1984.
[20] D.R. Miller, Reliability Calculation Using Randomization for Markovian Fault-Tolerant Computing Systems Proc. 13th IEEE Int'l Symp. Fault-Tolerant Computing (FTCS-13), pp 284-289, June 1983.
[21] A.P. Moorsel and W.H. Sanders, Adaptive Uniformization Comm. in Statistics Stochastic Models, vol. 10, no. 3, pp. 619-647, 1994.
[22] A.P.A. van Moorsel and W.H. Sanders, Transient Solution of Markov Models by Combining Adaptive&Standard Uniformization IEEE Trans. Reliability, vol. 46, no. 3, pp. 430-440, Sept. 1997.
[23] A. Reibman and K.S. Trivedi, Numerical Transient Analysis of Markov Models Computers and Operations Research, vol. 15, pp. 19-36, 1988.
[24] A. Reibman and K. Trivedi, Transient Analysis of Cumulative Measures of Markov Model Behavior Comm. in Statistics Stochastic Models, vol. 5, no. 4, pp. 683-710, 1989.
[25] S.M. Ross, Stochastic Processes. John Wiley&Sons, 1983
[26] W.H. Sanders, W.D. Obal II, M.A. Qureshi, and F.K. Widjanarko, The UltraSAN Modeling Environment Performance Evaluation, vol. 24, nos. 1-2, pp. 89-115, 1995.
[27] B. Sericola, Availability Analysis of Repairable Computer Systems and Stationarity Detection IEEE Trans. Computers, vol. 48, no. 11, pp. 1166-1172, Nov. 1999.

Index Terms:
Rewarded continuous-time Markov chains, transient analysis, randomization, quasistationary distribution.
Citation:
Juan A. Carrasco, "Transient Analysis of Some Rewarded Markov Models Using Randomization with Quasistationarity Detection," IEEE Transactions on Computers, vol. 53, no. 9, pp. 1106-1120, Sept. 2004, doi:10.1109/TC.2004.68
Usage of this product signifies your acceptance of the Terms of Use.