This Article 
 Bibliographic References 
 Add to: 
Uncertainty Analysis in Software Reliability Modeling by Bayesian Analysis with Maximum-Entropy Principle
November 2007 (vol. 33 no. 11)
pp. 781-795
In software reliability modeling, the parameters of the model are typically estimated from the test data of the corresponding component. However, the widely used point estimators are subject to random variations in the data, resulting in uncertainties in these estimated parameters. For large complex systems made up of many components, the uncertainty of each individual parameter amplifies the uncertainty of the total system reliability. Ignoring the parameter uncertainty can result in grossly underestimating the uncertainty in the total system reliability. This paper attempts to study and quantify the uncertainties in the software reliability modeling of a single component with correlated parameters and in a large system with numerous components. Previous works on quantifying uncertainties have assumed a sufficient amount of available data. However, a characteristic challenge in software testing and reliability is the lack of available failure data from a single test which often makes modeling difficult. This lack of data poses a bigger challenge in the uncertainty analysis of the software reliability modeling. To overcome this challenge, this paper proposes to utilize experts' opinions and historical data from previous projects to complement the small number of observations to quantify the uncertainties. This is done by combining the Maximum-Entropy Principle (MEP) into the Bayesian approach. This paper further considers the uncertainty analysis at the system level which contains multiple components, each with its respective model/parameter/uncertainty using a Monte Carlo approach. Some examples with different modeling approaches (NHPP, Markov, Graph theory) are illustrated to show the generality and effectiveness of the proposed approach. Furthermore, we illustrate how the proposed approach for considering the uncertainties in various components improves a large-scale system reliability model proposed in Dai & Levitin (2006) by relaxing a critical assumption.

[1] A.E. Abbas, “Entropy Methods for Joint Distributions in Decision Analysis,” IEEE Trans. Eng. Management, vol. 53, no. 1, pp. 146-159, 2006.
[2] T. Adams, “Total Variance Approach to Software Reliability Estimation,” IEEE Trans. Software Eng., vol. 22, no. 9, pp. 687-688, Sept. 1996.
[3] J. Berger, “The Case for Objective Bayesian Analysis,” Bayesian Analysis, vol. 1, no. 3, pp. 385-402, 2006.
[4] J. Berger, Statistical Decision Theory and Bayesian Analysis. Springer-Verlag, 1985.
[5] A.L. Berger, S.A. Della, and V.J. Della, “A Maximum Entropy Approach to Natural Language Processing,” Computational Linguistics, vol. 22, no. 1, pp. 39-71, 1996.
[6] J. Bernardo and A. Smith, Bayesian Theory. Wiley, 1994.
[7] Y.S. Dai and G. Levitin, “Reliability and Performance of Tree-Structured Grid Services,” IEEE Trans. Reliability, vol. 55, no. 2, pp.337-349, 2006.
[8] Y.S. Dai, M. Xie, K.L. Poh, and G.Q. Liu, “A Study of Service Reliability and Availability for Distributed Systems,” Reliability Eng. and System Safety, vol. 79, pp. 103-112, 2003.
[9] Y.S. Dai, M. Xie, K.L. Poh, and B. Yang, “Optimal Testing-Resource Allocation with Genetic Algorithm for Modular Software Systems,” J. Systems and Software, vol. 66, pp. 47-55, 2003.
[10] M.H. DeGroot and M.J. Schervish, Probability and Statistics. Addison-Wesley, 2002.
[11] G.L. Eyink and S. Kim, “A Maximum Entropy Method for Particle Filtering,” J. Statistical Physics, vol. 123, no. 5, pp. 1071-1128, 2005.
[12] A.L. Goel and K. Okumoto, “Time Dependent Error-Detection Rate Model for Software Reliability and Other Performance Measures,” IEEE Trans. Reliability, vol. 28, pp. 206-211, 1979.
[13] M. Goldstein, “Subjective Bayesian Analysis: Principles and Practice,” Bayesian Analysis, vol. 1, no. 3, pp. 403-420, 2006.
[14] B.R. Haverkort and A.M.H. Meeuwissen, “Sensitivity and Uncertainty Analysis of Markov-Reward Models,” IEEE Trans. Reliability, vol. 44, no. 1, pp. 147-154, 1995.
[15] D.E. Holmes, “Toward a Generalized Bayesian Network,” Proc. Am. Inst. Physics Conf.—Bayesian Inference and Maximum Entropy Methods in Science and Eng., vol. 872, pp. 195-202, 2006.
[16] E.T. Jaynes, “Information Theory and Statistical Mechanics,” Statistical Physics, pp. 181-218, 1963.
[17] Z. Jelinski and P.B. Moranda, “Software Reliability Research,” Statistical Computer Performance Evaluation, W. Freiberger, ed., pp.465-497, Academic Press, 1972.
[18] W.S. Jewell, “Bayesian Extensions to a Basic Model of Software Reliability,” IEEE Trans. Software Eng., vol. 11, no. 12, pp. 1465-1471, Dec. 1985.
[19] J. Kapur, Maximum-Entropy Models in Science and Engineering. John Wiley & Sons, 1989.
[20] S. Kim, F.B. Bastani, I.L. Yen, and I.R. Chen, “Systematic Reliability Analysis of a Class of Application-Specific Embedded Software Framework,” IEEE Trans. Software Eng., vol. 30, no. 4, pp.218-230, 2004.
[21] D. Kurowicka and R. Cooke, Uncertainty Analysis with High Dimensional Dependence Modeling. Wiley, 2006.
[22] M. Masera, “Uncertainty Propagation in Fault Tree Analyses Using Lognormal Distributions,” IEEE Trans. Reliability, vol. 36, no. 1, pp. 145-149, 1987.
[23] K.W. Miller, L.J. Morell, R.E. Noonan, S.K. Park, D.M. Nicol, B.W. Murrill, and J.M. Voas, “Estimating the Probability of Failure When Testing Reveals No Failures,” IEEE Trans. Software Eng., vol. 18, no. 1, pp. 33-43, Jan. 1992.
[24] I. Myrtveit, E. Stensrud, and M. Shepperd, “Reliability and Validity in Comparative Studies of Software Prediction Models,” IEEE Trans. Software Eng., vol. 31, no. 5, pp. 380-391, May 2005.
[25] P.D.T. O'Connor, “Quantifying Uncertainty in Reliability and Safety Studies,” Microelectronics and Reliability, vol. 35, nos. 9-10, pp. 1347-1356, 1995.
[26] H. Pham, Software Reliability. Springer-Verlag, 2000.
[27] C. Robert, The Bayesian Choice: A Decision Theoretic Motivation. Springer-Verlag, 1994.
[28] R.W. Selby, “Enabling Reuse-Based Software Development of Large-Scale Systems,” IEEE Trans. Software Eng., vol. 31, no. 6, pp.495-510, June 2005.
[29] C.E. Shannon, “A Mathematical Theory of Communication,” The Bell System Technical J., vol. 27, pp. 379-423, 1948.
[30] P. Soundappan, E. Nikolaidis, R.T. Haftka, R. Grandhi, and R. Canfield, “Comparison of Evidence Theory and Bayesian Theory for Uncertainty Modeling,” Reliability Eng. and System Safety, vol. 85, nos. 1-3, pp. 295-311, 2004.
[31] C.Y. Tseng, “Entropic Criterion for Model Selection,” Physica A: Statistical and Theoretical Physics, vol. 370, no. 2, pp. 530-538, 2005.
[32] K.S. Trivedi, Probability and Statistics with Reliability, Queuing, and Computer Applications. Prentice-Hall, 1982.
[33] D.A. Wooff, M. Goldstein, and F.P.A. Coolen, “Bayesian Graphical Models for Software Testing,” IEEE Trans. Software Eng., vol. 28, no. 5, pp. 510-525, May 2002.
[34] M. Xie, Y.S. Dai, and K.L. Poh, Computing System Reliability: Models and Analysis. Kluwer Academic, 2004.
[35] M. Xie, G.Y. Hong, and C. Wohlin, “A Study of the Exponential Smoothing Technique in Software Reliability Growth Prediction,” Quality and Reliability Eng. Int'l, vol. 13, no. 6, pp. 347-353, 1997.
[36] B. Yang and M. Xie, “A Study of Operational and Testing Reliability in Software Reliability Analysis,” Reliability Eng. and System Safety, vol. 70, pp. 323-329, 2000.
[37] L. Yin, M.A.J. Smith, and K.S. Trivedi, “Uncertainty Analysis in Reliability Modeling,” Proc. Ann. Reliability and Maintainability Symp., pp. 229-234, 2001.
[38] L. Yin and K.S. Trivedi, “Confidence Interval Estimation of NHPP-Based Software Reliability Models,” Proc. 10th Int'l Symp. Software Reliability Eng., pp. 6-11, 1999.

Index Terms:
Software Reliability, Uncertainty analysis, Bayesian method, Monte Carlo, Markov model, Graph theory
Yuan-Shun Dai, Min Xie, Quan Long, Szu-Hui Ng, "Uncertainty Analysis in Software Reliability Modeling by Bayesian Analysis with Maximum-Entropy Principle," IEEE Transactions on Software Engineering, vol. 33, no. 11, pp. 781-795, Nov. 2007, doi:10.1109/TSE.2007.70739
Usage of this product signifies your acceptance of the Terms of Use.