This Article 
 Bibliographic References 
 Add to: 
Recalibrating Software Reliability Models
April 1990 (vol. 16 no. 4)
pp. 458-470

There is no universally applicable software reliability growth model which can be trusted to give accurate predictions of reliability in all circumstances. A technique of analyzing predictive accuracy called the u-plot allows a user to estimate the relationship between the predicted reliability and the true reliability. It is shown how this can be used to improve reliability predictions in a very general way by a process of recalibration. Simulation results show that the technique gives improved reliability predictions in a large proportion of cases. However, a user does not need to trust the efficacy of recalibration, since the new reliability estimates produced by the technique are truly predictive and their accuracy in a particular application can be judged using the earlier methods. The generality of this approach suggests its use whenever a software reliability model is used. Indeed, although this work arose from the need to address the poor performance of software reliability models, it is likely to have applicability in other areas such as reliability growth modeling for hardware.

[1] A. A. Abdel-Ghaly, P. Y. Chan, and B. Littlewood, "Evaluation of competing software reliability predictions,"IEEE Trans. Software Eng., vol. SE-12, no. 9, Sept. 1986.
[2] J. Aitchison and I. R. Dunsmore,Statistical Prediction Analysis. Cambridge, England: Cambridge University Press, 1975.
[3] H. Ascher and H. Feingold,Repairable Systems Reliability(Lecture Notes in Statistics, no. 7). New York: Dekker, 1984.
[4] S. Brocklehurst, "On the effectiveness of adaptive software reliability modelling," Centre for Software Reliability, City Univ., London, Tech. Rep., Oct. 1987.
[5] P. Y. Chan, "Software reliability prediction," Ph.D. dissertation, City Univ., London, 1986.
[6] L. H. Crow, "Confidence interval procedures for reliability growth analysis," U.S. Army Material Systems Analysis Activity, Aberdeen, MD. Tech. Rep. 197, 1977.
[7] A. P. Dawid, "The well-calibrated Bayesian" (with discussion),J. Amer. Statist. Assoc., vol. 77, pp. 605-613, 1982.
[8] A. L. Goel and F. B. Bastani,IEEE Trans. Software Eng. (Special Issue on Software Reliability), vol. SE-11, no. 12, 1985 and vol. SE- 12, no. 1, 1986.
[9] A. L. Goel and K. Okumoto, "Time-dependent error-detection rate model for software reliability and other performance measures,"IEEE Trans. Rel., vol. R-28, pp. 206-211, 1979.
[10] Z. Jelinski and P. B. Moranda, "Software reliability research," inStatistical Computer Performance Evaluation, W. Freiberger, Ed. New York Academic, 1972, pp. 465-484.
[11] H. Joe and N. Reid, "Estimating the number of faults in a system,"J. Amer. Statist. Assoc., vol. 80, pp. 222-226, 1985.
[12] B. Littlewood and A. Sorfer, "A Bayesian modification to the Jelinski-Moranda software reliability model,"Software Eng. J., vol. 2, pp. 30-41, 1987.
[13] B. Littlewood, "Stochastic reliability growth: a model for fault-removal in computer programs and hardware designs,"IEEE Trans. Rel., vol. R-30, no. 4, pp. 313-320, Oct. 1981.
[14] B. Littlewood and J. L. Verrall, "A Bayesian reliability growth model for computer software,"J. Roy. Statist. Soc., C (Applied Statistics), vol. 22, pp, 332-346, 1973.
[15] D. R. Miller, "Exponential order statistic models of software reliability growth," George Washington, Univ., Washington, DC, Tech. Rep. T-496/84, 1984.
[16] J. D. Musaet al., Software Reliability Measurement, Prediction, Application. New York: McGraw-Hill International, 1987.
[17] J. D. Musa and K. Okumoto, "A logarithmic Poisson execution time model for software reliability measurement," inProc. Compsac' 84, Chicago, IL, 1984, pp. 230-238.
[18] J. D. Musa, "Software reliability data," Data Analysis Centre for Software, Rome Air Development Center, Rome, NY, Rep., 1979.
[19] A. F. M. Smith, A. M. Skene, J. E. H. Shaw, and J. C. Naylor, "Progress with numerical and graphical methods for practical Bayesian statistics,"Statistician, vol. 36, pp. 75-82, 1987.

Index Terms:
software reliability models recalibrating; simulation results; predictive accuracy; u-plot; software reliability.
S. Brocklehurst, P.Y. Chan, B. Littlewood, J. Snell, "Recalibrating Software Reliability Models," IEEE Transactions on Software Engineering, vol. 16, no. 4, pp. 458-470, April 1990, doi:10.1109/32.54297
Usage of this product signifies your acceptance of the Terms of Use.