This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Optimal Test Distributions for Software Failure Cost Estimation
March 1995 (vol. 21 no. 3)
pp. 219-228
In this article, we generalize the input-domain based software reliability measures by Nelson and by Weiss and Weyuker, introducing expected failure costs under the operational distribution as a measure for software unreliability. This approach incorporates in the reliability concept a distinction between different degrees of failure severity. It is shown how to estimate the proposed quantity by means of random testing, using the Importance Sampling technique from Rare Event Simulation. A test input distribution that yields an unbiased estimator with minimum variance is determined. The practical application of the presented method is outlined, and a detailed numerical example is given.

[1] B. Beizer, Software Testing Techniques, second ed. Boston, Mass.: Int'l Thomson Computer Press, 1990.
[2] B. Boehm,“Software risk management,”inProc. ESEC,Warwick, U.K., Sept. 1989, pp. 1–19.
[3] R. W. Butler and G. B. Finelli,“The infeasibility of quantifying the reliability of life-critical software,”IEEE Trans. Software Eng., vol. 19, pp. 3–12, Jan. 1993.
[4] L. Devroye,Non-Uniform Random Variate Generation. New York: Springer, 1986.
[5] W. D. Ehrenberger,“Combining probabilistic and deterministic verification efforts,”inProc. SAFECOMP '92,H. Frey, Ed. Oxford, England: Pergamon, 1992, pp. 299–304.
[6] J. S. Gourlay,“A mathematical framework for the investigation of testing,”IEEE Trans. Software Eng., vol. SE-9, pp. 686–709, Nov. 1983.
[7] W. J. Gutjahr,“Probabilistic software test efficiency measures and optimal test data selection,”in preparation.
[8] R.G. Hamlet, “Probable Correctness Theory,” Information Processing Letters, vol. 25, pp. 17–25, Apr. 1987.
[9] R. G. Hamlet,“Theoretical comparison of testing methods,”inProc. Third Symp. Testing, Anal., Verification, Key West, FL, Dec. 1989, pp. 28–37.
[10] M. H. Kalos and P. A. Whitlock,Monte Carlo Methods. New York: Wiley, 1986, vol. I.
[11] H. A. Linstone and M. Turoff,The Delphi Method. Reading, PA: Addison-Wesley, 1975.
[12] K. W. Milleret al.,“Estimating the probability of failure when testing reveals no failures,”IEEE Trans. Software Eng., vol. 18, pp. 33–43, Jan. 1992.
[13] P. G. Moore,The Business of Risk. Cambridge, England: Cambridge University Press, 1993.
[14] E. C. Nelson,“A statistical basis for software reliability assessment,”TRW-SS-73-03, Mar. 1973.
[15] E. C. Nelson,“Estimating software reliability from test data,”Microelectron., Reliability, vol. 17, pp. 67–74, 1978.
[16] C. V. Ramamoorthy and F. B. Bastani,“Software reliability—Status and perspectives,”IEEE Trans. Software Eng., vol. SE-8, pp. 354–371, July 1982.
[17] R. W. Selby, V. R. Basili, and F. T. Baker,“Cleanroom software development: An empirical investigation,”IEEE Trans. Software Eng., vol. SE-13, pp. 1027–1037, Sept. 1987.
[18] S. A. Sherer,Software Failure Risk.New York: Plenum, 1992.
[19] M. Smithson,Ignorance and Uncertainty. New York: Springer, 1988.
[20] Th. A. Thayer, M. Lipow, and E. C. Nelson,Software Reliability. Amsterdam, The Netherlands: North-Holland, 1978.
[21] S. N. Weiss and E. J. Weyuker,“An extended domain–based model of software reliability,”IEEE Trans. Software Eng., vol. 14, pp. 1512–1524, Oct. 1988.
[22] E. J. Weyuker and T. J. Ostrand,“Theories of program testing and the application of revealing subdomains,”IEEE Trans. Software Eng., vol. SE-6, pp. 236–245, May 1980.
[23] E. J. Weyuker, S. N. Weiss, and D. Hamlet,“Comparison of program testing strategies,”inProc. Symp. Testing, Anal., Verification(Victoria, B.C.), Oct. 1991, pp. 1–10.

Index Terms:
Input distributions, input-domain-based models, reliability estimation, risk assessment, software failure costs, software reliability, software testing
Citation:
Walter J. Gutjahr, "Optimal Test Distributions for Software Failure Cost Estimation," IEEE Transactions on Software Engineering, vol. 21, no. 3, pp. 219-228, March 1995, doi:10.1109/32.372149
Usage of this product signifies your acceptance of the Terms of Use.