This Article 
 Bibliographic References 
 Add to: 
Determining an Optimal Time Interval for Testing and Debugging Software
April 1991 (vol. 17 no. 4)
pp. 313-319

A decision-theoretic procedure for determining an optimal time interval for testing software prior to its release is proposed. The approach is based on the principles of decision-making under uncertainty and involves a maximization of expected utility. Two plausible forms for the utility function, one based on costs and the other involving the realized reliability of the software, are described. Using previous results on probabilistic models for software failure, the ensuing optimization problem (which can be addressed using numerical techniques) is outlined for the case of single-state testing. The sensitivity of the results to the various input parameters is discussed, and some directions for future research are outlined.

[1] G. Andretta and G. M. Kaufman, "Estimation of finite population properties when sampling is without replacement and proportional to magnitude,"J. Amer. Statist. Assoc., vol. 81, no. 395, pp. 657-666, 1986.
[2] L. J. Bain and D. L. Weeks, "A note on the truncated exponential distribution,"Ann. Math. Statist., vol. 35, pp. 1366-1367, 1964.
[3] S. R. Dalal and C. L. Mallows, "When to stop testing software?"J. Amer. Statist. Assoc., vol. 83, no. 403, pp. 872-879, 1986.
[4] B. Efron and R. Thisted, "Estimating the number of unseen species: How many words did Shakespeare know?"Biometrika, vol. 63, pp. 435-47, 1976.
[5] T. S. Ferguson and J. P. Hardwick, "Stopping rules for proofreading,"J. Appl. Prob., vol. 26, pp. 304-313, 1989.
[6] E. H. Forman and N. D. Singpurwalla, "An empirical stopping rule for debugging and testing computer software,"J. Amer. Statist. Assoc., vol. 72, pp. 750-757, 1977.
[7] E. H. Forman and N. D. Singpurwalla, "Optimal time intervals for testing hypotheses on computer software errors,"IEEE Trans. Rel., vol. R-28, pp. 250-253, 1979.
[8] W. S. Humphrey,Managing the Software Process. Reading, MA: Addison-Wesley, 1989.
[9] Z. Jelinski and P. B. Moranda, "Software reliability research," inStatistical Computer Performance Evaluation, W. Freiberger, Ed. New York: Academic, 1972, pp. 485-502.
[10] N. Langberg and N.D. Singpurwalla, "A unification of some software reliability models,"SIAM J. Sci. Statist. Comput., vol. 6, no. 3, pp. 781-790, 1985.
[11] D. V. Lindley,Making Decisions, 2nd ed. New York: Wiley, 1985.
[12] R. J. Meinhold and N. D. Singpurwalla, "Bayesian analysis of a commonly used model for describing software failures,"The Statistician, vol. 32, pp. 163-173, 1983.
[13] J. D. Musa and A. F. Ackerman, "Quantifying software validation: When to stop testing?"IEEE Software, pp. 19-27, May 1989.
[14] G. J. Myers,The Art of Software Testing. New York: Wiley, 1979.
[15] K. Okumoto and A. L. Goel, "Optimum release time for software systems, based on reliability and cost criteria,"J. Syst. Software, vol. 1, pp. 315-318, 1980.
[16] S. M. Ross, "Software reliability: The stopping rule problem,"IEEE Trans. Software Eng., vol. SE-11, no. 12, pp. 1472-1476, 1985.
[17] N. D. Singpurwalla, "Preposterior analysis in software testing," inStatistical Data Analysis and Inference, Y. Dodge, Ed. New York: Elsevier, pp. 581-595.
[18] N. E. Singpurwalla and R. Soyer, "Nonhomogeneous autoregressive process for tracking reliability growth, and their Bayesian analysis,"J. Roy. Statist. Soc., series B, vol. 54, no. 1, to be published, 1992.
[19] S. Yamada, H. Narihisa, and S. Osaki, "Optimum release policies for a software system with a scheduled software delivery time,"Int. J. Syst. Sci., vol. 15, no. 8, pp. 905-914, 1984.

Index Terms:
software testing; software debugging; decision theory; software reliability; optimal time interval; decision-making; uncertainty; maximization; expected utility; utility function; costs; probabilistic models; software failure; optimization problem; single-state testing; decision theory; program debugging; program testing; programming theory
N.D. Singpurwalla, "Determining an Optimal Time Interval for Testing and Debugging Software," IEEE Transactions on Software Engineering, vol. 17, no. 4, pp. 313-319, April 1991, doi:10.1109/32.90431
Usage of this product signifies your acceptance of the Terms of Use.