This Article 
 Bibliographic References 
 Add to: 
Bayesian Graphical Models for Software Testing
May 2002 (vol. 28 no. 5)
pp. 510-525

This paper describes a new approach to the problem of software testing. The approach is based on Bayesian graphical models and presents formal mechanisms for the logical structuring of the software testing problem, the probabilistic and statistical treatment of the uncertainties to be addressed, the test design and analysis process, and the incorporation and implication of test results. Once constructed, the models produced are dynamic representations of the software testing problem. They may be used to drive test design, answer what-if questions, and provide decision support to managers and testers. The models capture the knowledge of the software tester for further use. Experiences of the approach in case studies are briefly discussed.

[1] M. Bouissou, F. Martin, and A. Ourghanlian, “Assessment of a Safety-Critical System Including Software: A Bayesian Belief Network for Evidence Sources,” Proc. Ann. Reliability and Maintainability Symp., RAMS '99, 1999.
[2] A. Burr and M. Owen, Statistical Methods for Software Quality. London: Thomson, 1996.
[3] R.G. Cowell, A.P. Dawid, S.L. Lauritzen, and D.J. Spiegelhalter, Probabilistic Networks and Expert Systems. New York: Springer, 1999.
[4] S.R. Dalal and C.L. Mallows, “Factor-Covering Designs for Testing Software,” Technometrics, vol. 40, pp. 234–243, 1998.
[5] P. Frankl et al., "Evaluating Testing Methods by Delivered Reliability," IEEE Trans. Software Eng., Aug. 1998, pp. 586-602.
[6] S. French and D. Rios Insua, Statistical Decision Theory. London: Ar nold, 2000.
[7] D. Hamlet, "Are we testing for true reliability?" IEEE Software, pp. 21-27, July 1992.
[8] D. Hamlet and R. Taylor, "Partition Testing Does Not Inspire Confidence," IEEE Trans. Software Eng., vol. 16, pp. 1,402-1,412, Dec. 1990.
[9] R.M. Hierons and M.P. Wiper, “Estimation of Failure Rate Using Random and Partition Testing,” Software Testing, Verification, and Reliability vol. 7, pp. 153–164, 1997.
[10] F. Jensen, An Introduction to Bayesian Neworks. Springer Verlag, 1996.
[11] U. Kjærulff and L.C. van der Gaag, “Making Sensitivity Analysis Computationally Efficient,” Proc. 16th Conf. Uncertainty in Artificial Intelligence, 2000.
[12] L. Kuo, “Software Reliability,” Encyclopedia of Statistical Sciences, S. Kotz, ed., update vol. 3, pp. 671–680, 1999.
[13] S.L. Lauritzen, Graphical Models. Oxford: Clarendon, 1996.
[14] M. Neil and N. Fenton, “Predicting Software Quality Using Bayesian Belief Networks,” Proc. 21st Ann. Software Eng. Workshop NASA/Goddard Space Flight Center, 1996.
[15] A. O'Hagan, “Eliciting Expert Beliefs in Substantial Practical Applications,” The Statistician, vol. 47, pp. 21–35, 1998.
[16] J. Pearl, Probabilistic Reasoning in Intelligent Systems. San Mateo, Calif.: Morgan Kaufman, 1988.
[17] F. Redmill, “Why Systems Go Up in Smoke,” The Computer Bulletin, pp. 26–28, Sept. 1999.
[18] K. Rees, F.P.A. Coolen, M. Goldstein, and D.A. Wooff, “Managing the Uncertainties of Software Testing: A Bayesian Approach,” Quality and Reliability Eng. Int'l, vol. 17, pp. 191-203, 2001.
[19] G. Rothermel and M.J. Harrold, “Analyzing Regression Test Selection Techniques,” IEEE Trans. Software Eng., vol. 22, no. 8, pp. 529-551, Aug. 1996.
[20] G. Rothermel and M.J. Harrold, “Empirical Studies of a Safe Regression Test Selection Technique,” IEEE Trans. Software Eng., vol. 24, no. 6, pp. 401-419, June 1998.
[21] N.D. Singpurwalla and S.P. Wilson, “Software Reliability Modeling,” Int'l Statistical Rev., vol. 62, pp. 289–317, 1994.
[22] N.D. Singpurwalla and S.P. Wilson, Statistical Methods in Software Engineering: Reliability and Risk. New York: Springer, 1999.
[23] C. Smidts and D. Sova, “An Architectural Model for Software Reliability Quantification: Sources of Data,” Reliability Eng. and System Safety, vol. 64, pp. 279–290, 1999.
[24] D.J. Spiegelhalter, A.P. Dawid, S.L. Lauritzen, and R.G. Cowell, “Bayesian Analysis in Expert Systems,” Statistical Science, vol. 8, pp. 219–283, 1993.
[25] E.J. Weyuker and B. Jeng,“Analyzing partition testing strategies,” IEEE Trans. Software Engineering, vol. 17, pp. 703-711, 1991.
[26] E.J. Weyuker and T.J. Ostrand, “Theories of Program Testing and the Application of Revealing Subdomains,” IEEE Trans. Software Eng., vol. 6, pp. 236–246, 1980.
[27] T. Yamaura, “How to Design Practical Test Cases,” IEEE Software, pp. 30–36, Nov.-Dec. 1998.

Index Terms:
Bayesian graphical models, expert judgment, knowledge capture, software reliability, software testing, statistical methods, test design
D.A. Wooff, M. Goldstein, F.P.A. Coolen, "Bayesian Graphical Models for Software Testing," IEEE Transactions on Software Engineering, vol. 28, no. 5, pp. 510-525, May 2002, doi:10.1109/TSE.2002.1000453
Usage of this product signifies your acceptance of the Terms of Use.