This Article 
 Bibliographic References 
 Add to: 
Integrating Time Domain and Input Domain Analyses of Software Reliability Using Tree-Based Models
December 1995 (vol. 21 no. 12)
pp. 945-958
This paper examines two existing approaches to software reliability analysis, time domain reliability growth modeling and input domain reliability analysis, and presents a new approach that combines some of their individual strengths. An analysis method called tree-based modeling is used to build models based on the combined measurement data. This new approach can be used to assess the reliability of software systems, to track reliability change over time, and to identify problematic subparts characterized by certain input states or time periods. The results can also be used to guide various remedial actions aimed at reliability improvement. This approach has been demonstrated to be applicable and effective in the testing of several large commercial software systems developed in the IBM Software Solutions Toronto Laboratory.

[1] V.R. Basili, G. Caldiera, and G. Cantone, "A Reference Architecture for the Component Factory", ACM Trans. Software Eng. and Methodology, vol. 1, no. 1, pp. 53-80, Jan. 1992.
[2] V.R. Basili,M.V. Zelkowitz,F.E. McGarry,, and R.W. Reiter,“The software engineering laboratory,” Technical Report SEL-77-001, Software Engineering Laboratory, NASA/GSFC, Greenbelt, Md, 1977.
[3] B. Boehm, Software Engineering Economics, Prentice Hall, Upper Saddle River, N.J., 1981, pp. 533-535.
[4] L.C. Briand,V.R. Basili,, and C.J. Hetmanski,“Developing interpretable models with optimized set reduction for identifying high-risk software components,” IEEE Transactions on Software Engineering, vol. 19, no. 11, pp. 1,028-1,044, Nov. 1993.
[5] J.R. Brown and M. Lipow,“Testing for software reliability,” Proc. Int’l Conf. Reliable Software, pp. 518-527,Los Angeles, Calif, Apr. 1975.
[6] R. Chillarege et al., "Orthogonal Defect Classification: A Concept for In-Process Measurements," IEEE Trans. Software Eng., Vol. 18, No. 11, Nov. 1992, pp. 943-956.
[7] L.A. Clark and D. Pregibon,“Tree based models,” J.M. Chambers and T.J. Hastie, eds., Statistical Models in S, ch. 9, pp. 377-419. Wadsworth&Brooks/Cole, 1992.
[8] W.S. Cleveland, Visualizing Data. Summit, N.J.: Hobart Press, 1993.
[9] W.J. Farr and O.D. Smith,“Statistical modeling and estimation of reliability functions for software(SMERFS) users guide,” Tech. Report NSWC TR 84-373, Rev. 2, Naval Surface Warfare Center, Mar. 1991.
[10] A.L. Goel,“Software reliability models: Assumptions, limitations, andapplicability,” IEEE Trans. Software Engineering, vol. 11, no. 12, pp. 1,411-1,423, Dec. 1985.
[11] A.L. Goel and K. Okumoto,“A time dependent error detection rate model for software reliability andother performance measures,” IEEE Trans. Reliability, vol. 28, pp. 206-211, 1979.
[12] Z. Jelinski and P.L. Moranda,“Software reliability research,” W. Freiberger, ed., Statistical Computer Performance Evaluation. Academic Press, 1972, pp. 365-484.
[13] B. Littlewood and J.L. Verrall,“A Bayesian reliability growth model for computer software,” Applied Statistics, vol. 22, pp. 332-346, 1973.
[14] P. Lu and J. Tian,“Applying software reliability engineering in large-scale softwaredevelopment,” Proc. Third Int’l Conf. Software Quality,Lake Tahoe, Nev., pp. 323-330, Oct. 1993.
[15] J.D. Musa,“Software reliability engineering: Determining the operationalprofile,” IEEE Software, Mar. 1993.
[16] J.D. Musa,A. Iannino,, and K. Okumoto,Software Reliability: Measurement, Prediction and Application.New York: McGraw-Hill, 1987.
[17] E. Nelson,“Estimating software reliability from test data,” Microelectronics Reliability, vol. 17, pp. 67-74, 1978.
[18] J. Palma,J. Tian,, and P. Lu,“Collecting data for reliability analysis and modeling,” Proc. CASCON’93, IBM Canada Ltd. and National Research Council of Canada, Toronto, Ontario, Canada, pp. 483-494, Oct. 1993.
[19] A. Porter and R. Selby, "Empirically Guided Software Development Using Metric-Based Classification Trees," IEEE Software, no. 7, pp. 46-54, 1990.
[20] L.H. Putnam,“A general empirical solution to the macro software sizing and estimationproblem,” IEEE Trans. Software Engineering, pp. 345-361, July 1978.
[21] R.W. Selby and A.A. Porter,“Learning from examples: Generation and evaluation of decision trees for software resource analysis,” IEEE Trans. Software Engineering, vol. 14, no. 12, pp. 1,743-1,757, Dec. 1988.
[22] StatSci, S-PLUS Reference Manual, Version 3.2. StatSci, A Division of MathSoft, Inc., Seattle, Wash., Dec. 1993.
[23] R. Thayer,M. Lipow,, and E. Nelson., Software Reliability. North-Holland, 1978.
[24] J. Tian,“An integrated approach to test tracking and analysis,” J. Systems and Software, 1996. (to appear)
[25] J. Tian and J. Henshaw,“Tree-based defect analysis in testing,” Proc. Fourth Int’l Conf. Software Quality,McLean, Va., Oct. 1994.
[26] J. Tian,P. Lu,, and J. Palma,“Test execution based reliability measurement and modeling for largecommercial software,” IEEE Trans. Software Engineering, vol. 21, no. 5, pp. 405-414, May 1995.
[27] J. Troster and J. Tian,“Measurement and defect modeling for a legacy software system,” Annals of Software Engineering, vol. 1, pp. 95-118, Aug. 1995.
[28] M. Tsoukalas, M.J. Duran, and S. Ntafos, “On Some Reliability Estimation Problems in Random and Partition Testing,” IEEE Trans. Software Eng., vol. 19, no. 7, pp. 687-697, July 1993.
[29] E.J. Weyuker and B. Jeng,“Analyzing partition testing strategies,” IEEE Trans. Software Engineering, vol. 17, pp. 703-711, 1991.

Index Terms:
Software reliability, testing, reliability growth models, input state, data partitioning, tree-based modeling.
Jeff Tian, "Integrating Time Domain and Input Domain Analyses of Software Reliability Using Tree-Based Models," IEEE Transactions on Software Engineering, vol. 21, no. 12, pp. 945-958, Dec. 1995, doi:10.1109/32.489071
Usage of this product signifies your acceptance of the Terms of Use.