This Article 
 Bibliographic References 
 Add to: 
Test-Execution-Based Reliability Measurement and Modeling for Large Commercial Software
May 1995 (vol. 21 no. 5)
pp. 405-414
This paper studies practical reliability measurement and modeling for large commercial software systems based on test execution data collected during system testing. The application environment and the goals of reliability assessment were analyzed to identify appropriate measurement data. Various reliability growth models were used on failure data normalized by test case executions to track testing progress and provide reliability assessment. Practical problems in data collection, reliability measurement and modeling, and modeling result analysis were also examined. The results demonstrated the feasibility of reliability measurement in a large commercial software development environment and provided a practical comparison of various reliability measurements and models under such an environment.

[1] ANSI/AIAA, Am. Nat’l Standard: Recommended Practice for Software Reliability, No. ANSI/AIAA R-013-1992. Am. Inst. Aeronautics and Astronautics, Feb. 1992.
[2] J.D. Musa,A. Iannino,, and K. Okumoto,Software Reliability: Measurement, Prediction and Application.New York: McGraw-Hill, 1987.
[3] P. Lu and J. Tian,“Applying software reliability engineering in large-scale softwaredevelopment,” Proc. Third Int’l Conf. Software Quality,Lake Tahoe, Nev., pp. 323-330, Oct. 1993.
[4] J. Tian and P. Lu,“Software reliability measurement and modeling for multiple releases ofcommercial software,” Proc. Fourth Int. Symp. Software Reliability Eng., pp. 253-260, Nov. 1993.
[5] P.B. Moranda,“Prediction of software reliability during debugging,” Ann. Reliability and Maintainability Symp.,Washington, DC, pp. 327-332, Jan. 1975.
[6] J.D. Musa,“A theory of software reliability and its application,” IEEE Trans. Software Engineering, vol. 1, pp. 312-327, 1971.
[7] B. Littlewood and J.L. Verrall,“A Bayesian reliability growth model for computer software,” Applied Statistics, vol. 22, pp. 332-346, 1973.
[8] A.L. Goel and K. Okumoto,“A time dependent error detection rate model for software reliability andother performance measures,” IEEE Trans. Reliability, vol. 28, pp. 206-211, 1979.
[9] J.D. Musa and K. Okumoto, "A Logarithmic Poisson Execution Time Model for Software Reliability Measurement," Proc. Seventh Int'l Conf. Software Eng., IEEE CS Press, Los Alamitos, Calif., 1984, pp. 230-238.
[10] W.J. Farr and O.D. Smith,“Statistical modeling and estimation of reliability functions for software(SMERFS) users guide,” Tech. Rep. NSWC TR 84-373, Rev. 2, Naval Surface Warfare Center, Mar. 1991.
[11] J. Tian and P. Lu,“An integrated environment for software reliability modeling,” Proc. 17th Int’l Computer Software and Applications Conf., pp. 395-401, Nov. 1993.
[12] A.L. Goel,“Software reliability models: Assumptions, limitations, and applicability,” IEEE Trans. Software Eng., vol. 11, no. 12, pp. 1,411-1,423, Dec. 1985.
[13] J. Troster and J. Tian,“Measurement and defect modeling for a legacy software system,” Annals of Software Eng., (to appear), 1995.
[14] J. Tian and J. Henshaw,“Tree-based defect analysis in testing,” Proc. Fourth Int’l Conf. Software Quality,McLean, Virginia, Oct. 1994.
[15] B. Ray,I. Bhanari,, and R. Chillaege,“Reliability growth for typed defects,” Proc. IEEE Reliability and Maintainability Symp., 1991.

Index Terms:
Reliability, testing, large commercial software, data collection, measurement.
Jeff Tian, Peng Lu, Joe Palma, "Test-Execution-Based Reliability Measurement and Modeling for Large Commercial Software," IEEE Transactions on Software Engineering, vol. 21, no. 5, pp. 405-414, May 1995, doi:10.1109/32.387470
Usage of this product signifies your acceptance of the Terms of Use.