This Article 
 Bibliographic References 
 Add to: 
Measuring and Evaluating Maintenance Process Using Reliability, Risk, and Test Metrics
November/December 1999 (vol. 25 no. 6)
pp. 769-781

Abstract—In analyzing the stability of a maintenance process, it is important that it not be treated in isolation from the reliability and risk of deploying the software that result from applying the process. Furthermore, we need to consider the efficiency of the test effort that is a part of the process and a determinate of reliability and risk of deployment. The relationship between product quality and process capability and maturity has been recognized as a major issue in software engineering based on the premise that improvements in process will lead to higher quality products. To this end, we have been investigating an important facet of process capability—stability—as defined and evaluated by trend, change, and shape metrics, across releases and within a release. Our integration of product and process measurement serves the dual purpose of using metrics to assess and predict reliability and risk and to evaluate process stability. We use the NASA Space Shuttle flight software to illustrate our approach.

[1] “Recommended Practice for Software Reliability, R-013-1992,” Am. Nat'l Standards Inst./Am. Inst. of Aeronautics and Astronautics, 1993.
[2] C. Billings et al., “Journey to a Mature Software Process,” IBM Systems J., vol. 33, no. 1, pp. 46-61, 1994.
[3] L.C. Briand, V.R. Basili, and Y.-M. Kim, “Change Analysis Process to Characterize Software Maintenance Projects,” Proc. Int'l Conf. Software Maintenance, pp. 38-49, Victoria, B.C., Canada, Sept. 1994.
[4] W.H. Farr and O.D. Smith, “Statistical Modeling and Estimation of Reliability Functions for Software (SMERFS) Users Guide,” NAVSWC TR-84-373, rev. 3, Naval Surface Weapons Center, Sept. 1993.
[5] D. Gefen and S.L. Schneberger, “The Non-Homogeneous Maintenance Periods: A Case Study of Software Modifications,” Proc. Int'l Conf. Software Maintenance, pp. 134-141, Monterey, Calif., Nov. 1996.
[6] J. Henry, S. Henry, D. Kafura, and L. Matheson, “Improving Software Maintenance at Martin Marietta,” IEEE Software, vol. 11, no. 4, pp. 67-75, July 1994.
[7] C. Hollenbach et al., “Combining Quality and Software Improvement,” Comm. ACM, vol. 40, no. 6, pp. 41-45, June 1997.
[8] Private Communication with Ted Keller, IBM, Apr. 1998.
[9] T. Keller, N.F. Schneidewind, and P.A. Thornton, “Predictions for Increasing Confidence in the Reliability of the Space Shuttle Flight Software,” Proc. AIAA Computing in Aerospace 10, pp. 1-8, San Antonio, Tex., Mar. 1995.
[10] T.M. Khoshgoftaar et al., "Detection of Fault-Prone Software Modules During a Spiral Life Cycle," Proc. Int'l Conf. Software Maintenance, IEEE CS Press, Los Alamitos, Calif., 1996, pp. 69-76.
[11] M.M. Lehman, “Programs, Life Cycles, and Laws of Software Evolution,” Proc. IEEE, vol. 68, no. 9, Sept. 1980.
[12] “Software Release Schedules,” Lockheed Martin, Jan. 1998.
[13] T. Pearse and P. Oman, Maintainability Measurements on Industrial Source Code Maintenance Activities Proc. IEEE Int'l Conf. Software Maintenance, pp. 295-303, Oct. 1995.
[14] T.M. Pigoski and L.E. Nelson, Software Maintenance Metrics: A Case Study Proc. IEEE Int'l Conf. Software Maintenance, pp. 392-401, 1994.
[15] N.F. Schneidewind, “Measuring and Evaluating Maintenance Process Using Reliability, Risk, and Test Metrics,” Proc. Int'l Conf. Software Maintenance, pp. 232-239, Bari, Italy, Oct. 1997.
[16] N.F. Schneidewind, “Reliability Modeling for Safety Critical Software,” IEEE Trans. Reliability, vol. 46, no. 1, pp. 88-98, Mar. 1997.
[17] N.F. Schneidewind, “Software Reliability Model with Optimal Selection of Failure Data,” IEEE Trans. Software Eng., vol. 19, no. 11, pp. 1,095-1,104, Nov. 1993.
[18] N.F. Schneidewind and T.W. Keller, “Application of Reliability Models to the Space Shuttle,” IEEE Software, vol. 9, no. 4, pp. 28-33, July 1992.
[19] H. Sneed, “Modelling the Maintenance Process at Zurich Life Insurance,” Proc. Int'l Conf. Software Maintenance, pp. 217-226, Monterey, Calif., Nov. 1996.
[20] G.E. Stark, Measurements for Managing Software Maintenance Proc. IEEE Int'l Conf. Software Maintenance, pp. 152-161 1996.

Index Terms:
Maintenance process stability, product and process integration, reliability risk.
Norman F. Schneidewind, "Measuring and Evaluating Maintenance Process Using Reliability, Risk, and Test Metrics," IEEE Transactions on Software Engineering, vol. 25, no. 6, pp. 769-781, Nov.-Dec. 1999, doi:10.1109/32.824387
Usage of this product signifies your acceptance of the Terms of Use.