16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05)
Forecasting Field Defect Rates Using a Combined Time-Based and Metrics-Based Approach: A Case Study of OpenBSD
Chicago, Illinois
November 08-November 11
ISBN: 0-7695-2482-6
Open source software systems are critical infrastructure for many applications; however, little has been precisely measured about their quality. Forecasting the field defect-occurrence rate over the entire lifespan of a release before deployment for open source software systems may enable informed decision-making. In this paper, we present an empirical case study of ten releases of OpenBSD. We use the novel approach of predicting model parameters of software reliability growth models (SRGMs) using metrics-based modeling methods. We consider three SRGMs, seven metrics-based prediction methods, and two different sets of predictors. Our results show that accurate field defect-occurrence rate forecasts are possible for OpenBSD, as measured by the Theil forecasting statistic. We identify the SRGM that produces the most accurate forecasts and subjectively determine the preferred metrics-based prediction method and set of predictors. Our findings are steps towards managing the risks associated with field defects.
Index Terms:
Management, Measurement, Reliability, Experimentation, Metrics-based modeling, deployment and usage metrics, software and hardware configurations metrics, comparative study, open source software
Citation:
Paul Luo Li, Jim Herbsleb, Mary Shaw, "Forecasting Field Defect Rates Using a Combined Time-Based and Metrics-Based Approach: A Case Study of OpenBSD," issre, pp.193-202, 16th IEEE International Symposium on Software Reliability Engineering (ISSRE'05), 2005