Third International Software Metrics Symposium (METRICS'96)
An empirical study of the correlation between code coverage and reliability estimation
Berlin, GERMANY
March 25-March 26
ISBN: 0-8186-7364-8
Mei-Hwa Chen, Dept. of Comput. Sci., State Univ. of New York, Albany, NY, USA
M.R. Lyu, Dept. of Comput. Sci., State Univ. of New York, Albany, NY, USA
W.E. Wong, Dept. of Comput. Sci., State Univ. of New York, Albany, NY, USA
Existing time-domain models for software reliability often result in an overestimation, of such reliability because they do not take the nature of testing techniques into account. Since every testing technique has a limit to its ability to reveal faults in a given system, as a technique approaches its saturation region fewer faults are discovered and reliability growth phenomena are predicted from the models. When the software is turned over to field operation, significant overestimates of reliability are observed. We present a technique to solve this problem by addressing both time and coverage measures for the prediction of software failures. Our technique uses coverage information collected during testing to extract only effective data from a given operational profile. Execution time between test cases which neither increase coverage nor cause a failure as reduced by a parameterized factor. Experiments using this technique were conducted on a program created in a simulation environment with simulated faults and on an industrial automatic flight control project which contained several natural faults. Results from both experiments indicate that overestimation of reliability is reduced significantly using our technique. This new approach not only helps reliability growth models make more accurate predictions, but also reveals the efficiency of a testing profile so that more effective testing techniques can be conducted.
Index Terms:
software reliability; program testing; software performance evaluation; digital simulation; aircraft control; code coverage; software reliability estimation; time-domain models; software testing technique; software reliability growth; coverage measures; time measures; software failure prediction; software execution time; simulation environment; industrial automatic flight control
Citation:
Mei-Hwa Chen, M.R. Lyu, W.E. Wong, "An empirical study of the correlation between code coverage and reliability estimation," metrics, pp.133, Third International Software Metrics Symposium (METRICS'96), 1996