Issue No. 12 - December (2000 vol. 26)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/32.888629
<p><b>Abstract</b>—The software engineering literature contains many studies of the efficacy of fault finding techniques. Few of these, however, consider what happens when several different techniques are used together. We show that the effectiveness of such multitechnique approaches depends upon quite subtle interplay between their individual efficacies and <it>dependence</it> between them. The modeling tool we use to study this problem is closely related to earlier work on software <it>design diversity</it>. The earliest of these results showed that, under quite plausible assumptions, it would be unreasonable even to expect software versions that were developed “truly independently” to fail independently of one another. The key idea here was a “difficulty function” over the input space. Later work extended these ideas to introduce a notion of “forced” diversity, in which it became possible to obtain system failure behavior better even than could be expected if the versions failed independently. In this paper, we show that many of these results for <it>design</it> diversity have counterparts in diverse <it>fault detection</it> in a single software version. We define measures of fault finding effectiveness and of diversity and show how these might be used to give guidance for the optimal application of different fault finding procedures to a particular program. We show that the effects upon reliability of repeated applications of a particular fault finding procedure are not statistically independent—in fact, such an incorrect assumption of independence will always give results that are too optimistic. For <it>diverse</it> fault finding procedures, on the other hand, things are different: Here it is possible for effectiveness to be even greater than it would be under an assumption of statistical independence. We show that diversity of fault finding procedures is, in a precisely defined way, “a good thing” and should be applied as widely as possible. The new model and its results are illustrated using some data from an experimental investigation into diverse fault finding on a railway signalling application.</p>
Software fault, fault removal, software reliability, testing, reliability growth, diversity.
Peter T. Popov, Nick Shryane, Bev Littlewood, Lorenzo Strigini, "Modeling the Effects of Combining Diverse Software Fault Detection Techniques", IEEE Transactions on Software Engineering, vol. 26, no. , pp. 1157-1167, December 2000, doi:10.1109/32.888629