This Article 
 Bibliographic References 
 Add to: 
An Experimental Evaluation of Software Redundancy as a Strategy for Improving Reliability
July 1991 (vol. 17 no. 7)
pp. 692-702

The strategy of using multiple versions of independently developed software as a means to tolerate residual software design faults is discussed. The effectiveness of multiversion software is studied by comparing estimates of the failure probabilities of these systems with the failure probabilities of single versions. The estimates are obtained under a model of dependent failures and compared with estimates obtained when failures are assumed to be independent. The experimental results are based on 20 versions of an aerospace application developed and independently validated by 60 programmers from 4 universities. Descriptions of the application and development process are given, together with an analysis of the 20 versions.

[1] A. Avizienis and L. Chen, "On the implementation ofN-version programming for software fault tolerance during program execution," inProc. COMPSAC 77, Chicago, Nov. 1977.
[2] B. Randell, "System structure for software fault tolerance,"IEEE Trans. Software Eng., vol. SE-1, June 1975.
[3] D. E. Eckhardt and L. D. Lee, "A theoretical basis for the analysis of multiversion software subject to coincident errors,"IEEE Trans. Software Eng., vol. SE-11, pp. 1511-1517, Dec. 1985.
[4] B. Littlewood and D. R. Miller, "A conceptual model of multiversion software," inProc. 17th Symp. Fault-Tolerant Comput., June 1987.
[5] J. Knight and N. Leveson, "An experimental evaluation of the assumption of independence in multiversion programming,"IEEE Trans. Software Eng., vol. SE-12, no. 1, pp. 96-109, Jan. 1986.
[6] R. K. Scott, J. W. Gault, D. F. McAllister, and J. Wiggs, "Investigating version dependence in fault-tolerant software," AGARD 361, pp. 21.1-21.10, 1984.
[7] T. J. Shimeall and N. G. Leveson, "An empirical comparison of software fault tolerance and fault elimination," inProc. Second Workshop Software Test. Verification Analysis, Banff, Canada, July 1988.
[8] T. Anderson, P. A. Barret, D. N. Halliwell, and M. R. Moudling, "Software fault tolerance: An evaluation,"IEEE Trans. Software Eng., vol. SE-11, pp. 1502-1510, 1985.
[9] P. G. Bishop, D. G. Esp, M. Barnes, P. Humphreys, G. Dahl, and J. Lahti, "PODS--A project on diverse software,"IEEE Trans. Software Eng., vol. SE-12, no. 9, pp. 929-940, 1986.
[10] P. G. Bishop and F. D. Pullen, "PODS revisited--A study of software failure behavior," inProc. 18th Symp. Fault-Tolerant Comput., June 1988, pp. 2-8.
[11] A. Avizienis, M. Lyu, and W. Schutz, "In search of diversity: A six-language study of fault-tolerant flight control software," inDig. Papers, FTCS-18, Tokyo, Japan, 1988, pp. 15-22.
[12] J. P. J. Kelly and A. Avizienis, "A specification-oriented multiversion software experiment," inProc. 13th Symp. Fault-Tolerant Comput., June 1983.
[13] J. C. Knight and N. G. Leveson, "An empirical study of failure probabilities in multi-version software," inProc. 16th symp. Fault-Tolerant Comput., July 1986, pp. 165-170.
[14] K. S. Tso and A. Avizienis, "Community error recovery inN-version software: A design study with experimentation," inProc. 17th Symp. Fault-Tolerant Comput., July 1987.
[15] T. Anderson and P. A. Lee, Eds.,Fault Tolerance Principles and Practice, Englewood Cliffs, NJ: Prentice-Hall, 1981.
[16] H. Hecht, "Fault-tolerant software for real-time applications,"ACM Comput. Surveys, vol. 8, no. 4, pp. 391-407, Dec. 1976.
[17] A. Grnarov, J. Arlat, and A. Avizienis, "On the performance of software fault tolerance strategies," inProc. 10th Symp. Fault-Tolerant Comput., Oct. 1980, no. 251-253.
[18] J. C. Laprie, "Dependability evaluation of software systems in operation,"IEEE Trans. Software Eng., vol. SE-10, pp. 701-714, Nov. 1984.
[19] R. K. Scott, J. W. Gault and D. F. McAllisier, "Fault tolerant software reliability modeling,"IEEE Trans. Software Eng., vol. SE-13, pp. 582-592, May 1987.
[20] M. A. Fischler, O. Firschein, and D. L. Drew, "Distinct software: An approach to reliable computing, " inProc. 1975 USA--Japan Comput. Conf., pp. 573-579.
[21] A. Avizienis, "Design diversity--The challenge of the eighties," inProc. 12th Symp. Fault-Tolerant Comput., Aug. 1984, pp. 44-45.
[22] Litton Systems, Inc., "Preliminary design of an RSDIMU using two-degree-of-freedom tuned-Gimbal Gyroscopes," NASA CR-145035, Oct. 1976.
[23] L. Lauterbach, "Development ofN-version software samples for an experiment in software fault-tolerance," NASA Contractor Rep. 178363, Sept. 1987.
[24] J. Kelly, D. Eckhardt, M. Vouk, D. McAllister, and A. Caglayan, "A large scale second generation experiment in multi-version software: Description and early results," inDig. Papers, FTC-18, Tokyo, Japan, 1988, pp. 9-14.
[25] D. E. Eckhardt and L. D. Lee, "Fundamental differences in the reliabilitv ofN-modular redundancy andN-version programming,"J. Syst. Software, vol. 8, pp. 313-318, 1988.
[26] P. M. Nagel and J. A. Skrivan, "Software Reliability: Repetitive run experimentation and modeling," NASA CR 165836, NASA Langley Res. Cent., Hampton, VA, Feb. 1982.
[27] S. S. Brilliant, J. C. Knight, and N. G. Leveson, "Analysis of faults in anN-version software experiment,"IEEE Trans. Software Eng., vol. 16, Feb. 1990.

Index Terms:
experimental evaluation; software redundancy; multiple versions; independently developed software; residual software design faults; multiversion software; failure probabilities; dependent failures; experimental results; aerospace application; programmers; development process; fault tolerant computing; program testing; redundancy; software reliability
D.E. Eckhardt, A.K. Caglayan, J.C. Knight, L.D. Lee, D.F. McAllister, M.A. Vouk, J.P.J. Kelly, "An Experimental Evaluation of Software Redundancy as a Strategy for Improving Reliability," IEEE Transactions on Software Engineering, vol. 17, no. 7, pp. 692-702, July 1991, doi:10.1109/32.83905
Usage of this product signifies your acceptance of the Terms of Use.