This Article 
 Bibliographic References 
 Add to: 
Comparison of Physical and Software-Implemented Fault Injection Techniques
September 2003 (vol. 52 no. 9)
pp. 1115-1133
Jean Arlat, IEEE
Emmerich Fuchs, IEEE Computer Society

Abstract—This paper addresses the issue of characterizing the respective impact of fault injection techniques. Three physical techniques and one software-implemented technique that have been used to assess the fault tolerance features of the MARS fault-tolerant distributed real-time system are compared and analyzed. After a short summary of the fault tolerance features of the MARS architecture and especially of the error detection mechanisms that were used to compare the erroneous behaviors induced by the fault injection techniques considered, we describe the common distributed testbed and test scenario implemented to perform a coherent set of fault injection campaigns. The main features of the four fault injection techniques considered are then briefly described and the results obtained are finally presented and discussed. Emphasis is put on the analysis of the specific impact and merit of each injection technique.

[1] J. Arlat et al., "Fault Injection for Dependability Validation: A Methodology and Some Applications," IEEE Trans. Software Eng., Feb. 1990, pp. 166-182.
[2] J. Karlsson, P. Lidén, P. Dahlgren, R. Johansson, and U. Gunneflo, Using Heavy-Ion Radiation to Validate Fault-Handling Mechanisms IEEE Micro, vol. 14, no. 1, pp. 8-23, Feb. 1994.
[3] M. Hsueh, T. Tsai, and R. Iyer, “Fault Injection Techniques and Tools,” Computer, pp. 75–82, Apr. 1997.
[4] J.V. Carreira, D. Costa, and J.G. Silva, Fault Injection Spot-Checks Computer System Dependability IEEE Spectrum, vol. 36, pp. 50-55, Aug. 1999.
[5] J. Arlat, J. Boué, and Y. Crouzet, Validation-Based Development of Dependable Systems IEEE Micro, vol. 19, no. 4, pp. 66-79, July/Aug. 1999.
[6] J. Arlat, Fault Injection for the Experimental Validation of Fault-Tolerant Systems Proc. Workshop Fault-Tolerant Systems, pp. 33-40, 1992.
[7] J.H. Barton, E.W. Czeck, Z.Z. Segall, and D.P. Siewiorek, Fault Injection Experiments Using FIAT IEEE Trans. Computers, vol. 39, no. 4, pp. 575-582, Apr. 1990.
[8] G.A. Kanawati, N.A. Kanawati, and J.A. Abraham, FERRARI: A Flexible Software-Based Fault and Error Injection System IEEE Trans. Computers, vol. 44, no. 2, pp. 248-260, Feb. 1995.
[9] J. Carreira, H. Madeira, and J.G. Silva, Xception: A Technique for the Experimental Evaluation of Dependability in Modern Computers IEEE Trans. Software Eng., vol. 24, no. 2, pp. 125-136, Feb. 1998.
[10] D. Stott, G. Ries, M. Hsueh, and R. Iyer, “Dependability Analysis of a High-Speed Network Using Software-Implemented Fault Injection and Simulated Fault Injection,” IEEE Trans. Computers, vol. 47, no. 1, pp. 108–119, Jan. 1998.
[11] J. Arlat, J.-C. Fabre, M. Rodríguez, and F. Salles, Dependability of COTS Microkernel-Based Systems IEEE Trans. Computers, vol. 51, no. 2, pp. 138-163, Feb. 2002.
[12] H. Madeira, D. Costa, and M. Vieira, On the Emulation of Software Faults by Software Fault Injection Proc. Int'l Conf. Dependable Systems and Networks (DSN-2000), pp. 417-426, 2000.
[13] R. Chillarege and N.S. Bowen, “Understanding Large System Failures—A Fault Injection Experiment,” Proc. IEEE Int'l Symp. Fault-Tolerant Computing, pp. 356–363, June 1989.
[14] M. Daran and P. Thévenod-Fosse, Software Error Analysis: A Real Case Study Involving Real Faults and Mutations Proc. Int'l Symp. Software Testing and Analysis (ISSTA '96), pp. 158-171, 1996.
[15] A. Mukherjee and D.P. Siewiorek, Measuring Software Dependability by Robustness Benchmarking IEEE Trans. Software Eng., vol. 23, no. 6, pp. 366-378, June 1997.
[16] P. Koopman and J. DeVale, Comparing the Robustness of POSIX Operating Systems Proc. 29th Int'l Symp. Fault-Tolerant Computing (FTCS-29), pp. 30-37, 1999.
[17] J. Güthoff and V. Sieh, Combining Software-Implemented and Simulation-Based Fault Injection into a Single Fault Injection Method Proc. 25th Int'l Symp. Fault-Tolerant Computing (FTCS-25), pp. 196-206, 1995.
[18] C.R. Yount and D.P. Siewiorek, A Methodology for the Rapid Injection of Transient Hardware Errors IEEE Trans. Computers, vol. 45, no. 8, pp. 881-891, Aug. 1996.
[19] J. Christmansson, M. Hiller, and M. Rimén, An Experimental Comparison of Fault and Error Injection Proc. Ninth Int'l Symp. Software Reliability Eng., (ISSRE '98), pp. 369-378, 1998.
[20] P. Folkesson, S. Svensson, and J. Karlsson, A Comparison of Simulation Based and Scan Chain Implemented Fault Injection Proc. 28th Int'l Symp. Fault-Tolerant Computing (FTCS-28), pp. 284-293, 1998.
[21] Z. Kalbarczyk, G. Ries, M.S. Lee, Y. Xiao, J. Patel, and R.K. Iyer, Hierarchical Approach to Accurate Fault Modeling for System Evaluation Proc. Int'l Computer Performance and Dependability Symp. (IPDS '98), pp. 249-258, 1998.
[22] C. Constantinescu, Assessing Error Detection Coverage by Simulated Fault Injection Proc. Third European Dependable Computing Conf. (EDCC-3), pp. 161-170, 1999.
[23] J.L. Aidemark, J.P. Vinter, P. Folkesson, and J. Karlsson, GOOFI: A Generic Fault Injection Tool Proc. 2001 Int'l Conf. Dependable Systems and Networks (DSN-2001), pp. 83-88, 2001.
[24] J. Reisinger, A. Steininger, and G. Leber, The PDCS Implementation of MARS Hardware and Software Predictably Dependable Computing Systems, B. Randell, J.-C. Laprie, H. Kopetz and B. Littlewood, eds., pp. 209-224, Berlin: Springer, 1995.
[25] H. Kopetz and G. Bauer, The Time-Triggered Architecture Proc. IEEE, vol. 91, no. 1, pp. 112-126, Jan. 2003.
[26] D. Powell, Failure Mode Assumptions and Assumption Coverage Proc. 22nd Int'l Symp. Fault-Tolerant Computing (FTCS-22), pp. 386-395, 1992.
[27] J. Karlsson, P. Folkesson, J. Arlat, Y. Crouzet, G. Leber, and J. Reisinger, Application of Three Physical Fault Injection Techniques to the Experimental Assessment of the MARS Architecture Dependable Computing for Critical Applications (Proc. Fifth IFIP Working Conf. Dependable Computing for Critical Applications: DCCA-5), R.K. Iyer, M. Morganti, W.K. Fuchs and V. Gligor, eds., pp. 267-287, 1998.
[28] E. Fuchs, An Evaluation of the Error Detection Mechanisms in MARS Using Software-Implemented Fault Injection Proc. Second European Dependable Computing Conf. (EDCC-2), pp. 73-90, 1996.
[29] E. Fuchs, Validating the Fail-Silence of the MARS Architecture Dependable Computing for Critical Applications (Proc. Sixth IFIP Int'l Working Confer. Dependable Computing for Critical Applications: DCCA-6), M. Dal Cin, C. Meadows and W.H. Sanders, eds., pp. 225-247, 1998.
[30] D. Powell, Distributed Fault-Tolerance Lessons from Delta-4 IEEE Micro, vol. 14, no. 1, pp. 36-47, Feb. 1994.
[31] Philips Semiconductors, SCC68070 User Manual 1991, Part 1 Hardware, 1992.
[32] A. Vrchoticky, Modula/R Language Definition Technical Report no. 2/92, Institut für Technische Informatik, Technische Universität Wien, 1992.
[33] H. Kopetz, P. Holzer, G. Leber, and M. Schindler, The Rolling Ball on MARS Research Report no. 13/91, Vienna Univ. of Tech nology, 1991.
[34] S. Poledna, A. Burns, A. Wellings, and P. Barrett, Replica Determinism and Flexible Scheduling in Hard Real-Time Dependable Systems IEEE Trans. Computers, vol. 49, no. 2, pp. 100-111, Feb. 2000.
[35] H. Kopetz, G. Grünsteidl, and J. Reisinger, Fault-Tolerant Membership in a Synchronous Distributed Real-Time System Dependable Computing for Critical Applications, A. Avizienis and J.-C. Laprie, eds., pp. 411-429, Vienna: Springer-Verlag, 1991.
[36] C. Constantinescu, Impact of Deep Submicron Technology on Dependability of VLSI Circuits Proc. Int'l Conf. Dependable Systems and Networks (DSN-2002), pp. 205-209, 2002.
[37] P. Shivakumar et al., "Modeling the Effect of Technology Trends on the Soft Error Rate of Combinatorial Logic," Proc. Int'l Conf. Dependable Systems and Networks, IEEE CS Press, 2000, pp. 389-398.
[38] E. Normand, Single Event Upset at Ground Level IEEE Trans. Nuclear Science, vol. 43, no. 6, pp. 2742-2750, Feb. 1996.
[39] P. Hazucha, Background Radiation and Soft Errors in CMOS Circuits doctoral dissertation, no. 638, Linköping Univ., Sweden, 2000.
[40] R. Johansson, On Single Event Upset Error Manifestation Proc. First European Dependable Computing Conf. (EDCC-1), pp. 217-231, 1994.
[41] C.J. Walter, Evaluation and Design of an Ultra-Reliable Distributed Architecture for Fault Tolerance IEEE Trans. Reliability, vol. 39, no. 4, pp. 492-499, Oct. 1990.
[42] H. Madeira, M. Rela, F. Moreira, and J.G. Silva, RIFLE: A General Purpose Pin-Level Fault Injector Proc. First European Dependable Computing Conf. (EDCC-1), pp. 199-216, 1994.
[43] R.J. Martínez, P.J. Gil, G. Martín, C. Pérez, and J.J. Serrano, Experimental Validation of High-Speed Fault-Tolerant Systems Using Physical Fault Injection Dependable Computing for Critical Applications (Proc. Seventh IFIP Working Conf. Dependable Computing for Critical Applications: DCCA-7), C.B. Weinstock and J. Rushby, eds., pp. 249-265, Jan. 1999.
[44] Y. Crouzet, P. Thévenod-Fosse, and H. Waeselynck, Validation of Software Testing by Fault Injection: The SESAME Tool Proc. 11th Conf. Reliability and Maintainability, pp. 551-559, 1998.
[45] J.M. Voas and G. McGraw, Software Fault Injection. New York: Wiley Computer Publishing, 1998.
[46] E. Fuchs, Sofware Implemented Fault Injection PhD dissertation, Vienna Univ. of Technology, Austria, 1996.
[47] W. Kao, R. Iyer, and D. Tang, "FINE: A Fault Injection and Monitoring Environment for Tracing the UNIX System Behavior Under Faults," IEEE Trans. Software Eng., vol. 19, no. 11, pp. 1,105-1,118, Nov. 1993.
[48] P. Folkesson, Experimental Validation of a Fault-Tolerant System Using Physical Fault Injection Licenciate of Eng. thesis, Chalmers Univ. of Technology, Göteborg, Sweden, 1996.
[49] J. Arlat, A. Costes, Y. Crouzet, J.-C. Laprie, and D. Powell, Fault Injection and Dependability Evaluation of Fault-Tolerant Systems IEEE Trans. Computers, vol. 42, no. 8, pp. 913-923, Aug. 1993.
[50] A. Benso, M. Rebaudengo, I. Impagliazzo, and P. Marmo, Fault-List Collapsing for Fault Injection Experiments Proc. Ann. Reliability&Maintainability Symp. (RAMS '98), pp. 383-388, 1998.
[51] S. Satoh, Y. Tosaka, and S.A. Wender, Geometric Effect of Multiple-Bit Soft Errors Induced by Cosmic Ray Neutrons on DRAM's IEEE Electron Device Letters, vol. 21, no. 6, pp. 310-312, 2000.
[52] H. Kopetz, Time-Triggered Real-Time Computing Proc. IFAC World Congress, 2002, 15-IFAC/data/content/050065006.pdf.

Index Terms:
Fault injection techniques, experimental assessment, fault-tolerant computing, error detection coverage.
Jean Arlat, Yves Crouzet, Johan Karlsson, Peter Folkesson, Emmerich Fuchs, G?nther H. Leber, "Comparison of Physical and Software-Implemented Fault Injection Techniques," IEEE Transactions on Computers, vol. 52, no. 9, pp. 1115-1133, Sept. 2003, doi:10.1109/TC.2003.1228509
Usage of this product signifies your acceptance of the Terms of Use.