This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Systems-Theoretic Approach to Safety in Software-Intensive Systems
January-March 2004 (vol. 1 no. 1)
pp. 66-86
Traditional accident models were devised to explain losses caused by failures of physical devices in relatively simple systems. They are less useful for explaining accidents in software-intensive systems and for nontechnical aspects of safety such as organizational culture and human decision-making. This paper describes how systems theory can be used to form new accident models that better explain system accidents (accidents arising from the interactions among components rather than individual component failure), software-related accidents, and the role of human decision-making. Such models consider the social and technical aspects of systems as one integrated process and may be useful for other emergent system properties such as security. The loss of a Milstar satellite being launched by a Titan/Centaur launch vehicle is used as an illustration of the approach.

[1] R.L. Ackoff, Towards a System of Systems Concepts Management Science, vol. 17, no. 11, pp. 661-671, July 1971.
[2] W.R. Ashby, An Introduction to Cybernetics. London: Chapman and Hall, 1956.
[3] L. Bertalanffy, General Systems Theory: Foundations, Development, and Applications, G. Braziller, ed. New York, 1969.
[4] P. Checkland, Systems Thinking, Systems Practice New York: John Wiley&Sons, 1981.
[5] R.C. Conant and W.R. Ashby, Every Good Regulator of a System Must Be a Model of that System Int'l J. System Science, vol. 1, pp. 89-97, 1970.
[6] I. Muniz de Almeida and C.W. Johnson, Extending the Borders of Accident Investigation: Applying Novel Analysis Techniques to the Loss of the Brazilian Space Programme's Launch Vehicle VLS-1 V03 submitted for publication.
[7] N. Dulac and N. Leveson, An Approach to Design for Safety in Complex Systems Proc. Int'l Symp. Systems Eng. (INCOSE), June 2004.
[8] Columbia Accident Investigation Report H. Gehman, chair, Aug. 2003.
[9] Institute of Electrical and Electronics Engineers, IEEE Standard Computer Dictionary. New York, 1990.
[10] C.W. Johnson and C.M. Holloway, The ESA/SOHO Mission Interruption: Using STAMP Accident Analysis Technique for a Software Related Mishap Software Practice and Experience, vol. 33, pp. 1117-1198, 2003.
[11] J. Leplat, Occupational Accident Research and Systems Approach New Technology and Human Error, J. Rasmussen, K. Duncan, and J. Leplat, eds., pp. 181-191, New York: John Wiley&Sons, 1987.
[12] N.G. Leveson, Safeware: System Safety and Computers. Addison-Wesley, 1995.
[13] N.G. Leveson, A New Accident Model for Engineering Safer Systems Safety Science, vol. 42, no. 4,Elsevier, Apr. 2004.
[14] N.G. Leveson, The Role of Software in Spacecraft Accidents AIAA J. Spacecraft and Rockets, July 2004.
[15] N.G. Leveson, A New Approach to System Safety Engineering Manuscript in preparation, draft can be viewed athttp://sunnyday.mit.edubook2.pdf, 2004.
[16] N. Leveson, M. Daouk, N. Dulac, and K. Marais, Applying STAMP in Accident Analysis Proc. Second Workshop Investigation and Reporting of Accidents, Sept. 2003.
[17] J.L. Lions, Ariane 501 Failure: Report by the Inquiry Board European Space Agency, 19 July 1996.
[18] R.F. MilesJr., Introduction Systems Concepts: Lectures on Contemporary Approaches to Systems, Ralph F. Miles Jr., ed., pp. 1-12, New York: John F. Wiley&Sons, 1973.
[19] NASA/ESA Investigation Board. SOHO Mission Interruption. NASA, 31 Aug. 1998.
[20] J.G. Pavlovich, Formal Report of Investigation of the 30 April 1999 Titan IV B/Centaur TC-14/Milstar-3 (B-32) Space Launch Mishap US Air Force, 1999.
[21] C. Perrow, Normal Accidents. Basic Books, 1984 (republished by Princeton University Press, 1999).
[22] J.L. Piper, Chain of Events: The Government Cover-Up of the Black Hawk Incident and the Friendly Fire Death of Lt. Laura Piper. Brasseys Inc., 2001.
[23] S. Ramo, The Systems Approach Systems Concepts: Lectures on Contemporary Approaches to Systems, Ralph F. Miles, Jr., ed., pp. 13-32, New York: John F. Wiley&Sons, 1973.
[24] J. Rasmussen, Risk Management in a Dynamic Society: A Modelling Problem Safety Science, vol. 27, nos. 2/3, pp. 183-213, Elsevier Science Ltd., 1997.
[25] J. Rasmussen and I. Svedung, Proactive Risk Management in a Dynamic Society. Swedish Rescue Services Agency, 2000.
[26] J. Rasmussen, Human Error and the Problem of Causality in Analysis of Accidents Human Factors in Hazardous Situations, D.E. Broadbent, J. Reason, and A. Baddeley, eds., pp. 1-12, Oxford: Clarendon Press, 1990.
[27] J. Rasmussen, A.M. Pejtersen, and L.P. Goodstein, Cognitive System Engineering. John Wiley&Sons, 1994.
[28] W.P. Rogers, Report of the Presidential Commission on the Space Shuttle Challenger Accident US Government Accounting Office, Washington, D.C., 1986.
[29] US Government Accounting Office, Office of Special Investigations, Operation Provide Comfort: Review of Air Force Investigation of Black Hawk Fratricide Incident (GAO/T-OSI-98-13) US Government Printing Office, Washington, D.C., 1997.
[30] G. Weinberg, An Introduction to General Systems Thinking New York: John Wiley&Sons, 1975.
[31] N. Weiner, Cybernetics: or the Control and Communication in the Animal and the Machine. second ed. The MIT Press, 1965.
[32] D.D. Woods, Lessons from Beyond Human Error: Designing for Resilience in the Face of Change and Surprise Proc. Design for Safety Workshop, NASA Ames Research Center, 2000.

Index Terms:
Software safety, system safety, accident models, software engineering.
Citation:
Nancy G. Leveson, "A Systems-Theoretic Approach to Safety in Software-Intensive Systems," IEEE Transactions on Dependable and Secure Computing, vol. 1, no. 1, pp. 66-86, Jan.-March 2004, doi:10.1109/TDSC.2004.1
Usage of this product signifies your acceptance of the Terms of Use.