This Article 
 Bibliographic References 
 Add to: 
A Methodology for Architecture-Level Reliability Risk Analysis
June 2002 (vol. 28 no. 6)
pp. 529-547

Abstract—Risk assessment is an essential process of every software risk management plan. Several risk assessment techniques are based on the subjective judgement of domain experts. Subjective risk assessment techniques are human intensive and error-prone. Risk assessment should be based on product attributes that we can quantitatively measure using product metrics. This paper presents a methodology for reliability risk assessment at the early stages of the development lifecycle, namely, the architecture level. We describe a heuristic risk assessment methodology that is based on dynamic metrics. The methodology uses dynamic complexity and dynamic coupling metrics to define complexity factors for the architecture elements (components and connectors). Severity analysis is performed using Failure Mode and Effect Analysis (FMEA) as applied to architecture models. We combine severity and complexity factors to develop heuristic risk factors for the architecture components and connectors. Based on analysis scenarios, we develop a risk assessment model that represents components, connectors, component risk factors, connector risk factors, and probabilities of component interactions. We also develop a risk analysis algorithm that aggregates risk factors of components and connectors to the architectural level. Using the risk aggregation and the risk analysis model, we show how to analyze the overall risk factor of the architecture as the function of the risk factors of its constituting components and connectors. A case study of a pacemaker architecture is used to illustrate the application of the methodology. The methodology is used to identify critical components and connectors and to investigate the sensitivity of the architecture risk factor to changes in the heuristic risk factors of the architecture elements.

[1] R. Allen and D. Garlan, "A Formal Basis for Architectural Connection," ACM Trans. Software Eng. and Methodology, July 1997.
[2] H. Ammar, T. Nikzadeh, and J. Dugan, “A Methodology for Risk Assessment of Functional Specification of Software Systems Using Coherent Petri Nets,” Proc. Fourth Int'l Software Metrics Symp., Metrics '97, pp. 108-117, Nov. 1997.
[3] L. Bass et al., Software Architecture in Practice, Addison Wesley Longman, 1997.
[4] J. Bowles, “The New SEA FMECA Standard,” Proc. 1998 Ann. Reliability and Maintainability Symp., pp. 48-53, Jan. 1998.
[5] S.R. Chidamber and C.F. Kemerer, "A Metrics Suite for Object Oriented Design," IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476-493, 1994.
[6] B. Douglass, Real-Time UML: Developing Efficient Objects for Embedded Systems. Addison-Wesley, 1998.
[7] “Procedures for Performing Failure Mode Effects and Criticality Analysis,” US MIL_STD_1629 Nov. 1974, US MIL_STD_1629A Nov. 1980, US MIL_STD_1629A/Notice 2, Nov. 1984.
[8] D. Harel, “Statecharts: A Visual Approach to Complex Systems,” Science of Computer Programming, Vol. 8, No. 3, pp. 231–274, 1987.
[9] R. Harrison, S. Counsell, and R. Nithi, “Coupling Metrics for Object Oriented Design,” Proc. Fifth Metrics Symp., pp. 150-157, Nov. 1998.
[10] D. Heimann, “Using Complexity Tracking in Software Development,” Proc. 1995 Ann. Reliability and Maintainability Symp., pp. 433-437, 1995.
[11] J.P. Hudepohl et al., "Emerald: Software Metrics and Models on the Desktop," IEEE Software, Vol. 13, No. 5, Sept. 1996, pp. 56-60.
[12] Int'l Telecomm. Union (ITU-T) Recommendation Z.120 (10/96) for Message Sequence Charts (MSC), , 1996.
[13] T. Khoshgoftaar, J. Munson, and D. Lanning, “Dynamic System Complexity,” Proc. Int'l Software Metrics Symp., Metrics '93, pp. 129-140, May 1993.
[14] T. Khoshgoftaar and J. Munson, “Predicting Software Development Errors Using Software Complexity Metrics,” Proc. 1995 Software Reliability and Testing, pp. 20-28, 1995.
[15] H. Kumamoto and E. Henley, Probabilistic Risk Assessment for Engineers and Scientists, second ed. IEEE Press, 1996.
[16] M. Lyu, J. Yu, E. Keramidas, and S. Dalal, “Armor: Analyzer for Reducing Module Operational Risk,” Proc. 25th Int'l Symp. Fault-Tolerant Computing, pp. 137-142, June 1995.
[17] N. Medvidovic and R.N. Taylor, "A Classification and Comparison Framework for Software Architecture Description Languages," to be published in IEEE Trans. Software Eng.
[18] J.C. Munson and T.M. Khoshgoftaar, “Software Metrics for Reliability Assessment,” Handbook of Software Reliability Eng., M.R. Lyu ed., chapter 12, 1996.
[19] J. Musa, G. Fuoco, N. Irving, D. Kropfl, and B. Julin, “The Operational Profile,” Handbook of Software Reliability Eng., M.R. Lye ed., chapter 5, pp. 167-216, 1996.
[20] “Software Safety,” Nasa Technical Standard. NASA-STD-8719.13A, Sept. 1997, .
[21] ObjecTime User Guide. Kanata, Ontario, Canada: ObjecTime Ltd., 1998.
[22] K. Rauscher and D. Rak, “Gray Box Risk Assessment for Telecomm. Software Modifications,” Proc. IEEE Int'l Conf. Comm., vol. 1, pp. 208-213, June 1996.
[23] Risk Management Procedures and Guidelines, NASA's Procedures and Guidelines, JSC/, NPG 8705-DRAFT 1-6 Sepol EDITION.DOC, Sept. 2001.
[24] Rational Rose Real-Time,, 2002.
[25] “Potential Failure Mode and Effect Analysis in Design (Design FEMCA) and Potential Failure Mode and Effects Analysis in Manufacturing and Assembly Process (Process FEMCA) Reference Manual,” Soc. Automotive Eng., Surface Vehicle Recommended Practice, vol. J1739, July 1994.
[26] “Recommended best practices for Failure Mode, Effects, and Criticality Analysis Procedures,” Soc. Automotive Eng., Aerospace Recommended Practice, 1999.
[27] B. Selic, G. Gullekson, and P. Ward, Real-Time Object-Oriented Modeling, John Wiley and Sons, New York, 1994.
[28] M. Shaw and D. Garlan, Software Architecture: Perspectives on an Emerging Discipline. Prentice Hall, 1996.
[29] The Unified Modeling Language Resource Center,http://www.omg.orgUNL, 2002.
[30] S. Vestal, “A Cursory Overview and Comparison of Four Architecture Description Languages,” technical report, Honeywell Technology Center, Feb. 1993.
[31] K. Weidenhaupt, K. Pohl, M. Jarke, and P. Haumer, "Scenarios in System Development: Current Practice," IEEE Software, pp. 34-45, Mar./Apr. 1998.
[32] S. Yacoub and H. Ammar, “A Matrix-Based Approach to Measure Coupling in Object-Oriented Designs,” J. Object Oriented Programming, JOOP, Oct. 2000.
[33] S.M. Yacoub, B. Cukic, and H.H. Ammar, “Scenario-Based Reliability Analysis of Component-Based Software,” Proc. Int'l Symp. Software Reliability Eng. (ISSRE '99), Nov. 1999.
[34] S. Yacoub, H. Ammar, and T. Robinson, “Dynamic Metrics for Object Oriented Designs,” Proc. Sixth Int'l Symp. Software Metrics, Metrics '99, pp. 50-61, Nov. 1999.
[35] S. Yacoub, B. Cukic, and H. Ammar, “A Component-Based Approach to Reliability Analysis of Distributed Software Systems,” Proc. 18th IEEE Symp. Reliable Distributed Systems, SRDS '99, pp. 158-167, Oct. 1999.
[36] H. Ammar, B. Cukic, C. Fuhrman, and A. Mili, “A Comparative Analysis of Hardware and Software Fault Tolerance: Impact on Software Reliability Engineering,” Annals of Software Eng., vol. 10, Spring 2000.
[37] H.H. Ammar, T. Nikzadeh, and J.B. Dugan, “Risk Assessment of Software Systems Specifications,” IEEE Trans. Reliability, vol. 49, Fall 2000.
[38] L.G. Williams and C.U. Smith, “Performance Engineering of Software Architectures,” Proc. Workshop Software and Performance, Oct. 1998.
[39] R. Kazman and M. Bruth, “Assessing Architectural Complexity,” Proc. Second European Conf. Software Maintenance and Eng., pp. 104-112, Mar. 1998.
[40] R. Kazman, M. Klein, M. Barbacci, H. Lipson, T. Longstaff, and S. Carriere, “The Architecture Tradeoff Analysis Method,” Proc. Fourth Int'l Conf. Eng. of Complex Computer Systems, ICECCS '98, Aug. 1998.

Index Terms:
Reliability risk analysis, risk assessment, risk modeling, component-dependency graphs, software architecture, severity measures, and dynamic metrics.
Sherif M. Yacoub, Hany H. Ammar, "A Methodology for Architecture-Level Reliability Risk Analysis," IEEE Transactions on Software Engineering, vol. 28, no. 6, pp. 529-547, June 2002, doi:10.1109/TSE.2002.1010058
Usage of this product signifies your acceptance of the Terms of Use.