This Article 
 Bibliographic References 
 Add to: 
An Attack Surface Metric
May/June 2011 (vol. 37 no. 3)
pp. 371-386
Pratyusa K. Manadhata, Symantec Research Labs, Culver City
Jeannette M. Wing, Carnegie Mellon University, Pittsburgh and US National Science Foundation, Arlington
Measurement of software security is a long-standing challenge to the research community. At the same time, practical security metrics and measurements are essential for secure software development. Hence, the need for metrics is more pressing now due to a growing demand for secure software. In this paper, we propose using a software system's attack surface measurement as an indicator of the system's security. We formalize the notion of a system's attack surface and introduce an attack surface metric to measure the attack surface in a systematic manner. Our measurement method is agnostic to a software system's implementation language and is applicable to systems of all sizes; we demonstrate our method by measuring the attack surfaces of small desktop applications and large enterprise systems implemented in C and Java. We conducted three exploratory empirical studies to validate our method. Software developers can mitigate their software's security risk by measuring and reducing their software's attack surfaces. Our attack surface reduction approach complements the software industry's traditional code quality improvement approach for security risk mitigation and is useful in multiple phases of the software development lifecycle. Our collaboration with SAP demonstrates the use of our metric in the software development process.

[1] Toward a Safer and More Secure Cyberspace, S.E. Goodman and H.S. Lin, eds. The Nat'l Academics Press, 2007.
[2] Computing Research Assoc. (CRA), "Four Grand Challenges in Trustworthy Computing," trustworthy.computing.pdf , Nov. 2003.
[3] G. McGraw, "From the Ground Up: The DIMACS Software Security Workshop," IEEE Security and Privacy, vol. 1, no. 2, pp. 59-66, Mar./Apr. 2003.
[4] R.B. Vaughn, R.R. Henning, and A. Siraj, "Information Assurance Measures and Metrics—State of Practice and Proposed Taxonomy," Proc. Hawaii Int'l Conf. System Sciences, 2003.
[5] SAP AG, "SAP—Business Software Solutions Applications and Services," http:/, 2009.
[6] M. Howard, "Fending off Future Attacks by Reducing Attack Surface," library/en-us/dncode/htmlsecure02132003.asp , 2003.
[7] M. Howard, J. Pincus, and J. Wing, "Measuring Relative Attack Surfaces," Proc. Workshop Advanced Developments in Software and Systems Security, 2003.
[8] P.K. Manadhata and J.M. Wing, "Measuring a System's Attack Surface," Technical Report CMU-CS-04-102, Carnegie Mellon Univ., Jan. 2004.
[9] N. Lynch and M. Tuttle, "An Introduction to Input/Output Automata," CWI-Quarterly, vol. 2, no. 3, pp. 219-246, Sept. 1989.
[10] P.K. Manadhata, "An Attack Surface Metric," PhD dissertation, Carnegie Mellon Univ., Dec. 2008.
[11] Y.Y. Haimes, Risk Modeling, Assessment, and Management. Wiley, 2004.
[12] N.E. Fenton and M. Neil, "A Critique of Software Defect Prediction Models," IEEE Trans. Software Eng., vol. 25, no. 5, pp. 675-689, Sept./Oct. 1999.
[13] R. Gopalakrishna, E. Spafford, and J. Vitek, "Vulnerability Likelihood: A Probabilistic Approach to Software Assurance," Technical Report CERIAS TR 2005-06, Purdue Univ., 2005.
[14] D. DaCosta, C. Dahn, S. Mancoridis, and V. Prevelakis, "Characterizing the Security Vulnerability Likelihood of Software Functions," Proc. Int'l Conf. Software Maintenance, 2003.
[15] H. Chen, D. Wagner, and D. Dean, "Setuid Demystified," Proc. 11th USENIX Security Symp., pp. 171-190, 2002.
[16] S. Poznyakoff, "GNU Cflow,", 2009.
[17] B. Kitchenham, S.L. Pfleeger, and N. Fenton, "Towards a Framework for Software Measurement Validation," IEEE Trans. Software Eng., vol. 21, no. 12, pp. 929-944, Dec. 1995.
[18] N. Schneidewind, "Methodology for Validating Software Metrics," IEEE Trans. Software Eng., vol. 18, no. 5, pp. 410-422, May 1992.
[19] E. Weyuker, "Evaluating Software Complexity Measures," IEEE Trans. Software Eng., vol. 14, no. 9, pp. 1357-1365, Sept. 1988.
[20] N.E. Fenton and S.L. Pfleeger, Software Metrics: A Rigorous and Practical Approach, 1998.
[21] L. Briand, K.E. Emam, and S. Morasca, "Theoretical and Empirical Validation of Software Product Measures," TR ISERN-95-03, Fraunhofer Inst. for Experimental Software Eng., 1995.
[22] Handbook of Consumer Psychology, C.P. Haugtvedt, P.M. Herr, and F.R. Kardes, eds. Psychology Press, 2008.
[23] M.Y. Liu and I. Traore, "Properties for Security Measures of Software Products," Applied Math. and Information Science J., vol. 1, no. 2, pp. 129-156, May 2007.
[24] Microsoft Corporation, "Microsoft Security Bulletin Search," current.aspx, 2010.
[25] W.R. Shadish, T.D. Cook, and D.T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin Company, 2001.
[26] Handbook of Survey Research, P. Rossi, J. Wright, and A. Anderson, eds. The Academic Press, 1983.
[27] Y. Miyazaki and K. Mori, "Cocomo Evaluation Tailoring," Proc. Eighth Int'l Conf. Software Eng., pp. 292-299, 1985.
[28] C.F. Kemerer, "An Empirical Validation of Software Cost Estimation Models," Comm. ACM, vol. 30, no. 5, pp. 416-429, 1987.
[29] M.G. Mendonça and V.R. Basili, "Validation of an Approach for Improving Existing Measurement Frameworks," IEEE Trans. Software Eng., vol. 26, no. 6, pp. 484-499, June 2000.
[30] R. Likert, "A Technique for the Measurement of Attitudes," Archives of Psychology, vol. 22, no. 140, pp. 5-55, June 1932.
[31] NIST, "National Vulnerability Database," http:/, 2010.
[32] MITRE, "CWE—Common Weakness Enumeration," http:/cwe., 2010.
[33] S.M. Christey personal communication, 2007.
[34] M. Howard personal communication, 2005.
[35] Microsoft Security Research and Defense, http://blogs.technet. com/srd/archive/2008/ 02/06the-kill_2d00_bit-faq _3a00_ part-1-of-3.aspx , Feb. 2008.
[36] G. Markham, "Reducing Attack Surface," http://weblogs. 2007/02reducing_attack_ surface.html , 2009.
[37] P.K. Manadhata, Y. Karabulut, and J.M. Wing, "Report: Measuring the Attack Surfaces of Enterprise Software," Proc. Int'l Symp. Eng. Secure Software and Systems, 2009.
[38] Eclipse, "Eclipse—An Open Development Platform," http:/, 2010.
[39] M. Sharp, J. Sawin, and A. Rountev, "Building a Whole-Program Type Analysis in Eclipse," Proc. Eclipse Technology Exchange Workshop Object-Oriented Programming, Systems, Languages, and Applications, pp. 6-10, 2005.
[40] Eclipse, "Eclipse Package org.eclipse.jdt.internal.corext.callhierarchy," jdt/ internal/corext/call hierarchy package-summary.html, 2009.
[41] M. Fagan, "Design and Code Inspections to Reduce Errors in Program Development," IBM Systems J., vol. 15, no. 3, pp. 182-211, 1976.
[42] J. Alves-Foss and S. Barbosa, "Assessing Computer Security Vulnerability," ACM SIGOPS Operating Systems Rev., vol. 29, no. 3, pp. 3-13, 1995.
[43] J. Voas, A. Ghosh, G. McGraw, F. Charron, and K. Miller, "Defining an Adaptive Software Security Metric from a Dynamic Software Failure Tolerance Measure," Proc. Ann. Conf. Computer Assurance, 1996.
[44] R. Ortalo, Y. Deswarte, and M. Kaâniche, "Experimenting with Quantitative Evaluation Tools for Monitoring Operational Security," IEEE Trans. Software Eng., vol. 25, no. 5, pp. 633-650, Sept./Oct. 1999.
[45] B. Schneier, "Attack Trees: Modeling Security Threats," Dr. Dobb's J., vol. 24, no. 12, pp. 21-29, 1999.
[46] M.A. McQueen, W.F. Boyer, M.A. Flynn, and G.A. Beitel, "Time-to-Compromise Model for Cyber Risk Reduction Estimation," Proc. ACM Conf. Computer and Comm. Security Workshop Quality of Protection, Sept. 2005.
[47] D.J. Leversage and E.J. Byres, "Estimating a System's Mean Time-to-Compromise," IEEE Security and Privacy, vol. 6, no. 1, pp. 52-60, Jan./Feb. 2008.
[48] D.M. Nicol, "Modeling and Simulation in Security Evaluation," IEEE Security and Privacy, vol. 3, no. 5, pp. 71-74, Sept./Oct. 2005.
[49] B. Littlewood, S. Brocklehurst, N. Fenton, P. Mellor, S. Page, D. Wright, J.D.J. McDermid, and D. Gollman, "Towards Operational Measures of Computer Security," J. Computer Security, vol. 2, no. 2/3, pp. 211-230, 1993.
[50] B.B. Madan, K. Goseva-Popstojanova, K. Vaidyanathan, and K.S. Trivedi, "Modeling and Quantification of Security Attributes of Software Systems," Proc. Int'l Conf. Dependable Systems and Networks, pp. 505-514, 2002.
[51] S.E. Schechter, "Computer Security Strength & Risk: A Quantitative Approach," PhD dissertation, Harvard Univ., 2004.
[52] M. Dacier and Y. Deswarte, "Privilege Graph: An Extension to the Typed Access Matrix Model," Proc. European Symp. Research in Computer Security, pp. 319-334, 1994.
[53] MuSecurity, "What Is a Security Analyzer," http://www. security.html, 2009.

Index Terms:
Code design, life cycle, product metrics, protection mechanisms, risk mitigation, software security.
Pratyusa K. Manadhata, Jeannette M. Wing, "An Attack Surface Metric," IEEE Transactions on Software Engineering, vol. 37, no. 3, pp. 371-386, May-June 2011, doi:10.1109/TSE.2010.60
Usage of this product signifies your acceptance of the Terms of Use.