This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Why Measuring Security Is Hard
July/August 2010 (vol. 8 no. 4)
pp. 46-54
Shari Pfleeger, RAND Corporation , Arlington
Robert Cunningham, MIT Lincoln Laboratory, Lexington
For many years, we've been trying to measure "security" so that we can increase accountability, demonstrate compliance, and determine whether and by how much our investments in products and processes are making our systems more secure. This article investigates why security measurement is difficult and what strategies might help address our needs.

1. A. Avizienis et al., "Basic Concepts and Taxonomy of Dependable and Secure Computing," IEEE Trans. Dependable and Secure Computing, vol. 1, no. 1, 2004, pp. 11–33.
2. S.L. Pfleeger, "Useful Cybersecurity Metrics," IT Professional, July/Aug. 2009, pp. 38–45.
3. S.L. Pfleeger and J.M. Atlee, Software Engineering: Theory and Practice, 4th ed., Prentice Hall, 2009.
4. M.M. Lehman, "Programs, Life Cycles, and Laws of Software Evolution," Proc. IEEE, vol. 68, no. 9, 1980, pp. 1060–1076.
5. R.P. Lippmann et al., "Evaluating Intrusion Detection Systems: The 1998 DARPA Off-Line Intrusion Detection Evaluation," Proc. 2000 DARPA Information Survivability Conf. and Exposition (DISCEX 00), IEEE CS Press, vol. 2, pp. 12–26.
6. E. Tenner, Why Things Bite Back: Technology and the Revenge of Unintended Consequences, Vintage Books, 1996.
7. M. Whitty, Trust and Risk in the Workplace, SurfControl, 2007.
8. B. Lampson, "Practical Principles for Computer Security," Software System Reliability and Security: Proc. 2006 Marktoberdorf Summer School, NATO Science Series, IOS Press, 2006; http://research.microsoft.com/en-us/um/people/ blampson/74-PracticalPrinciplesSecurity 74-PracticalPrinciplesSecurity.pdf .
9. X.H. Chen et al., "Model Checking One Million Lines of C Code," Proc. 11th Ann. Network and Distributed System Security Symp. (NDSS 04), Internet Soc., 2004; www.isoc.org/isoc/conferences/ndss/04/proceedings/ PapersChen.pdf.
10. P. Kocher, "Timing Attacks on Implementations of Diffie-Hellman, RSA, DSS, and Other Systems," Advances in Cryptology—Crypto '96, LNCS 1109, Springer, 1996, pp. 104–113; www.cryptography.com/public/pdfTimingAttacks.pdf .
11. A. Garcia and B. Horowitz, "The Potential for Underinvestment in Internet Security: Implications for Regulatory Policy," J. Regulatory Economics, vol. 31, no. 1, 2007, pp. 37–51; http://ssrn.comabstract=889071.
12. V. Smith, "Economics in the Laboratory," J. Economic Perspectives, vol. 8, no. 1, 1994, pp. 113–131.
13. B. Horowitz and J. Crawford, "Application of Collaborative Risk Analysis to Cyber Security Investment Decisions," Financial Services Technology Consortium Innovation J., vol. 2, no. 1, 2007, pp. 2–5.
14. D. Kahneman and A. Tversky, "Prospect Theory: An Analysis of Decision under Risk," Econometrica, vol. 47, no. 2, 1979, pp. 263–291.
15. D. Watts, "So You Can't Pick the Hits? Neither Can Anyone Else," Washington Post,4 Jan. 2009, p. B04.
16. M. Howard, J. Pincus, and J. Wing, "Measuring Relative Attack Surfaces," Computer Security in the 21st Century, D.T. Lee et al., eds., Springer, 2005, pp. 109–137.
17. A. Ozment, and S. Schechter, "Milk or Wine: Does Software Security Improve with Age?" Proc. 15th Usenix Security Symp., Usenix, 2006, pp. 93–104.

Index Terms:
security and privacy, measurement
Citation:
Shari Pfleeger, Robert Cunningham, "Why Measuring Security Is Hard," IEEE Security & Privacy, vol. 8, no. 4, pp. 46-54, July-Aug. 2010, doi:10.1109/MSP.2010.60
Usage of this product signifies your acceptance of the Terms of Use.