The Community for Technology Leaders

What Are the Numbers?

Martin R. Stytz, Institute for Defense Analyses

Pages: p. 12

Reading Andrew Jaquith's Security Metrics: Replacing Fear, Uncertainty, and Doubt will inspire you to start identifying and measuring meaningful computer security performance factors and begin the process of transforming our shaman-like discipline into a science. Jaquith, who co-founded @stake and is currently the Yankee Group's lead security analyst, offers a refreshing, erudite, and usable approach to security metrics while being engaging, thought-provoking, and never dull. Above all, the call to action for the development and use of meaningful security metrics is one that every computer security practitioner should heed.

The book prompts readers to rethink what they know about computer security and the measures used to asses it. A continuing theme is to look for new points of view for computer security metrics and to consider why we don't have better measures and what these measures would be. Jaquith's analysis spurs computer security practioners to devise, experiment, assess, and use better measures of computer security than those we have today.

Jaquith emphasizes our need to develop metrics that are far beyond today's state of the art but relevant and understandable by decision-makers and useful for security assessments. He stresses that metrics must measure computer security as well as improve it. Jaquith frequently reminds the reader that the foundation for any discipline's improvement is the ability to measure its key aspects, rate of change, and factors that affect performance. In short, unless we can analyze meaningful measurements, we can't know if security is improving or getting worse. He also sounds a stern warning: unless we can make a business case for computer security spending, the current level of spending will not be sustained.

The book consists of four sections. The first and fourth sections are excellent bookends on security metrics, describing their development, design, and use, as well as metric data acquisition and assessment. These portions of the book describe what metrics are, why they're needed, how to design scoring systems for security assessments, how to collect the required data, and how to use metrics to make assessments. The book's middle sections address topics that are slightly less important, but only because they're fields unto themselves and already well-represented in other books: data analysis and data presentation. The author does an excellent job of introducing these topics and providing heuristics for the use of various analytical and presentation techniques. In particular, the second section presents a discussion of security metrics throughout the enterprise, their use for planning, acquisition, and delivery, the monitoring of security metrics data, and what metrics can tell us about an application's performance.


Security Metrics is a well-rounded, authoritative, and eminently readable book that addresses the need for metrics and for further research in this critical area of computer security. Everyone who must address computer security—practioners, managers, and executives—or those who must address computer security issues—maximizing security, addressing risk, and allocating resources—should read this book. It's a very useful companion to Bruce Schneier's Beyond Fear because it propels us toward measuring and analyzing security performance rather than just reacting to events without judging the effectiveness of our reaction. And like Beyond Fear, Jaquith's Security Metrics should be found in every computer security practitioner's library.

About the Authors

Martin R. Stytz is a research staff member at the Institute for Defense Analyses. His research interests include computer security, intelligent systems and agents, and distributed systems. Stytz has a PhD in computer science and engineering from the University of Michigan. He is a member of the IEEE, the ACM, and the International Society for Photo-Optical Engineering (SPIE). Contact him at
65 ms
(Ver 3.x)