Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities
Issue No. 06 - November/December (2011 vol. 37)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TSE.2010.81
Jason A. Osborne , North Carolina State University, Raleigh
Yonghee Shin , DePaul University, Chicago
Andrew Meneely , North Carolina State University, Raleigh
Laurie Williams , North Carolina State University, Raleigh
Security inspection and testing require experts in security who think like an attacker. Security experts need to know code locations on which to focus their testing and inspection efforts. Since vulnerabilities are rare occurrences, locating vulnerable code locations can be a challenging task. We investigated whether software metrics obtained from source code and development history are discriminative and predictive of vulnerable code locations. If so, security experts can use this prediction to prioritize security inspection and testing efforts. The metrics we investigated fall into three categories: complexity, code churn, and developer activity metrics. We performed two empirical case studies on large, widely used open-source projects: the Mozilla Firefox web browser and the Red Hat Enterprise Linux kernel. The results indicate that 24 of the 28 metrics collected are discriminative of vulnerabilities for both projects. The models using all three types of metrics together predicted over 80 percent of the known vulnerable files with less than 25 percent false positives for both projects. Compared to a random selection of files for inspection and testing, these models would have reduced the number of files and the number of lines of code to inspect or test by over 71 and 28 percent, respectively, for both projects.
Fault prediction, software metrics, software security, vulnerability prediction.
Jason A. Osborne, Yonghee Shin, Andrew Meneely, Laurie Williams, "Evaluating Complexity, Code Churn, and Developer Activity Metrics as Indicators of Software Vulnerabilities", IEEE Transactions on Software Engineering, vol. 37, no. , pp. 772-787, November/December 2011, doi:10.1109/TSE.2010.81