Issue No. 03 - May/June (2011 vol. 37)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TSE.2010.90
Jin Liu , Wuhan University, Wuhan
Shi Ying , Wuhan University, Wuhan
Zihan Jia , Xi'an Jiaotong University, Xi'an
Qinbao Song , Xi'an Jiaotong University, Xi'an
Martin Shepperd , Brunel University, Uxbridge
BACKGROUND—Predicting defect-prone software components is an economically important activity and so has received a good deal of attention. However, making sense of the many, and sometimes seemingly inconsistent, results is difficult. OBJECTIVE—We propose and evaluate a general framework for software defect prediction that supports 1) unbiased and 2) comprehensive comparison between competing prediction systems. METHOD—The framework is comprised of 1) scheme evaluation and 2) defect prediction components. The scheme evaluation analyzes the prediction performance of competing learning schemes for given historical data sets. The defect predictor builds models according to the evaluated learning scheme and predicts software defects with new data according to the constructed model. In order to demonstrate the performance of the proposed framework, we use both simulation and publicly available software defect data sets. RESULTS—The results show that we should choose different learning schemes for different data sets (i.e., no scheme dominates), that small details in conducting how evaluations are conducted can completely reverse findings, and last, that our proposed framework is more effective and less prone to bias than previous approaches. CONCLUSIONS—Failure to properly or fully evaluate a learning scheme can be misleading; however, these problems may be overcome by our proposed framework.
Software defect prediction, software defect-proneness prediction, machine learning, scheme evaluation.
Jin Liu, Shi Ying, Zihan Jia, Qinbao Song, Martin Shepperd, "A General Software Defect-Proneness Prediction Framework", IEEE Transactions on Software Engineering, vol. 37, no. , pp. 356-370, May/June 2011, doi:10.1109/TSE.2010.90