This Article 
 Bibliographic References 
 Add to: 
Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults
October 2006 (vol. 32 no. 10)
pp. 771-789
Hareton Leung, IEEE Computer Society
In the last decade, empirical studies on object-oriented design metrics have shown some of them to be useful for predicting the fault-proneness of classes in object-oriented software systems. This research did not, however, distinguish among faults according to the severity of impact. It would be valuable to know how object-oriented design metrics and class fault-proneness are related when fault severity is taken into account. In this paper, we use logistic regression and machine learning methods to empirically investigate the usefulness of object-oriented design metrics, specifically, a subset of the Chidamber and Kemerer suite, in predicting fault-proneness when taking fault severity into account. Our results, based on a public domain NASA data set, indicate that 1) most of these design metrics are statistically related to fault-proneness of classes across fault severity, and 2) the prediction capabilities of the investigated metrics greatly depend on the severity of faults. More specifically, these design metrics are able to predict low severity faults in fault-prone classes better than high severity faults in fault-prone classes.

[1] T. Gyimóthy, R. Ference, and L. Siket, “Empirical Validation of Object-Oriented Metrics on Open Source Software for Fault Prediction,” IEEE Trans. Software Eng., vol. 31, no. 10, pp. 897-910, Oct. 2005.
[2] R. Subramanyan and M.S. Krisnan, “Empirical Analysis of CK Metrics for Object-Oriented Design Complexity: Implications for Software Defects,” IEEE Trans. Software Eng., vol. 29, no. 4, pp297-310, Apr. 2003.
[3] M. Alshayeb and L. Wei, “An Empirical Validation of Object-Oriented Metrics in Two Different Iterative Software Processes,” IEEE Trans. Software Eng., vol. 29, no. 11, pp. 1043-1049, Nov. 2003.
[4] K. El Emam, S. Benlarbi, N. Goel, and S.N. Rai, “The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics,” IEEE Trans. Software Eng., vol. 27, no. 7, pp. 630-650, July 2001.
[5] S.R. Chidamber, D.P. Darcy, and C.F. Kemerer, “Managerial Use of Metrics for Object-Oriented Software: An Exploratory Analysis,” IEEE Trans. Software Eng., vol. 24, no. 8, pp. 629-639, Aug. 1998.
[6] V.R. Basili, L.C. Briand, and W.L. Melo, “A Validation of Object-Oriented Design Metrics as Quality Indicators,” IEEE Trans. Software Eng., vol. 22, no. 10, pp. 751-761, Oct. 1996.
[7] M.M.T. Thwin and T.S. Quah, “Application of Neural Networks for Software Quality Prediction Using Object-Oriented Metrics,” J.Systems and Software, vol. 76, no. 2, pp. 147-156, 2005.
[8] G. Succi, W. Pedrycz, M. Stefanovic, and J. Miller, “Practical Assessment of the Models for Identification of Defect-Prone Classes in Object-Oriented Commercial Systems Using Design Metrics,” J. Systems and Software, vol. 65, no. 1, pp. 1-12, 2003.
[9] L.C. Briand, J. Wüst, J.W. Daly, and D.V. Porter, “Exploring the Relationships between Design Measures and Software Quality in Object-Oriented Systems,” J. Systems and Software, vol. 51, no. 3, pp. 245-273, 2000.
[10] H. Lounis and L. Ait-Mehedine, “Machine-Learning Techniques for Software Product Quality Assessment,” Proc. Fourth Int'l Conf. Quality Software, pp. 102-109, 2004.
[11] P. Yu, T. Systa, and H. Muller, “Predicting Fault-Proneness Using OO Metrics: An Industrial Case Study,” Proc. Sixth European Conf. Software Maintenance and Reeng., pp. 99-107, 2002.
[12] L.C. Briand, J. Daly, V. Porter, and J. Wüst, “A Comprehensive Empirical Validation of Design Measures for Object-Oriented Systems,” Proc. Fifth Int'l Software Metrics Symp., pp. 246-257, 1998.
[13] L.C. Briand and J. Wüst, “Empirical Studies of Quality Models in Object-Oriented Systems,” Advances in Computers, D.M. Zelkowitz, ed., pp. 97-166, vol. 56, Academic, 2002.
[14] S.R. Chidamber and C.F. Kemerer, “A Metrics Suite for Object-Oriented Design,” IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476-493, June 1994.
[15] N.E. Fenton and M. Neil, “A Critique of Software Defect Prediction Models,” IEEE Trans. Software Eng., vol. 25, no. 5, pp.675-689, May 1999.
[16] T.J. Ostrand, E.J. Weyuker, and R.M. Bell, “Predicting the Location and Number of Faults in Large Software Systems,” IEEE Trans. Software Eng., vol. 31, no. 4, pp. 340-355, Apr. 2005.
[17] L.C. Briand, J. Wüst, and H. Lounis, “Replicated Case Studies for Investigating Quality Factors in Object-Oriented Designs,” Empirical Software Eng., vol. 6, no. 1, pp. 11-58, 2001.
[18] L. Rosenberg and L. Hyatt, “Software Quality Metrics for Object-Oriented System Environments,” NASA Technical Report SATC-TR-95-1001, 1995.
[19] W.G. Hopkins, A New View of Statistics. Sport Science, 2003.
[20] D. Belsley, E. Kuh, and R. Welsch, Regression Diagnostics: Identifying Influential Data and Sources of Collinearity. John Wiley and Sons, 1980.
[21] L.C. Briand and J. Wust, “Modeling Development Effort in Object-Oriented Systems Using Design Properties,” IEEE Trans. Software Eng., vol. 27, no. 11, pp. 963-986, Nov. 2001.
[22] A. Albert and A. Anderson, “On the Existence of Maximum Likelihood Estimates in Logistic Regression Models,” Biometrika, vol. 71, pp. 1-10, 1984.
[23] L.C. Briand, W.L. Melo, and J. Wust, “Assessing the Application of Fault-Proneness Models Across Object-Oriented Software Projects,” IEEE Trans. Software Eng., vol. 28, no. 7, pp. 706-720, July 2002.
[24] G. Denaro, M. Pezzè, and S. Morasca, “Towards Industrially Relevant Fault-Proneness Models,” Int'l J. Software Eng. and Knowledge Eng., vol. 13, no. 4, pp. 395-417, 2003.
[25] T.M. Khoshgoftaar, Y. Liu, and N. Seliya, “A Multiobjective Module-Order Model for Software Quality Enhancement,” IEEE Trans. Evolutionary Computation, vol. 8, no. 6, pp. 593-608, 2004.
[26] L.C. Briand, J. Wust, S.V. Ikonomovski, and H. Lounis, “Investigating Quality Factors in Object-Oriented Designs: An Industrial Case Study,” Proc. 21st Int'l Conf. Software Eng., pp. 345-354, 1999.
[27] M.H. Tang, M.H. Kao, and M.H. Chen, “An Empirical Study on Object-Oriented Metrics,” Proc. Sixth Int'l Software Metrics Symp., pp. 242-249, 1999.
[28] N. Ohlsson and H. Alberg, “Predicting Fault-Prone Software Modules in Telephone Switches,” IEEE Trans. Software Eng., vol. 22, no. 12, pp. 886-894, Dec. 1996.
[29] R.S. Michalski and K.A. Kaufman, “Learning Patterns in Noisy Data: The AQ Approach,” Machine Learning and Applications, G.Paliouras, V. Karkaletsis, and C. Spyropoulos, eds., pp. 22-38, Springer-Verlag, 2001.
[30] T.J. Ostrand and E.J. Weyuker, “The Distribution of Faults in a Large Industrial Software System,” Proc. Int'l Symp. Software Testing and Analysis, pp. 55-64, 2002.
[31] A.A. Porter, “Using Measurement-Driven Modeling to Provide Empirical Feedback to Software Developers,” J. Systems and Software, vol. 20, no. 3, pp. 237-243, 1993.

Index Terms:
Object-oriented, faults, fault-proneness, metrics, prediction, cross validation.
Yuming Zhou, Hareton Leung, "Empirical Analysis of Object-Oriented Design Metrics for Predicting High and Low Severity Faults," IEEE Transactions on Software Engineering, vol. 32, no. 10, pp. 771-789, Oct. 2006, doi:10.1109/TSE.2006.102
Usage of this product signifies your acceptance of the Terms of Use.