This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Do Crosscutting Concerns Cause Defects?
July/August 2008 (vol. 34 no. 4)
pp. 497-515
Marc Eaddy, Columbia University, NY
Thomas Zimmermann, University of Calgary, Calgary
Kaitlin D. Sherwood, University of British Columbia, Vancouver
Vibhav Garg, Columbia University, NY
Gail C. Murphy, University of British Columbia, Vancouver
Nachiappan Nagappan, Microsoft Research, Redmond
Alfred V. Aho, Columbia University, NY
There is a growing consensus that crosscutting concerns harm code quality. An example of a crosscutting concern is a functional requirement whose implementation is distributed across multiple software modules. We asked the question, "How much does the amount that a concern is crosscutting affect the number of defects in a program?" We conducted three extensive case studies to help answer this question. All three studies revealed a moderate to strong statistically significant correlation between the degree of scattering and the number of defects. This paper describes the experimental framework we developed to conduct the studies, the metrics we adopted and developed to measure the degree of scattering, the studies we performed, the efforts we undertook to remove experimental and other biases, and the results we obtained. In the process, we have formulated a theory that explains why increased scattering might lead to increased defects.

[1] A.V. Aho, M.S. Lam, R. Sethi, and J.D. Ullman, Compilers: Principles, Techniques, and Tools, second ed. Addison Wesley, 2006.
[2] G. Antoniol, G. Canfora, G. Casazza, A. De Lucia, and E. Merlo, “Recovering Traceability Links between Code and Documentation,” IEEE Trans. Software Eng., vol. 28, no. 10, pp. 970-983, Oct. 2002.
[3] M. Bartsch and R. Harrison, “Towards an Empirical Validation of Aspect-Oriented Coupling Measures,” Proc. Workshop Assessment of Aspect Techniques, 2007.
[4] V. Basili, F. Shull, and F. Lanubile, “Building Knowledge through Families of Experiments,” IEEE Trans. Software Eng., vol. 25, no. 4, pp. 456-473, July/Aug. 1999.
[5] K.v.D. Berg, J.M. Conejero, and J. Hernández, “Analysis of Crosscutting across Software Development Phases Based on Traceability,” Proc. Workshop Aspect-Oriented Requirements Eng. and Architecture Design (Early Aspects), 2006.
[6] T.J. Biggerstaff, B.G. Mitbander, and D. Webster, “The Concept Assignment Problem in Program Understanding,” Proc. 15th Int'l Conf. Software Eng., pp. 482-498, 1993.
[7] S. Biyani and P. Santhanam, “Exploring Defect Data from Development and Customer Usage on Software Modules over Multiple Releases,” Proc. Ninth Int'l Symp. Software Reliability Eng., 1998.
[8] L. Briand, J. Wuest, J. Daly, and V. Porter, “Exploring the Relationships between Design Measures and Software Quality in Object Oriented Systems,” J. Systems and Software, vol. 51, pp. 245-273, 2000.
[9] M. Bruntink, A.v. Deursen, and T. Tourwé, “Discovering Faults in Idiom-Based Exception Handling,” Proc. 28th Int'l Conf. Software Eng., 2006.
[10] G. Canfora, L. Cerulo, and M.D. Penta, “On the Use of Line Co-Change for Identifying Crosscutting Concern Code,” Proc. 22nd Int'l Conf. Software Maintenance, 2006.
[11] M. Cartwright and M. Shepperd, “An Empirical Investigation of an Object-Oriented Software System,” IEEE Trans. Software Eng., vol. 26, no. 8, pp. 786-796, Aug. 2000.
[12] P. Cederqvist et al., Version Management with CVS. Network Theory, Ltd., 2002.
[13] S. Chidamber and C. Kemerer, “A Metrics Suite for Object Oriented Design,” IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476-493, June 1994.
[14] J. Cohen, Statistical Power Analysis for the Behavioral Sciences, second ed. Lawrence Erlbaum Assoc., 1988.
[15] B. Collins-Sussman, B.W. Fitzpatrick, and C.M. Pilato, Version Control with Subversion. O'Reilly, 2004.
[16] D. Čubranić, G.C. Murphy, J. Singer, and K.S. Booth, “Hipikat: A Project Memory for Software Development,” IEEE Trans. Software Eng., vol. 31, no. 6, pp. 446-465, June 2005.
[17] M. Eaddy, A. Aho, and G.C. Murphy, “Identifying, Assigning, and Quantifying Crosscutting Concerns,” Proc. Workshop Assessment of Contemporary Modularization Techniques, 2007.
[18] ECMA, “ECMAScript Standard,” vol. ECMA-262 v3, ISO/IEC 16262, 2007.
[19] T. Eisenbarth, R. Koschke, and D. Simon, “Locating Features in Source Code,” IEEE Trans. Software Eng., vol. 29, no. 3, pp. 210-224, Mar. 2003.
[20] A.D. Eisenberg and K. De Volder, “Dynamic Feature Traces: Finding Features in Unfamiliar Code,” Proc. 21st Int'l Conf. Software Maintenance, pp. 337-346, 2005.
[21] K. El Emam, “A Methodology for Validating Software Product Metrics,” Technical Report NRC 44142, Nat'l Research Council of Canada, 2000.
[22] K. El Emam, S. Benlarbi, N. Goel, and S.N. Rai, “The Confounding Effect of Class Size on the Validity of Object-Oriented Metrics,” IEEE Trans. Software Eng., vol. 27, no. 7, pp. 630-650, July 2001.
[23] N.E. Fenton and N. Ohlsson, “Quantitative Analysis of Faults and Failures in Complex Software Systems,” IEEE Trans. Software Eng., vol. 26, no. 8, pp. 797-814, Aug. 2000.
[24] E. Figueiredo, A. Garcia, C. Sant'Anna, U. Kulesza, and C. Lucena, “Assessing Aspect-Oriented Artifacts: Towards a Tool-Supported Quantitative Method,” Proc. Ninth ECOOP Workshop Quantitative Approaches in Object-Oriented Software Eng., 2005.
[25] F.C. Filho, N. Cacho, E. Figueiredo, R. Maranhao, A. Garcia, and C.M.F. Rubira, “Exceptions and Aspects: The Devil Is in the Details,” Foundations of Software Eng., pp. 152-162, 2006.
[26] R.E. Filman, T. Elrad, S. Clarke, and M. Aksit, Aspect-Oriented Software Development. Addison-Wesley, 2005.
[27] M. Fischer, M. Pinzger, and H. Gall, “Populating a Release History Database from Version Control and Bug Tracking Systems,” Proc. 19th Int'l Conf. Software Maintenance, pp. 23-32, 2003.
[28] A. Garcia, C. Sant'Anna, E. Figueiredo, U. Kulesza, C. Lucena, and A.v. Staa, “Modularizing Design Patterns with Aspects: A Quantitative Study,” Proc. Int'l Conf. Aspect-Oriented Software Development, 2005.
[29] C. Gibbs, C.R. Liu, and Y. Coady, “Sustainable System Infrastructure and Big Bang Evolution: Can Aspects Keep Pace,” Proc. 19th European Conf. Object-Oriented Programming, pp. 241-261, 2005.
[30] P. Greenwood, T.T. Bartolomei, E. Figueiredo, M. Dósea, A. Garcia, N. Cacho, C. Sant'Anna, S. Soares, P. Borba, U. Kulesza, and A. Rashid, “On the Impact of Aspectual Decompositions on Design Stability: An Empirical Study,” Proc. 21st European Conf. Object-Oriented Programming, pp. 176-200, 2007.
[31] J.E. Hannay, D.I.K. Sjøberg, and T. Dybå, “A Systematic Review of Theory Use in Software Engineering Experiments,” IEEE Trans. Software Eng., vol. 33, no. 2, pp. 87-107, Feb. 2007.
[32] R. Harrison, S. Counsel, and R. Nithi, “Experimental Assessment of the Effect of Inheritance on the Maintainability of Object-Oriented Systems,” J. Systems and Software, vol. 52, pp. 173-179, 2000.
[33] R. Harrison, L. Samaraweera, M. Dobie, and P. Lewis, “An Evaluation of Code Metrics for Object-Oriented Programs,” Information and Software Technology, vol. 38, pp. 443-450, 1996.
[34] IEEE Standard 610.12-1990, IEEE Standard Glossary of Software Engineering Terminology, IEEE, 1990.
[35] ISO/IEC, “Information Technology—Software Product Evaluation,” IDS 14598-1, 1996.
[36] E.J. Jackson, A User's Guide to Principal Components. John Wiley & Sons, 1991.
[37] S.K. Kachigan, Statistical Analysis. Radius Press, 1986.
[38] S.H. Kan, Metrics and Models in Software Quality Engineering, second ed. Addison-Wesley, 2003.
[39] M. Kersten and G.C. Murphy, “Using Task Context to Improve Programmer Productivity,” Foundations of Software Eng., 2006.
[40] T.M. Khoshgoftaar, E.B. Allen, N. Goel, A. Nandi, and J. McMullan, “Detection of Software Modules with High Debug Code Churn in a Very Large Legacy System,” Proc. Seventh Int'l Symp. Software Reliability Eng., pp. 364-371, 1996.
[41] T.M. Khoshgoftaar, E.B. Allen, W.D. Jones, and J.P. Hudepohl, “Classification-Tree Models of Software Quality over Multiple Releases,” IEEE Trans. Reliability, vol. 49, no. 1, pp. 4-11, 2000.
[42] G. Kiczales, J. Irwin, J. Lamping, J.-M. Loingtier, C.V. Lopes, C. Maeda, and A. Mendhekar, “Aspect-Oriented Programming,” ACM Computing Surveys, vol. 28, no. 4es, p. 154, 1996.
[43] B. Kitchenham, S.L. Pfleeger, and N. Fenton, “Towards a Framework for Software Measurement Validation,” IEEE Trans. Software Eng., vol. 21, no. 12, pp. 929-944, Dec. 1995.
[44] A.J. Ko, R. DeLine, and G. Venolia, “Information Needs in Collocated Software Development Teams,” Proc. 29th Int'l Conf. Software Eng., 2007.
[45] A. Lai and G.C. Murphy, “The Structure of Features in Java Code: An Exploratory Investigation,” Proc. Workshop Multi-Dimensional Separation of Concerns, 1999.
[46] S. Letovsky and E. Soloway, “Delocalized Plans and Program Comprehension,” IEEE Software, vol. 3, no. 3, pp. 41-49, 1986.
[47] M. Lippert and C.V. Lopes, “A Study on Exception Detection and Handling Using Aspect-Oriented Programming,” Proc. 22nd Int'l Conf. Software Eng., pp. 418-427, 2000.
[48] A.v. Mayrhauser, A.M. Vans, and A.E. Howe, “Program Understanding Behaviour during Enhancement of Large-Scale Software,” Software Maintenance: Research and Practice, vol. 9, pp. 299-327, 1997.
[49] N. Nagappan and T. Ball, “Use of Relative Code Churn Measures to Predict System Defect Density,” Proc. 27th Int'l Conf. Software Eng., 2005.
[50] D. Poshyvanyk, Y.-G. Guéhéneuc, A. Marcus, G. Antoniol, and V. Rajlich, “Feature Location Using Probabilistic Ranking of Methods Based on Execution Scenarios and Information Retrieval,” IEEE Trans. Software Eng., vol. 33, no. 6, pp. 420-432, June 2007.
[51] R. Purushothaman and D.E. Perry, “Toward Understanding the Rhetoric of Small Source Changes,” IEEE Trans. Software Eng., vol. 31, no. 6, pp. 511-526, June 2005.
[52] M. Revelle, T. Broadbent, and D. Coppit, “Understanding Concerns in Software: Insights Gained from Two Case Studies,” Proc. 13th IEEE Int'l Workshop Program Comprehension, 2005.
[53] M.P. Robillard, “Representing Concerns in Source Code,” PhD thesis, Computer Science Dept., Univ. of British Columbia, Nov. 2003.
[54] M.P. Robillard, W. Coelho, and G.C. Murphy, “How Effective Developers Investigate Source Code: An Exploratory Study,” IEEE Trans. Software Eng., vol. 30, no. 12, pp. 889-903, Dec. 2004.
[55] M.P. Robillard and F. Weigand-Warr, “ConcernMapper: Simple View-Based Separation of Scattered Concerns,” Proc. Workshop Eclipse Technology eXchange, 2005.
[56] M.J. Rochkind, “The Source Code Control System,” IEEE Trans. Software Eng., vol. 1, no. 4, pp. 364-370, 1975.
[57] J. Śliwerski, T. Zimmermann, and A. Zeller, “When Do Changes Induce Fixes,” Proc. Workshop Mining Software Repositories, 2005.
[58] P.H. Sneath and R.R. Sokal, Numerical Taxonomy. W.H. Freeman, 1973.
[59] S.M. Sutton Jr. and I. Rouvellou, “Concern Modeling for Aspect-Oriented Software Development,” Aspect-Oriented Software Development, pp. 479-505, Addison-Wesley, 2005.
[60] S.L. Tsang, S. Clarke, and E. Baniassad, “An Evaluation of Aspect-Oriented Programming for Java-Based Real-time Systems Development,” Proc. Seventh Int'l Symp. Object-Oriented Real-Time Distributed Computing, 2004.
[61] N. Wilde and M.C. Scully, “Software Reconnaissance: Mapping Program Features to Code,” J. Software Maintenance and Evolution: Research and Practice, vol. 7, no. 1, pp. 49-62, 1995.
[62] W.E. Wong, S.S. Gokhale, and J.R. Horgan, “Quantifying the Closeness between Program Components and Features,” J.Systems and Software, vol. 54, no. 2, pp. 87-98, 2000.
[63] M.V. Zelkowitz and D.R. Wallace, “Experimental Models for Validating Technology,” Computer, vol. 31, no. 5, pp. 23-31, May 1998.
[64] C. Zhang and H.-A. Jacobsen, “Quantifying Aspects in Middleware Platforms,” Proc. Int'l Conf. Aspect-Oriented Software Development, pp. 130-139, 2003.
[65] W. Zhao, L. Zhang, Y. Liu, J. Sun, and F. Yang, “SNIAFL: Towards a Static Noninteractive Approach to Feature Location,” ACM Trans. Software Eng. and Methodology, vol. 15, no. 2, pp. 195-226, 2006.

Index Terms:
Metrics/Measurement, Validation, Quality analysis and evaluation, Software Construction, Software Quality/SQA, Correlation and regression analysis, User/Machine Systems, Distribution Maintenance and Enhancement, Software Engineering
Citation:
Marc Eaddy, Thomas Zimmermann, Kaitlin D. Sherwood, Vibhav Garg, Gail C. Murphy, Nachiappan Nagappan, Alfred V. Aho, "Do Crosscutting Concerns Cause Defects?," IEEE Transactions on Software Engineering, vol. 34, no. 4, pp. 497-515, July-Aug. 2008, doi:10.1109/TSE.2008.36
Usage of this product signifies your acceptance of the Terms of Use.