This Article 
 Bibliographic References 
 Add to: 
An Empirical Study on the Relationship Between Software Design Quality, Development Effort and Governance in Open Source Projects
November/December 2008 (vol. 34 no. 6)
pp. 765-782
Eugenio Capra, Politecnico di Milano, Milano
Chiara Francalanci, Politecnico di Milano, Milan
Francesco Merlo, Politecnico di Milano, Milan
The relationship among software design quality, development effort, and governance practices is a traditional research problem. However, the extent to which consolidated results on this relationship remain valid for open source (OS) projects is an open research problem. An emerging body of literature contrasts the view of open source as an alternative to proprietary software and explains that there exists a continuum between closed and open source projects. This paper hypothesizes that as projects approach the OS end of the continuum, governance becomes less formal. In turn a less formal governance is hypothesized to require a higher-quality code as a means to facilitate coordination among developers by making the structure of code explicit and facilitate quality by removing the pressure of deadlines from contributors. However, a less formal governance is also hypothesized to increase development effort due to a more cumbersome coordination overhead. The verification of research hypotheses is based on empirical data from a sample of 75 major OS projects. Empirical evidence supports our hypotheses and suggests that software quality, mainly measured as coupling and inheritance, does not increase development effort, but represents an important managerial variable to implement the more open governance approach that characterizes OS projects which, in turn, increases development effort.

[1] M. Agrawal and K. Chari, “Software Effort, Quality, and Cycle Time: A Study of CMM Level 5 Projects,” IEEE Trans. Software Eng., vol. 33, no. 3, pp. 145-156, Mar. 2007.
[2] Y. Ahn, J. Suh, S. Kim, and H. Kim, “A Software Maintenance Project Effort Estimation Model Based on Function Points,” J.Software Maintenance Evolution: Research and Practice, vol. 15, no. 2, pp. 71-85, 2003.
[3] J.L. Arbuckle, Amos 5.0 Update to the Amos User's Guide. Small Waters, 2003.
[4] R.P. Bagozzi and Y. Yi, “On the Evaluation of Structural Equation Models,” J. Academy of Marketing Science, vol. 16, no. 1, pp. 74-94, 1988.
[5] R.D. Banker and S.A. Slaughter, “A Study on the Effects of Software Development Practices on Software Maintenance Effort,” Proc. Int'l Conf. Software Maintenance, pp. 197-205, 1996.
[6] R.D. Banker, G.B. Davis, and S.A. Slaughter, “Software Development Practices, Software Complexity, and Software Maintenance Performance: A Field Study,” Management Science, vol. 44, no. 4, pp. 433-450, 1998.
[7] V.R. Basili and R.W. Weiter Jr., “A Controlled Experiment Quantitatively Comparing Software Development Approaches,” IEEE Trans. Software Eng., vol. 7, no. 3, pp. 279-401, 1981.
[8] V.R. Basili, L. Briand, S. Condon, Y.-M. Kim, W.L. Melo, and J.D. Valett, “Understanding and Predicting the Process of Software Maintenance Releases,” Proc. 18th Int'l Conf. Software Eng., pp. 464-474, 1996.
[9] V.R. Basili, L.C. Briand, and W.L. Melo, “A Validation of Object-Oriented Design Metrics as Quality Indicators,” IEEE Trans. Software Eng., vol. 22, no. 10, pp. 751-761, Oct. 1996.
[10] A. Battacherjee, “Understanding Information Systems Continuance: An Expectation-Confirmation Model,” MIS Quarterly, vol. 25, no. 3, pp. 351-370, 2001.
[11] K. Beck, Extreme Programming Explained. Addison-Wesley, 1999.
[12] P.M. Bentler, “Comparative Fit Indexes in Structural Models,” Psychological Bull., vol. 107, no. 2, pp. 238-246, 1990.
[13] J. Bergus, The5 Types of Open Source Projects, http://www.power postgresql.com5_types, 2005.
[14] A. Bianchi, D. Caivano, F. Lanubile, and G. Visaggio, “Evaluating Software Degradation through Quality,” Proc. Seventh IEEE Int'l Software Metrics Symp., pp. 210-219, 2001.
[15] A.B. Binkley and S.R. Scatch, “Validation of the Coupling Dependency Metric as a Predictor of Run Time Failures and Maintenance Measures,” Proc. 20th Int'l Conf. Software Eng., pp.452-455, 1998.
[16] B. Boehm, Software Engineering Economics. Prentice Hall, 1981.
[17] B. Boehm, Software Cost Estimation with COCOMO II. Prentice Hall, 2000.
[18] B. Boehm and R. Turner, Balancing Agility and Discipline: A Guide for the Perplexed. Addison-Wesley, 2002.
[19] A. Boulanger, “Open Source versus Proprietary Software: Is One More Reliable and Secure than the Other,” IBM Systems J., vol. 44, no. 2, p. 239, 2005.
[20] L.C. Briand, K. El Emam, and S. Morasca, “On the Application of Measurement Theory in Software Engineering,” J. Empirical Software Eng., vol. 1, no. 1, pp. 61-88, 1996.
[21] L.C. Briand, J.W. Daly, and J.K. Wust, “A Unified Framework for Coupling Measurement in Object Oriented Systems,” IEEE Trans. Software Eng., vol. 25, no. 1, pp. 91-121, Jan./Feb. 1999.
[22] F. Brito e Abreu, “The MOOD Metrics Set,” Proc. ECOOP Workshop Metrics, 1995.
[23] E. Capra, “Software Design Quality and Development Effort: An Empirical Study on the Role of Governance in Open Source Projects,” PhD dissertation, Dept. of Electronics and Information, Politecnico di Milano, 2008.
[24] E. Capra and A.I. Wasserman, “A Framework for Evaluating Managerial Styles in Open Source Projects,” Proc. Int'l Conf. Open Source Systems, 2008.
[25] E. Capra and A.I. Wasserman, “Evaluating Software Engineering Processes in Commercial and Community Open Source Projects,” Proc. First Int'l Workshop Emerging Trends in FLOSS Research and Development, 2007.
[26] E. Capra, C. Francalanci, and F. Merlo, “The Economics of Open Source Software: An Empirical Analysis of Maintenance Costs,” Proc. Int'l Conf. Software Maintenance, pp. 395-404, 2007.
[27] E.G. Carmines and J.P. McIver, “Analyzing Models with Unobserved Variables: Analysis of Covariance Structures,” Social Measurement: Current Issues, G.W. Bohrnstedt and E.F. Borgatta, eds., pp. 65-115, Sage Publications, 1981.
[28] T. Chan, S. Chung, and T. Ho, “An Econometric Model to Estimate Software Rewriting and Replacement Times,” IEEE Trans. Software Eng., vol. 22, no. 8, pp. 580-598, Aug. 1996.
[29] N. Chapin, J.E. Hale, K.M. Khan, J.F. Ramil, and W.-G. Tan, “Types of Software Evolution and Software Maintenance,” J.Software Maintenance and Evolution: Research and Practice, no. 13, pp. 3-30, 2001.
[30] J.Y. Chen and J.F. Lu, “A New Metric for Object-Oriented Design,” J. Information System and Software Technology, vol. 35, no. 4, pp. 232-240, 1993.
[31] S.R. Chidamber and C.F. Kemerer, “A Metrics Suite for Object Oriented Design,” IEEE Trans. Software Eng., vol. 20, no. 6, pp. 476-493, June 1994.
[32] S.R. Chidamber, D.P. Darcy, and C.F. Kemerer, “Managerial Use of Metrics for Object-Oriented Software: An Exploratory Analysis,” IEEE Trans. Software Eng., vol. 24, no. 8, pp. 629-639, Aug. 1998.
[33] M.B. Chrissis, M. Konrad, and S. Shrum, CMMI Guidelines for Process Integration and Product Improvement. Addison-Wesley, 2003.
[34] D.P. Darcy, C.F. Kemerer, S.A. Slaughter, and J.E. Tomayko, “The Structural Complexity of Software: An Experimental Test,” IEEE Trans. Software Eng., vol. 31, no. 11, pp. 982-995, Nov. 2005.
[35] J. Ekanayake, A. Bernstein, and M. Pinzger, “Improving Defect Prediction Using Temporal Features and Nonlinear Models,” Proc. Int'l Workshop Principles of Software Evolution), pp. 11-18, 2007.
[36] T.J. Emerson, “A Discriminant Metric for Module Comprehension,” Proc. Seventh Int'l Conf. Software Eng., pp. 294-431, 1984.
[37] N. Fenton and S.L. Pfleeger, Software Metrics: A Rigorous and Practical Approach. Int'l Thomson Computer Press, 1997.
[38] R.T. Fielding, “Shared Leadership in the Apache Project,” Comm. ACM, vol. 42, no. 4, pp. 42-43, 1999.
[39] B. Fitzgerald, “A Critical Look at Open Source,” Computer, vol. 37, no. 7, pp. 92-94, July 2004.
[40] B. Fitzgerald, “The Transformation of Open Source Software,” MIS Quarterly, vol. 30, no. 2, pp. 587-598, 2006.
[41] K. Fogel, Producing Open Source Software. O'Reilly, 2006.
[42] C. Fornell and D.F. Larcker, “Evaluating Structural Equation Models with Unobservable Variables and Measurement Errors: Algebra and Statistics,” J. Marketing Research, vol. 18, no. 3, pp.383-388, 1981.
[43] M. Fowler, K. Beck, J. Brant, W. Opdyke, and D. Roberts, Refactoring: Improving the Design of Existing Code. Addison-Wesley, 2001.
[44] C. Ghezzi, M. Jazayeri, and D. Mandrioli, Fundamentals of Software Engineering, second ed. Prentice Hall PTR, Apr. 2002.
[45] P.J. Gomes and N.R. Joglekar, “The Costs of Coordinating Distributed Software Development Tasks,” Boston Univ. School of Management Working Paper, Oct. 2004.
[46] R. Goldman and R.P. Gabriel, Innovation Happens Elsewhere: Open Source as Business Strategy. Morgan Kaufmann, Apr. 2005.
[47] G. Goth and B. Massey, “Open Source Business Models: Ready for Prime Time,” IEEE Software, vol. 22, no. 6, pp. 98-102, Nov./Dec. 2005.
[48] M. Griffiths, “Most Software Development Metrics Are Misleading and Counterproductive,” Agile J., http://www.agilejournal. com/content/view 107, Oct. 2006.
[49] T. Gyimothy, R. Ferenc, and I. Siket, “Empirical Validation of Object Oriented Metrics on Open Source Software for Fault Prediction,” IEEE Trans. Software Eng., vol. 31, no. 10, pp. 897-910, Oct. 2005.
[50] J.F. Hair, R.E. Anderson, R.L. Tatham, and W.C. Black, Multivariate Data Analysis with Readings. Prentice Hall, 1995.
[51] M.H. Halstead, “Elements of Software Science,” Elsevier Computer Science Library, 1977.
[52] R. Harrison, S. Counsell, and R. Nithi, “An Evaluation of the MOOD Set of Object-Oriented Software Metrics,” IEEE Trans. Software Eng., vol. 24, no. 6, pp. 491-496, June 1998.
[53] D.E. Harter, M.S. Krishnan, and S.A. Slaughter, “Effects of Process Maturity on Quality, Cycle Time and Effort in Software Product Development,” Management Science, vol. 46, no. 4, pp. 451-466, 2000.
[54] S. Henry and D. Kafura, “Software Structure Metrics Based on Information Flow,” IEEE Trans. Software Eng., vol. 7, no. 5, pp. 510-518, 1981.
[55] J.D. Herbsleb and D.R. Goldenson, “A Systematic Survey of CMM Experience and Results,” Proc. 18th Int'l Conf. Software Eng., pp.323-330, 1996.
[56] M. Hitz and B. Montazeri, “Measure Coupling and Cohesion in Object-Oriented Systems,” Proc. Int'l Symp. Applied Corporate Computing, Oct. 1995.
[57] M. Hitz and B. Montazeri, “Chidamber and Kemerer's Metrics Suite: A Measurement Theory Perspective,” IEEE Trans. Software Eng., vol. 22, no. 4, pp. 267-271, Apr. 1996.
[58] J. Howison and K. Crowston, “The Social Structure of Free and Open Source Software Development,” First Monday [Online], vol. 10, no. 2, Feb. 2005.
[59] J. Howison and K. Crowston, “The Perils and Pitfalls of Mining SourceForge,” Proc. Int'l Workshop Mining Software Repositories, pp.7-12, 2004.
[60] L. Hu and P.M. Bentler, “Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives,” Structural Equation Modeling, vol. 6, no. 1, pp. 1-55, 1999.
[61] K. Ishikawa, Introduction to Quality Control. Union of Japanese Scientists and Engineers Press, 1989.
[62] Software Engineering—Product Quality, ISO/IEC, TR 9126:2003, Int'l Organization for Standardization, Geneva, Switzerland, 2003.
[63] Software Engineering—Software Product Quality Requirements and Evaluation (SQuaRE), ISO/IEC, TR 25000:2005, Int'l Organization for Standardization, Geneva, Switzerland, 2005.
[64] J. Jurison, “Software Project Management: The Manager's View,” Comm. Assoc. Information Systems, vol. 2, no. 3,article no. 2, 1999.
[65] D. Kaplan, “Structural Equation Modeling: Foundations and Extensions,” Advanced Quantitative Techniques in the Social Sciences Series, vol. 10, 2000.
[66] C.F. Kemerer, “An Empirical Validation of Software Cost Estimation Models,” Comm. ACM, vol. 30, no. 5, pp. 416-430, 1987.
[67] M. Kersten and G.C. Murphy, “Using Task Context to Improve Programmer Productivity,” Proc. 14th SIGSOFT Symp. Foundations of Software Eng., pp. 1-11, 2006.
[68] M. Kersten and G.C. Murphy, “Mylar: A Degree-of-Interest Model for IDEs,” Proc. Fourth Int'l Conf. Aspect-Oriented Software Development, pp. 159-168, 2005.
[69] L.J. Kirsch, “The Management of Complex Tasks in Organizations: Controlling the Systems Development Process,” Organization Science, vol. 7, no. 1, pp. 1-21, 1996.
[70] R.B. Kline, Principles and Practice of Structural Equation Modeling. Guilford Press, 1988.
[71] P. Knab, M. Pinzger, and A. Bernstein, “Predicting Defect Densities in Source Code Files with Decision Tree Learners,” Proc. Int'l Workshop Mining Software Repositories, pp. 119-125, 2006.
[72] S. Koch, “Agile Principles and Open Source Software Development: A Theoretical and Empirical Discussion,” Extreme Programming and Agile Processes in Software Engineering, pp. 85-93, Springer, 2004.
[73] M. Komuro, “Experience of Applying SPC Techniques to Software Development Processes,” Proc. 28th Int'l Conf. Software Eng., pp.577-585, 2006.
[74] S. Krishnamurthy, “A Managerial Overview of Open Source Software,” Business Horizons, pp. 47-56, Sept./Oct. 2003.
[75] S. Kvale, Interviews: An Introduction to Qualitative Research Interviewing. Sage Publications, 1996.
[76] M. Lanza and R. Marinescu, Object-Oriented Metrics in Practice— Using Software Metrics to Characterize, Evaluate, and Improve the Design of Object-Oriented Systems. Springer, 2006.
[77] W. Li and S. Henry, “Maintenance Metrics for the Object Oriented Paradigm,” Proc. IEEE Int'l Software Metrics Symp., pp. 52-60, 1993.
[78] B. Lientz and B. Swanson, Software Maintenance Management. Addison-Wesley, 1981.
[79] R. Lincke and W. Lowe, “Compendium of Software Quality Standards and Metrics—Version 1.0,” technical report, Vaxjo Universitet, 2007.
[80] A. MacCormack, J. Rusnak, and C.Y. Baldwin, “Exploring the Structure of Complex Software Designs: An Empirical Study of Open Source and Proprietary Code,” Management Science, no. 52, pp. 1015-1030, July 2006.
[81] T.J. McCabe, “A Complexity Measure,” Proc. Second Int'l Conf. Software Eng., p. 407, 1976.
[82] T. Mens and S. Demeyer, “Future Trends in Software Evolution Metrics,” Proc. Fourth Int'l Workshop Principles of Software Evolution, pp. 83-86, 2001.
[83] T. Mens and T. Tourwé, “A Survey of Software Refactoring,” IEEE Trans. Software Eng., vol. 30, no. 2, pp. 126-139, Feb. 2004.
[84] M. Michlmayr, “Managing Volunteer Activity in Free Software Projects,” Proc. USENIX Ann. Technical Conf., p. 39, 2004.
[85] A. Mockus and T.L. Graves, “Identifying Productivity Drivers by Modeling Work Units Using Partial Data,” Technometrics, vol. 42, no. 2, pp. 168-179, 2001.
[86] M. Monga, “From Bazaar to Kibbutz: How Freedom Deals with Coherence in the Debian Project,” Proc. Workshop Open Source Software Eng., pp. 71-75, 2004.
[87] N. Nagappan, T. Ball, and A. Zeller, “Mining Metrics to Predict Component Failures,” Proc. 28th Int'l Conf. Software Eng., pp. 452-461, 2006.
[88] Open Source Initiative (OSI), http:/, 2008.
[89] E.I. Oviedo, “Control Flow, Data Flow and Programmers Complexity,” Proc. Int'l Computer Software and Applications Conf., pp. 146-152, 1980.
[90] R.E. Park, “Software Size Measurement: A Frame Work for Counting Source Code Statements,” Technical Report CMU/SEI-92-TR-020, Carnegie Mellon Univ., 1992.
[91] J.F. Ramil, “Continual Resource Estimation for Evolving Software,” Proc. Int'l Conf. Software Maintenance, pp. 289-293, 2003.
[92] E.S. Raymond, The Art of Unix Programming. Addison-Wesley Professional, 2003.
[93] P.S. Renz, Project Governance: Implementing Corporate Governance and Business Ethics in Nonprofit Organizations. Physica-Verl, 2007.
[94] D. Riehle, “The Economic Motivation of Open Source Software: Stakeholder Perspective,” Computer, vol. 40, no. 4, pp. 25-32, Apr. 2007.
[95] R. Robbes, “Mining a Change-Based Software Repository,” Proc. Fourth Int'l Workshop Mining Software Repositories, p. 15, 2007.
[96] L.H. Rosenberg, “Applying and Interpreting Object Oriented Metrics,” technical report, Software Assurance Technology Center NASA SATC, Apr. 1998.
[97] R.C. Sharble and S.S. Cohen, “The Object-Oriented Brewery: A Comparison of Two Object-Oriented Development Methods,” ACM SIGSOFT Software Eng. Notes, vol. 18, no. 2, pp. 60-73, 1993.
[98] S.A. Slaughter, D.E. Harter, and M.S. Krishnan, “Evaluating the Cost of Software Quality,” Comm. ACM, vol. 41, no. 8, pp. 67-73, Aug. 1998.
[99] S.A. Slaughter, J. Roberts, and I. Hann, “Communication Networks in an Open Source Software Project,” Proc. Int'l Conf. Open Source Systems, pp. 297-306, 2006.
[100] J. Sonnenfeld, “Good Governance and the Misleading Myths of Bad Metrics,” Academy of Management Executives, vol. 18, no. 1, pp.108-113, 2004.
[101] R. Subramanyam and M.S. Krishnan, “Empirical Analysis of CK Metrics for Object Oriented Design Complexity: Implications for Software Defects,” IEEE Trans. Software Eng., vol. 29, no. 4, pp.297-310, Apr. 2003.
[102] C.R. Symons, “Function Point Analysis: Difficulties and Improvements,” IEEE Trans. Software Eng., vol. 14, no. 1, pp. 2-11, Jan. 1988.
[103] Y. Tan and V. Mookerjee, “Comparing Uniform and Flexible Policies for Software Maintenance and Replacement,” IEEE Trans. Software Eng., vol. 31, no. 3, pp. 238-256, Mar. 2005.
[104] I. Tervonen and P. Kerola, “Towards Deeper Co-Understanding of Software Quality,” Information and Software Technology, vol. 39, no. 14-15, pp. 995-1003, 1998.
[105] A. Tille, Custom Debian Distributions, talks/paper-cdddebian-cdd.en.pdf, Jan. 2007.
[106] D. Troy and S. Zweben, “Measuring the Quality of Structured Design,” McGraw-Hill Int'l Series in Software Eng., pp. 214-226, 1993.
[107] L.R. Tucker and C. Lewis, “A Reliability Coefficient for Maximum Likelihood Factor Analysis,” Psychometrika, vol. 38, no. 11-10, 1973.
[108] M. Van Genuchten, “Why Is Software Late? An Empirical Study of Reasons for Delay in Software Development,” IEEE Trans. Software Eng., vol. 17, no. 6, pp. 582-590, 1991.
[109] C. Wohlin, P. Runeson, M. Höst, M.C. Ohlsson, B. Regnell, and A. Wesslén, Experimentation in Software Engineering: An Introduction. Springer, 2000.
[110] Y. Zhou and H. Leung, “Empirical Analysis of Object Oriented Design Metrics for Predicting High and Low Severity Faults,” IEEE Trans. Software Eng., vol. 32, no. 6, pp. 771-789, June 2006.
[111] T. Zimmermann, R. Premraj, and A. Zeller, “Prediction Defects for Eclipse Make to Fix This Bug,” Proc. Workshop Predictor Models in Software Eng. (PROMISE), 2007.

Index Terms:
Cost estimation, Qualitative process analysis, Organizational management and coordination, Management, Complexity measures, Process metrics, Metrics/Measurement, Software Engineering, Software/Software Engineering
Eugenio Capra, Chiara Francalanci, Francesco Merlo, "An Empirical Study on the Relationship Between Software Design Quality, Development Effort and Governance in Open Source Projects," IEEE Transactions on Software Engineering, vol. 34, no. 6, pp. 765-782, Nov.-Dec. 2008, doi:10.1109/TSE.2008.68
Usage of this product signifies your acceptance of the Terms of Use.