This Article 
 Bibliographic References 
 Add to: 
An Empirical Investigation of the Key Factors for Success in Software Process Improvement
May 2005 (vol. 31 no. 5)
pp. 410-424
Tore Dyb?, IEEE Computer Society
Understanding how to implement software process improvement (SPI) successfully is arguably the most challenging issue facing the SPI field today. The SPI literature contains many case studies of successful companies and descriptions of their SPI programs. However, the research efforts to date are limited and inconclusive and without adequate theoretical and psychometric justification. This paper extends and integrates models from prior research by performing an empirical investigation of the key factors for success in SPI. A quantitative survey of 120 software organizations was designed to test the conceptual model and hypotheses of the study. The results indicate that success depends critically on six organizational factors, which explained more than 50 percent of the variance in the outcome variable. The main contribution of the paper is to increase the understanding of the influence of organizational issues by empirically showing that they are at least as important as technology for succeeding with SPI and, thus, to provide researchers and practitioners with important new insights regarding the critical factors of success in SPI.

[1] P. Abrahamsson, “Is Management Commitment a Necessity after All in Software Process Improvement?” Proc. 26th Euromicro Conf., vol. 2, pp. 246-253, Sept. 2000.
[2] P. Abrahamsson, “Commitment Development in Software Process Improvement: Critical Misconceptions,” Proc. 23rd Int'l Conf. Software Eng. (ICSE 2001), pp. 71-80, 2001.
[3] S.L Ahire, D.Y. Golhar, and M.A. Waller, “Development and Validation of TQM Implementation Constructs,” Decision Sciences, vol. 27, no. 1, pp. 23-56, 1996.
[4] A. Anastasi and S. Urbina, Psychological Testing, seventh ed. Upper Saddle River, New Jersey: Prentice-Hall, 1997.
[5] C. Argyris and D.A. Schön, Organizational Learning II: Theory, Method, and Practice, Reading, Mass.: Addison-Wesley, 1996.
[6] Y. Baruch, “Response Rate in Academic Studies— A Comparative Analysis,” Human Relations, vol. 52, no. 4, pp. 421-438, 1999.
[7] V.R. Basili and G. Caldiera, “Improve Software Quality by Reusing Knowledge and Experience,” Sloan Management Rev., vol. 37, no. 1, pp. 55-64, Autumn 1995.
[8] V.R. Basili and H.D. Rombach, “The TAME Project: Towards Improvement-Oriented Software Environments,” IEEE Trans. Software Eng., vol. 14, no. 6, pp. 758-773, 1988.
[9] V.R. Basili, F.E. McGarry, R. Pajerski, and M.V. Zelkowitz, “Lessons Learned from 25 Years of Process Improvement: The Rise and Fall of the NASA Software Engineering Laboratory,” Proc. 24th Int'l Conf. Software Eng. (ICSE 2002), pp. 69-79, 2002.
[10] M. Beer, R.A. Eisenstat, and B. Spector, “Why Change Programs Don't Produce Change,” Harvard Business Rev., vol. 68, no. 6, pp. 158-166, 1990.
[11] P.L. Berger and T. Luckmann, The Social Construction of Reality: A Treatise in the Sociology of Knowledge. Harmondsworth: Penguin Books, 1966.
[12] M. Berry and R. Jeffery, “An Instrument for Assessing Software Measurement Programs,” Empirical Software Eng., vol. 5, no. 3, pp. 183-200, Nov. 2000.
[13] M. Biró and C. Tully, “The Software Process in the Context of Business Goals and Performance,” Better Software Practice for Business Benefit: Principles and Experience, R. Messnarz and C. Tully, eds. CS Press, pp. 15-27, 1999.
[14] M. Biró, R. Messnarz, and A.G. Davison, “The Impact of National Cultural Factors on the Effectiveness of Process Improvement Methods: The Third Dimension,” ASQ Software Quality Professional, vol. 4, no. 4, pp. 34-41, 2002.
[15] S.A. Black and L.J. Porter, “Identification of the Critical Factors of TQM,” Decision Sciences, vol. 27, no. 1, pp. 1-21, 1996.
[16] L.C. Briand, K. El Emam, and S. Morasca, “On the Application of Measurement Theory in Software Engineering,” Empirical Software Eng., vol. 1, no. 1, pp. 61-88, 1996.
[17] J.G. Brodman and D.L. Johnson, “Return on Investment (ROI) from Software Process Improvement as Measured by US Industry,” Software Process Improvement and Practice, pp. 35-47, 1995.
[18] J.S. Brown and P. Duguid, “Organizational Learning and Communities of Practice: Toward a Unified View of Working, Learning, and Innovation,” Organization Science, vol. 2, no. 1, pp. 40-57, 1991.
[19] R.B. Cattell, “The Scree Test for the Number of Factors,” Multivariate Behavioral Research, vol. 1, pp. 245-276, 1966.
[20] J. Cohen, Statistical Power Analysis for the Behavioral Sciences, second ed. Hillsdale, New Jersey: Laurence Erlbaum, 1988.
[21] A.L. Comrey and H.B. Lee, A First Course on Factor Analysis, second ed. Hillsdale, N.J.: Erlbaum, 1992.
[22] R. Conradi and T. Dybå, “An Empirical Study on the Utility of Formal Routines to Transfer Knowledge and Experience,” Proc. Joint Eighth European Software Eng. Conf. (ESEC) and Ninth ACM SIGSOFT Int'l Symp. Foundations of Software Eng. (FSE), Sept. 2001.
[23] L.J. Cronbach, “Coefficient Alpha and the Internal Consistency of Tests,” Psychometrica, vol. 16, pp. 297-334, 1951.
[24] L.J. Cronbach, “Test Validation,” Educational Measurement, R.L. Thorndike, ed., second ed., Washington: Am. Council on Education, pp. 443-507, 1971.
[25] P.B. Crosby, Quality is Still Free: Making Quality Certain in Uncertain Times. New York: McGraw-Hill, 1996.
[26] T.G. Cummings and C.G. Worley, Organization Development and Change, eighth ed. Cincinnati, Ohio: South-Western College Publishing, 2004.
[27] M.K. Daskalantonakis, “A Practical View of Software Measurement and Implementation Experiences within Motorola,” IEEE Trans. Software Eng., vol. 18, no. 11, pp. 998-1010, 1992.
[28] T.H. Davenport, Process Innovation: Reengineering Work through Information Technology. Boston, Mass.: Harvard Business School Press, 1993.
[29] C. Debou and A. Kuntzmann-Combelles, “Linking Software Process Improvement to Business Strategies: Experience from Industry,” Software Process: Improvement and Practice, vol. 5, pp. 55-64, 2000.
[30] C. Debou, D. Courtel, H.-B. Lambert, N. Fuchs, and M. Haux, “Alcatel's Experience with Process Improvement,” Better Software Practice for Business Benefit: Principles and Experience, R. Messnarz and C. Tully, eds., CS Press, pp. 281-301, 1999.
[31] C. Deephouse, T. Mukhopadhyay, D.R. Goldenson, and M.I. Kellner, “Software Processes and Project Performance,” J. Management Information Systems, vol. 12, no. 3, pp. 187-205, 1996.
[32] T. DeMarco, Controlling Software Projects: Management, Measurement and Estimation. New York: Yourdon Press, 1982.
[33] W.E. Deming, Out of the Crisis. Cambridge, Mass.: MIT Center for Advanced Eng. Study, 1986.
[34] R. Dion, “Process Improvement and the Corporate Balance Sheet,” IEEE Software, vol. 10, no. 4, pp. 28-35, 1993.
[35] T. Dybå, “An Instrument for Measuring the Key Factors of Success in Software Process Improvement,” Empirical Software Eng., vol. 5, no. 4, pp. 357-390, 2000.
[36] T. Dybå, “Improvisation in Small Software Organizations,” IEEE Software, vol. 17, no. 5, pp. 82-87, Sept.-Oct. 2000.
[37] T. Dybå, “A Dynamic Model of Software Engineering Knowledge Creation,” Managing Software Eng. Knowledge, A. Aurum et al., eds., Berlin: Springer Verlag, pp. 95-117, 2003.
[38] T. Dybå, “Factors of Software Process Improvement Success in Small and Large Organizations: An Empirical Study in the Scandinavian Context,” Proc. Joint Ninth European Software Eng. Conf. (ESEC) and 11th SIGSOFT Symp. Foundations Software Eng. (FSE-11), pp. 148-157, 2003.
[39] T. Dybå and N.B. Moe, “Rethinking the Concept of Software Process Assessment,” Proc. European Software Process Improvement Conf. (EuroSPI '99), Oct. 1999.
[40] T. Dybå, T. Dingsøyr, and N.B. Moe, Process Improvement in Practice: A Handbook for IT Companies, The Kluwer Int'l Series in Software Eng., Boston: Kluwer Academic Publishers, 2004.
[41] P. Ehn, “Scandinavian Design: On Participation and Skill,” Usability— Turning Technologies into Tools, P.S. Adler and T.A. Winograd, eds., New York: Oxford Univ. Press, pp. 96-132, 1992.
[42] SPICE: The Theory and Practice of Software Process Improvement and Capability Determination, K. El Emam et al., eds., CS Press, 1998.
[43] K. El Emam, P. Fusaro, and B. Smith, “Success Factors and Barriers for Software Process Improvement,” Better Software Practice for Business Benefit: Principles and Experience, R. Messnarz and C. Tully, eds., CS Press, pp. 355-371, 1999.
[44] K. El Emam, D.R. Goldenson, J. McCurley, and J. Herbsleb, “Modeling the Likelihood of Software Process Improvement: An Exploratory Study,” Empirical Software Eng., vol. 6, no. 3, pp. 207-229, 2001.
[45] N.E. Fenton and S.H. Pfleeger, Software Metrics: A Rigorous and Practical Approach. London: Int'l Thomson Computer Press, 1996.
[46] W.L. French and C.H. Bell Jr., Organization Development: Behavioral Science Interventions for Organization Improvement, sixth ed. Upper Saddle River, New Jersey: Prentice-Hall, 1999.
[47] D.R. Goldenson and J.D. Herbsleb, “After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success,” Technical Report, CMU/SEI-95-TR-009, Carnegie Mellon Univ., Software Eng. Inst., 1995.
[48] R.B. Grady, Successful Software Process Improvement. Upper Saddle River, New Jersey: Prentice-Hall, 1997.
[49] R.B. Grady and D. Caswell, Software Metrics: Establishing a Company-Wide Program. Englewood Cliffs, New Jersey: Prentice-Hall, 1987.
[50] D.C. Hambrick, M.A. Geletkanycz, and J.W. Fredrickson, “Top Executive Commitment to the Status Quo: Some Tests for Its Determinants,” Strategic Management J., vol. 14, no. 6, pp. 401-418, 1993.
[51] M. Hammer, Beyond Reengineering: How the Process-Centered Organization is Changing Our Work and Our Lives. London: HarperCollins, 1996.
[52] M. Hammer and J. Champy, Reengineering the Corporation: A Manifesto for Business Revolution. New York: Harper Business, 1993.
[53] W.L. Hays, Statistics, fifth ed. New York: Harcourt Brace, 1994.
[54] J.D. Herbsleb and D.R. Goldenson, “A Systematic Survey of CMM Experience and Results,” Proc. 18th Int'l Conf. Software Eng. (ICSE-18), pp. 323-330, 1996.
[55] W.S. Humphrey, Managing the Software Process. Reading, Mass.: Addison-Wesley, 1989.
[56] W.S. Humphrey, Managing Technical People: Innovation, Teamwork, and the Software Process. Reading, Mass.: Addison-Wesley, 1997.
[57] W.S. Humphrey, T. Snyder, and R. Willis, “Software Process Improvement at Hughes Aircraft,” IEEE Software, vol. 8, no. 4, pp. 11-23, 1991.
[58] Juran's Quality Handbook, J.M. Juran and A.B. Godfrey, eds., fifth ed., New York: McGraw-Hill, 1999.
[59] H.F. Kaiser, “A Second Generation Little Jiffy,” Psychometrika, vol. 35, pp. 401-417, 1970.
[60] R.S. Kaplan and D.P. Norton, The Balanced Scorecard: Translating Strategy into Action. Boston, Mass.: Harvard Business School Press, 1996.
[61] B.A. Kitchenham, Software Metrics: Measurement for Software Process Improvement. Oxford, England: NCC Blackwell, 1996.
[62] S. Konishi, “Normalizing Transformations of some Statistics in Multivariate Analysis,” Biometrika, vol. 68, no. 3, pp. 647-651, 1981.
[63] J. Lave and E. Wenger, Situated Learning: Legitimate Peripheral Participation. Cambridge: Cambridge Univ. Press, 1991.
[64] J.G. March, “Exploration and Exploitation in Organizational Learning,” Organization Science, vol. 2, no. 1, pp. 71-87, 1991.
[65] J.G. March, The Pursuit of Organizational Intelligence. Malden, Mass.: Blackwell, 1999.
[66] J.G. March and R.I. Sutton, “Organizational Performance as a Dependent Variable,” Organization Science, vol. 8, no. 6, pp. 698-706, 1997.
[67] J. Mart, Cultures in Organizations: Three Perspectives. New York: Oxford Univ. Press, 1992.
[68] E. Mayo, The Human Problems of an Industrial Civilization. Boston: Harvard Univ. Press, 1933.
[69] E. Mayo, The Social Problems of an Industrial Civilization. Boston: Harvard Univ. Press, 1945.
[70] B. McFeeley, IDEAL: A User's Guide for Software Process Improvement, Handbook, CMU/SEI-96-HB-01, Carnegie Mellon Univ., Software Eng. Inst., 1996.
[71] T. Mehner, “Siemens Process Assessment Approach,” Better Software Practice for Business Benefit: Principles and Experience, R. Messnarz and C. Tully, eds., CS Press, pp. 199-212, 1999.
[72] R. Messnarz, “Summary and Outlook,” Better Software Practice for Business Benefit: Principles and Experience, R. Messnarz and C. Tully, eds., CS Press, pp. 389-393, 1999.
[73] D.A. Nadler, Feedback and Organization Development: Using Data-Based Methods. Reading, Mass.: Addison-Wesley, 1977.
[74] I. Nonaka, “A Dynamic Theory of Organizational Knowledge Creation,” Organization Science, vol. 5, no. 1, pp. 14-37, 1994.
[75] I. Nonaka and H. Takeuchi, The Knowledge-Creating Company: How Japanese Companies Create the Dynamics of Innovation. New York: Oxford Univ. Press, 1995.
[76] J.C. Nunnally and I.A. Bernste, Psychometric Theory, third ed. New York: McGraw Hill, 1994.
[77] T.C. Powell, “Total Quality Management as Competitive Advantage: A Review and Empirical Study,” Strategic Management J., vol. 16, no. 1, pp. 15-37, 1995.
[78] K. Pulford, A. Kuntzmann-Combelles, and S. Shirlaw, A Quantitative Approach to Software Management: The AMI Handbook. Wokingham, England: Addison-Wesley, 1996.
[79] A. Rainer and T. Hall, “Key Success Factors for Implementing Software Process Improvement: A Maturity-Based Analysis,” J. Systems and Software, vol. 62, no. 2, pp. 71-84, 2002.
[80] S. Rifkin, “What Makes Measuring Software So Hard?” IEEE Software, vol. 18, no. 3, pp. 41-45, 2001.
[81] J. Rooijmans, H. Aerts, and M. van Genuchten, “Software Quality in Consumer Electronics Products,” IEEE Software, vol. 13, no. 1, pp. 55-64, 1996.
[82] J.V. Saraph, P.G. Benson, and R.G. Schroeder, “An Instrument for Measuring the Critical Factors of Quality Management,” Decision Sciences, vol. 20, no. 4, pp. 810-829, 1989.
[83] E.H. Schein, “Culture: The Missing Concept in Organization Studies,” Administrative Science Quarterly, vol. 41, pp. 229-240, 1996.
[84] P.M. Senge, The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday, 1990.
[85] H.A. Simon, “Bounded Rationality and Organizational Learning,” Organization Science, vol. 2, no. 1, pp. 125-134, 1991.
[86] D. Stelzer and W. Mellis, “Success Factors of Organizational Change in Software Process Improvement,” Software Process — Improvement and Practice, vol. 4, no. 4, pp. 227-250, 1998.
[87] D. Stelzer, W. Mellis, and G. Herzwurm, “Software Process Improvement via ISO 9000? Results of Two Surveys among European Software Houses,” Proc. 29th Hawaii Int'l Conf. Systems Sciences, Jan. 1996.
[88] J. Stevens, Applied Multivariate Statistics for the Social Sciences, fourth ed. Mahwah, N.J.: Lawrence Erlbaum, 2002.
[89] D.W. Straub, “Validating Instruments in MIS Research,” MIS Quarterly, vol. 13, no. 2, pp. 147-169, 1989.
[90] F.W. Taylor, The Principles of Scientific Management. Newton Library Harper & Row, 1911.
[91] E. Thorsrud, B. Sørensen, and B. Gustavsen, “Sociotechnical Approach to Industrial Democracy in Norway,” Handbook of Work Organization and Society, R. Dubin, ed., Chicago: Rand McNally, pp. 648-687, 1976.
[92] E. Trist, “The Evolution of Socio-Technical Systems: A Conceptual Framework and an Action Research Program,” Occasional Papers No. 2, Toronto, Ontario: Ontario Quality of Working Life Center, 1981.
[93] R. van Solingen and E. Berghout, The Goal/Question/Metric Method: A Practical Guide for Quality Improvement of Software Development. London: McGraw-Hill, 1999.
[94] P.F. Velleman and L. Wilkinson, “Nominal, Ordinal, Interval, and Ratio Typologies are Misleading,” The Am. Statistician, vol. 47, no. 1, pp. 65-72, 1993.
[95] M. Weber, C. Wright Mills, and H.H. Gerth, From Max Weber: Essays in Sociology. Oxford: Oxford Univ. Press, 1958.
[96] E. Wenger, Communities of Practice: Learning, Meaning, and Identity. Cambridge: Cambridge Univ. Press, 1998.
[97] S.M. Yusof and E. Aspinwall, “Critical Success Factors for Total Quality Management Implementation in Small and Medium Enterprises,” Total Quality Management, vol. 10, nos. 4-5, pp. 803-809, 1999.
[98] S. Zahran, Software Process Improvement: Practical Guidelines for Business Success. Harlow, England: Addison-Wesley, 1998.

Index Terms:
Index Terms- Empirical software engineering, software process improvement, critical success factors, organizational issues, survey research.
Tore Dyb?, "An Empirical Investigation of the Key Factors for Success in Software Process Improvement," IEEE Transactions on Software Engineering, vol. 31, no. 5, pp. 410-424, May 2005, doi:10.1109/TSE.2005.53
Usage of this product signifies your acceptance of the Terms of Use.