This Article 
 Bibliographic References 
 Add to: 
Measurement Programs in Software Development: Determinants of Success
September 2002 (vol. 28 no. 9)
pp. 863-875

Abstract—Measurement programs in software organizations are an important source of control over quality and cost in software development. The findings of this research presented here are based on an industry-wide survey conducted to examine the factors that influence success in software metrics programs. Our approach is to go beyond the anecdotal information on metrics programs that exists in the literature and use the industry-wide survey data to rigorously test for the effects of various factors that affect metrics programs success. We measure success in metrics programs using two variables—use of metrics information in decision-making and improved organizational performance. The various determinants of metrics program success are divided into two sets—organizational variables and technical variables. The influence of these variables on metrics programs success is tested using regression analysis. Our results support some of the factors discussed in the anecdotal literature such as management support, goal alignment, and communication and feedback. Certain other factors such as metrics quality and the ease of data collection are not as strongly influential on success. We conclude the paper with a detailed discussion of our results and suggestions for future work.

[1] I. Aaen, A. Siltanen, C. Sorensen, and V.-P. Tahvanainen, “A Tale of Two Countries: CASE Experiences and Expectations,” The Impact of Computer Supported Technologies on Information Systems Development, IFIP Transactions, K.E. Kendall, K. Lyytinen, and J. DeGross, eds., pp. 61-91, 1992.
[2] I. Aaen, “Problems in CASE Introduction: Experiences from User Organizations,” Information and Software Technology, vol. 36, pp. 643-654, Nov. 1994.
[3] J.V. Baldridge and R.A. Burnham, “Organizational Innovation: Individual, Organizational, and Environmental Impacts,” Administrative Science Quarterly, vol. 20, pp. 165-176, June 1975.
[4] V.R. Basili and D.M. Weiss, “A Methodology for Collecting ValidSoftware Engineering Data,” IEEE Trans. Software Eng., vol. 10, pp. 728-738, 1984.
[5] J.C. Brancheau and J.C. Wetherbe, “The Adoption of Spreadsheet Software: Testing Innovation Diffusion Theory in the Context of End-User Computing,” Information Systems Research, vol. 1, pp. 115-143, June 1990.
[6] L.C. Briand, C.M. Differding, and H.D. Rombach, “Practical Guidelines for Measurement-Based Process Improvement,” Software Process—Improvement and Practice, vol. 2, pp. 253-280, 1996.
[7] M.K. Daskalantonakis, "A Practical View of Software Management and Implementation Experiences within Motorola," IEEE Trans. Software Eng. Vol. 18, No. 11, 1992, pp. 998-1009.
[8] S. Fenick, “Implementing Management Metrics: An Army Program,” IEEE Software, pp. 65-72, Mar. 1990.
[9] P. Finlay and A.C. Mitchell, “Perceptions of the Benefits from the Introduction of CASE: An Empirical Study,” MIS Quarterly, vol. 18, pp. 353-370, Dec. 1994.
[10] T. Hall and N. Fenton, “Implementing Effective Software Metrics Programs,” IEEE Software, pp. 55-65, Mar./Apr. 1997.
[11] J. Iivari, “Why are CASE Tools Not Used?” Comm. ACM, vol. 39, pp. 94-103, Oct. 1996.
[12] J. Iversen and L. Mathiassen, “Lessons from Implementing a Software Metrics Program,” Proc. Hawaii Int'l Conf. System Sciences (HICSS-33), 2000.
[13] R. Jefferey and M. Berry, "A Framework for Evaluation and Prediction of Metrics Program Success," 1st Int'l Software Metrics Symp., IEEE Computer Soc. Press, Los Alamitos, Calif., 1993, pp. 28-39.
[14] C.F. Kemerrer, “How the Learning Curve Affects CASE Tool Adoption,” IEEE Software, vol. 9, pp. 23-28, 1992.
[15] J. Kimberley and M.J. Evanisko, “Organizational Innovation: The Influence of Individual, Organizational, and Contextual Factors on Hospital Adoption of Technological and Administrative Innovations,” Academy of Management J., vol. 24, pp. 689-713, Dec. 1981.
[16] J. Kmenta, Elements of Econometrics. New York: MacMillan Publishing, 1989.
[17] S.K. Majumdar and S. Venkataraman, “New Technology Adoption in US Telecommunications: The Role of Competitive Pressures and Firm-Level Inducements,” Research Policy, vol. 22, pp. 521-536, 1993.
[18] S.K. Majumdar, “Does New Technology Adoption Pay? Electronic Switching Patterns and Firm-Level Performance in US Telecommunications,” Research Policy, vol. 24, pp. 803-822, 1995.
[19] J. Mayes, “Achieving Business Objectives III: A Real-World Software Process Improvement Implementation,” IT Metrics Strategies, vol. VI, no. 8, Aug. 2000.
[20] J.W. Meyer and B. Rowan, “Institutionalized Organizations: Formal Structure as Myth and Ceremony,” Am. J. Sociology, vol. 83, pp. 340-363, 1983.
[21] R.J. Offen and R. Jeffery, “Establishing Software Measurement Programs,” IEEE Software, pp. 45-53, Mar./Apr. 1997.
[22] M.C. Paulk, B. Curtis, M.B. Chrissis, and C.V. Weber, “Capability Maturity Model for Software, Version 1.1,” Technical Report SEI-93-TR-24, Software Eng. Inst., Carnegie Mellon Univ., 1993.
[23] S.L. Pfleeger, "Lessons Learned in Building a Corporate Metrics Program," IEEE Software, May 1993, pp. 67-74.
[24] G. Premkumar, K. Ramamurthy, and S. Nilakanta, “Implementation of Electronic Data Interchange: An Innovation Diffusion Perspective,” J. Management Information Systems, vol. 11, pp. 157-186, Aug. 1994.
[25] K. Ramamurthy and G. Premkumar, “Determinants and Outcomes of Electronic Data Interchange Diffusion,” IEEE Trans. Eng. Management, vol. 42, pp. 332-351, Nov. 1995.
[26] L. Rosenberg and L. Hyatt, “Developing a Successful Metrics Program,” Proc. Eighth Ann. Software Technology Conf., Apr. 1996.
[27] R. Rosenthal and R.L. Rosnow, Essentials of Behavioral Research: Methods and Data Analysis, second ed. McGraw-Hill, 1991.
[28] C. Seddio, “Integrating Test Metrics within a Software Engineering Measurement Program at Eastman Kodak Company: A Follow-Up Case Study,” J. Systems&Software, vol. 20, pp. 227-235, 1993.

Index Terms:
Software metrics programs, empirical methods, survey, regression analysis, software development, measurement programs, determinants of success, software engineering.
Anandasivam Gopal, M.S. Krishnan, Tridas Mukhopadhyay, Dennis R. Goldenson, "Measurement Programs in Software Development: Determinants of Success," IEEE Transactions on Software Engineering, vol. 28, no. 9, pp. 863-875, Sept. 2002, doi:10.1109/TSE.2002.1033226
Usage of this product signifies your acceptance of the Terms of Use.