This Article 
 Bibliographic References 
 Add to: 
Reasons for Software Effort Estimation Error: Impact of Respondent Role, Information Collection Approach, and Data Analysis Method
December 2004 (vol. 30 no. 12)
pp. 993-1007
This study aims to improve analyses of why errors occur in software effort estimation. Within one software development company, we collected information about estimation errors through: 1) interviews with employees in different roles who are responsible for estimation, 2) estimation experience reports from 68 completed projects, and 3) statistical analysis of relations between characteristics of the 68 completed projects and estimation error. We found that the role of the respondents, the data collection approach, and the type of analysis had an important impact on the reasons given for estimation error. We found, for example, a strong tendency to perceive factors outside the respondents' own control as important reasons for inaccurate estimates. Reasons given for accurate estimates, on the other hand, typically cited factors that were within the respondents' own control and were determined by the estimators' skill or experience. This bias in types of reason means that the collection only of project managers' viewpoints will not yield balanced models of reasons for estimation error. Unfortunately, previous studies on reasons for estimation error have tended to collect information from project managers only. We recommend that software companies combine estimation error information from in-depth interviews with stakeholders in all relevant roles, estimation experience reports, and results from statistical analyses of project characteristics.

[1] K. Moløkken and M. Jørgensen, “A Review of Surveys on Software Effort Estimation,” Proc. IEEE Int'l Symp. Empirical Software Eng. (ISESE 2003), pp. 223-230, 2003.
[2] D. Phan, D. Vogel, and J. Nunamaker, “The Search for Perfect Project Management,” Computerworld, pp. 95-100, 1988.
[3] M. van Genuchten, “Why is Software Late? An Empirical Study of Reasons for Delay in Software Development,” IEEE Trans. Software Eng., vol. 17, no. 6, pp. 582-590, June 1991.
[4] A.L. Lederer and J. Prasad, “Causes of Inaccurate Software Development Cost Estimates,” J. Systems and Software, vol. 31, no. 2, pp. 125-134, 1995.
[5] G.H. Subramanian and S. Breslawski, “An Empirical Analysis of Software Effort Estimate Alterations,” J. Systems and Software, vol. 31, no. 2, pp. 135-141, 1995.
[6] A.L. Lederer and J. Prasad, “Information Systems Software Cost Estimating: A Current Assessment,” J. Information Technology, vol. 8, no. 1, pp. 22-33, 1993.
[7] A. Gray, S. MacDonnell, and M. Shepperd, “Factors Systematically Associated with Errors in Subjective Estimates of Software Development Effort: The Stability of Expert Judgment,” Proc. Sixth Int'l Software Metrics Symp., pp. 216-227, 1999.
[8] H.J. Thambain and D.L. Wilemon, “Criteria for Controlling Projects According to Plan,” Project Management J., pp. 75-81, June 1986.
[9] D.W.M. Chan and M.M. Kumaraswamy, “A Comparative Study of Causes of Time Overruns in Hong Kong Construction Projects,” Int'l J. Project Management, vol. 15, no. 1, pp. 55-63, 1997.
[10] H.-T. Tan and M.G. Lipe, “Outcome Effects: The Impact of Decision Process and Outcome Controllability,” J. Behavioral Decision Making, vol. 10, no. 4, pp. 315-325, 1997.
[11] J. Pearl, Causality: Models, Reasoning, and Inference. Cambridge Univ. Press, 2000.
[12] P. Brickman, K. Ryan, and C.B. Wortman, “Causal Chains: Attribution of Responsibility as a Function of Immediate and Prior Causes,” J. Personality and Social Psychology, vol. 32, pp. 1060-1067, 1975.
[13] K.R. Hammond, Human Judgement and Social Policy: Irreducible Uncertainty, Inevitable Error, Unavoidable Injustice. New York: Oxford Univ. Press, 1996.
[14] B. Brehmer, “In One Word: Not from Experience,” Acta Psychologica, vol. 45, pp. 223-241, 1980.
[15] M. Jørgensen, “A Review of Studies on Expert Estimation of Software Development Effort,” J. Systems and Software, vol. 70, nos. 1-2, pp. 37-60, 2004.
[16] B. Kitchenham, S.L. Pfleeger, B. McColl, and S. Eagan, “An Empirical Study of Maintenance and Development Estimation Accuracy,” J. Systems and Software, vol. 64, no. 1, pp. 57-77, 2002.
[17] B. Flyvebjerg, B.S. Holm, and S. Buhl, “Underestimating Costs in Public Works Projects,” J. Am. Planning Assoc., vol. 68, no. 3, pp. 279-295, 2002.
[18] A.L. Lederer and J. Prasad, “The Validation of a Political Model of Information Systems Development Cost Estimating,” ACM Trans. Computer Personnel, vol. 13, no. 2, pp. 47-57, 1991.
[19] C. Argyris, On Organizational Learning. Cambridge, Mass.: Blackwell, 1993.
[20] M. Jørgensen, “Regression Models of Software Development Effort Estimation Accuracy and Bias,” Empirical Software Eng., vol. 9, no. 4, 2004.
[21] M. Jørgensen, “Realism in Assessment of Effort Estimation Uncertainty: It Matters How You Ask,” IEEE Trans. Software Eng., vol. 30, no. 4, pp. 209-217, Apr. 2004.
[22] C. Johnson, “The Causes of Human Error in Medicine,” Cognition, Technology & Work, vol. 4, pp. 65-70, 2002.

Index Terms:
Cost estimation, review and evaluation, performance evaluation.
Magne J?rgensen, Kjetil Mol?kken-?stvold, "Reasons for Software Effort Estimation Error: Impact of Respondent Role, Information Collection Approach, and Data Analysis Method," IEEE Transactions on Software Engineering, vol. 30, no. 12, pp. 993-1007, Dec. 2004, doi:10.1109/TSE.2004.103
Usage of this product signifies your acceptance of the Terms of Use.