This Article 
 Bibliographic References 
 Add to: 
The Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: A Randomized Controlled Field Experiment
September/October 2011 (vol. 37 no. 5)
pp. 695-707
Magne Jørgensen, University of Oslo and Simula Research Laboratory, Lysaker
Stein Grimstad, University of Oslo and Simula Research Laboratory, Lysaker
Studies in laboratory settings report that software development effort estimates can be strongly affected by effort-irrelevant and misleading information. To increase our knowledge about the importance of these effects in field settings, we paid 46 outsourcing companies from various countries to estimate the required effort of the same five software development projects. The companies were allocated randomly to either the original requirement specification or a manipulated version of the original requirement specification. The manipulations were as follows: 1) reduced length of requirement specification with no change of content, 2) information about the low effort spent on the development of the old system to be replaced, 3) information about the client's unrealistic expectations about low cost, and 4) a restriction of a short development period with start up a few months ahead. We found that the effect sizes in the field settings were much smaller than those found for similar manipulations in laboratory settings. Our findings suggest that we should be careful about generalizing to field settings the effect sizes found in laboratory settings. While laboratory settings can be useful to demonstrate the existence of an effect and better understand it, field studies may be needed to study the size and importance of these effects.

[1] J. Aranda and S.M. Easterbroook, "Anchoring and Adjustment in Software Estimation," European Software Eng. Conf./ACM SIGSOFT Symp. Foundations of Software Eng., 2005.
[2] B. Boehm, C. Abts, A.W. Brown, S. Chulani, B.K. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece, Software Cost Estimation with Cocomo II. Prentice-Hall, 2000.
[3] B. Brehmer, "In One Word: Not from Experience," Acta Psychologica, vol. 45, pp 223-241, 1980.
[4] E. Cowley, "Processing Exaggerated Advertising Claims," J. Business Research, vol. 59, pp. 728-734, 2006.
[5] J.S. Edwards and T.T. Moores, "A Conflict between the Use of Estimating and Planning Tools in the Management of Information Systems," European J. Information Systems, vol. 3, no. 2, pp. 139-147, 1994.
[6] B. Englich, T. Mussweiler, and F. Strack, "Playing Dice with Criminal Sentences: The Influence of Irrelevant Anchors on Experts' Judicial Decision Making," Personality and Social Psychology Bull., vol. 32, no. 2, pp. 188-200, 2006.
[7] T. Gilovich, M. Kerr, and M.V. Medvec, "Effect of Temporal Perspective on Subjective Confidence," J. Personality and Social Psychology, vol. 64, pp. 552-560, 1993.
[8] P. Goodwin, "Enhancing Judgmental Sales Forecasting: The Role of Laboratory Research," Forecasting with Judgment, G. Wright and P. Goodwin, eds., pp. 91-112, John Wiley & Sons, 1998.
[9] H.P. Grice, "Logic and Conversation," Syntax and Semantics 3: Speech Acts, P. Cole and J.L. Morgan, eds., pp. 41-58, Academic Press, 1975.
[10] S. Grimstad and M. Jørgensen, "The Impact of Irrelevant Information on Estimates of Software Development Effort," Proc. Australian Software Eng. Conf., 2007.
[11] F.J. Heemstra and R.J. Kusters, "Function Point Analysis: Evaluation of a Software Cost Estimation Model," European J. Information Systems, vol. 1, no. 4, pp. 223-237, 1991.
[12] C.E Hewitt, N. Mitchell, and D.J. Torgerson, "Listen to the Data When Results Are Not Significant," British Medical J., vol. 336, no. 7634, pp. 23-25, 2008.
[13] J. Hihn and H. Habib-Agahi, "Cost Estimation of Software Intensive Projects: A Survey of Current Practices," Proc. Int'l Conf. Software Eng., 1991.
[14] M. Jørgensen, "A Review of Studies on Expert Estimation of Software Development Effort," J. Systems and Software, vol. 70, nos. 1/2, pp. 37-60, 2004.
[15] M. Jørgensen, "The 'Magic Step' of Judgment-Based Software Effort Estimation," Proc. Int'l Conf. Cognitive Economics, 2005.
[16] M. Jørgensen and G. Carelius, "An Empirical Study of Software Project Bidding," IEEE Trans. Software Eng., vol. 30, no. 12, pp. 953-969, Dec. 2004.
[17] M. Jørgensen and S. Grimstad, "Avoiding Irrelevant and Misleading Information when Estimating Development Effort," IEEE Software, vol. 25, no. 3, pp. 78-83, May/June 2008.
[18] M. Jørgensen and S. Grimstad, "Judgment-Updating among Software Professionals," Proc. Second Int'l Conf. Software Knowledge Information Management and Applications, 2008.
[19] M. Jørgensen and A.B. Kanten, "Temporal Distance and Software Development Effort Estimation," 2009.
[20] M. Jørgensen and D.I.K. Sjøberg, "Impact of Effort Estimates on Software Project Work," Information and Software Technology, vol. 43, no. 15, pp. 939-948, 2001.
[21] M. Jørgensen and D.I.K. Sjøberg, "The Impact of Customer Expectation on Software Development Effort Estimates," Int'l J. Project Management, vol. 22, pp. 317-325, 2004.
[22] D.J. Koehler, "Explanation, Imagination, and Confidence in Judgment," Psychological Bull., vol. 110, no. 3, pp. 499-519, 1991.
[23] A. Liberman, M. Sagristano, and Y. Trope, "The Effect of Temporal Distance on Level of Mental Construal," J. Experimental Social Psychology, vol. 38, pp. 523-535, 2002.
[24] N. Liberman and Y. Trope, "The Role of Feasibility and Desirability Considerations in Near and Distant Future Decisions: A Test of Temporal Construal Theory," J. Personality and Social Psychology, vol. 75, pp. 5-18, 1998.
[25] T. Mussweiler, "Comparison Processes in Social Judgment: Mechanisms and Consequences," Psychological Rev., vol. 110, no. 3, pp. 472-489, 2003.
[26] T. Mussweiler and F. Strack, "The Use of Category and Exemplar Knowledge in the Solution of Anchoring Tasks," J. Personality and Social Psychology, vol. 78, pp. 1038-1052, 2000.
[27] G.B. Northcraft and M.A. Neale, "Expert, Amateurs, and Real Estate: An Anchoring-and-Adjustment Perspective on Property Pricing Decisions," Organizational Behavior and Human Decision Processes, vol. 39, pp. 228-241, 1987.
[28] L.H. Putnam, "A General Empirical Solution to the Macro Software Sizing and Estimating Problem," IEEE Trans. Software Eng., vol. 4, no. 4, pp. 345-361, July 1978.
[29] N. Schwarz, "In Social Context: Biases, Shortcomings, and the Logic of Conversation," Advances in Experimental Social Psychology, M.P. Zanna, ed., pp. 125-162, Academic Press, 1994.
[30] Y. Trope and A. Liberman, "Social Hypothesis Testing: Cognitive and Motivational Factors," Social Psychology: Handbook of Basic Principles, E.T. Higgins and A.W. Kurglanski, eds., pp. 239-270, Guilford Press, 1996.
[31] Y. Trope and A. Liberman, "Temporal Construal," Psychological Rev., vol. 110, pp. 403-421, 2003.

Index Terms:
Cost estimation, software psychology, requirements/specifications.
Magne Jørgensen, Stein Grimstad, "The Impact of Irrelevant and Misleading Information on Software Development Effort Estimates: A Randomized Controlled Field Experiment," IEEE Transactions on Software Engineering, vol. 37, no. 5, pp. 695-707, Sept.-Oct. 2011, doi:10.1109/TSE.2010.78
Usage of this product signifies your acceptance of the Terms of Use.