This Article 
 Bibliographic References 
 Add to: 
Optimal Project Feature Weights in Analogy-Based Cost Estimation: Improvement and Limitations
February 2006 (vol. 32 no. 2)
pp. 83-92
Cost estimation is a vital task in most important software project decisions such as resource allocation and bidding. Analogy-based cost estimation is particularly transparent, as it relies on historical information from similar past projects, whereby similarities are determined by comparing the projects' key attributes and features. However, one crucial aspect of the analogy-based method is not yet fully accounted for: the different impact or weighting of a project's various features. Current approaches either try to find the dominant features or require experts to weight the features. Neither of these yields optimal estimation performance. Therefore, we propose to allocate separate weights to each project feature and to find the optimal weights by extensive search. We test this approach on several real-world data sets and measure the improvements with commonly used quality metrics. We find that this method 1) increases estimation accuracy and reliability, 2) reduces the model's volatility and, thus, is likely to increase its acceptance in practice, and 3) indicates upper limits for analogy-based estimation quality as measured by standard metrics.

[1] C. Jones, Estimating Software Costs. McGraw-Hill, 1998.
[2] M. Shepperd and C. Schofield, “Estimating Software Project Effort Using Analogies,” IEEE Trans. Software Eng., vol. 23, no. 12, pp. 736-743, Nov. 1997.
[3] I. Wieczorek, “Improved Software Cost Estimation— A Robust and Interpretable Modeling Method and a Comprehensive Empirical Investigation,” PhD dissertation, Fraunhofer Inst. Für Experimentelles Software Eng., 2001.
[4] G. Finnie and G. Wittig, “A Comparison of Software Effort Estimation Techniques: Using Function Points with Neural Networks, Case Based Reasoning and Regression Models,” J. Systems Software, vol. 39, pp. 281-289, 1997.
[5] E. Mendes, I. Watson, C. Triggs, N. Mosley, and S. Counsell, “A Comparative Study of Cost Estimation Models for Web Hypermedia Applications,” Empirical Software Eng., vol. 8, pp. 163-196, 2003.
[6] M. Auer and S. Biffl, “Increasing the Accuracy and Reliability of Analogy-Based Cost Estimation Techniques with Extensive Project Feature Dimension Weighting,” Proc. ACM-IEEE Int'l Symp. Empirical Software Eng. (ISESE '04), Aug. 2004.
[7] B. Boehm, E. Horowitz, R. Madachy, D. Reifer, B. Clark, B. Steece, A. Brown, S. Chulani, and C. Abts, Software Cost Estimation with Cocomo II. Prentice Hall, 2000.
[8] J. Bode, “Decision Support with Neural Networks in the Management of Research and Development: Concepts and Application to Cost Estimation,” Information and Management, no. 34, pp. 33-40, 1998.
[9] C. Briand and V.R. Basili, “A Pattern Recognition Approach for Software Engineering Data Analysis,” IEEE Trans. Software Eng., vol. 18, no. 11, pp. 931-942, 1992.
[10] R. Hughes, “Expert Judgement as an Estimating Method,” Information and Software Technology, vol. 38, no. 2, pp. 67-75, 1996.
[11] M. Shepperd, C. Schofield, and B. Kitchenham, “Effort Estimation Using Analogy,” Proc. 18th Int'l Conf. Software Eng. (ICSE '96), pp. 170-178, Mar. 1996.
[12] L. Angelis and I. Stamelos, “A Simulation Tool for Efficient Analogy Based Cost Estimation,” Empirical Software Eng., vol. 5, pp. 35-68, 2000.
[13] I. Myrtveit and E. Stensrud, “A Controlled Experiment to Assess the Benefits of Estimating with Analogy and Regression Models,” IEEE Trans. Software Eng., vol. 25, no. 4, pp. 510-525, July/Aug. 1999.
[14] E. Mendes, N. Mosley, and S. Counsell, “Do Adaptation Rules Improve Web Cost Estimation?” Proc. 14th ACM Conf. Hypertext and Hypermedia (HYPERTEXT '03), Aug. 2003.
[15] M. Auer, B. Graser, and S. Biffl, “An Approach to Visualizing Empirical Software Project Portfolio Data Using Multidimensional Scaling,” Proc. IEEE Int'l Conf. Information Reuse and Integration (IRI '03), Oct. 2003.
[16] F. Walkerden and R. Jeffery, “An Empirical Study of Analogy-Based Software Effort Estimation,” Empirical Software Eng., vol. 4, pp. 135-158, 1999.
[17] T. Foss, E. Stensrud, B. Kitchenham, and I. Myrtveit, “A Simulation Study of the Model Evaluation Criterion MMRE,” IEEE Trans. Software Eng., vol. 29, no. 11, pp. 985-995, 2003.
[18] A. Albrecht and S. Gaffney, “Software Function, Source Lines of Code and Development Effort Prediction: A Software Science Validation,” IEEE Trans. Software Eng., vol. 9, no. 6, pp. 639-648, 1983.
[19] J. Desharnais, “Analyse Statistique de la Productivitie des Projets Informatique a Partie de la Technique des Point des Fonction,” Master's thesis, Univ. of Montreal, 1989.
[20] C. Kemerer, “An Empirical Validation of Software Cost Estimation Models,” Comm. ACM, pp. 416-429, May 1987.
[21] I. Wieczorek and M. Ruhe, “How Valuable is Company-Specific Data Compared to Multicompany Data for Software Cost Estimation?” Proc. Eighth Int'l Symp. Software Metrics (METRICS '02), pp. 237-248, June 2002.
[22] K. Maxwell and L.v. Wassenhove, “Software Development Productivity of European Space, Military, and Industrial Applications,” IEEE Trans. Software Eng., vol. 22, no. 10, pp. 706-718, 1996.

Index Terms:
Software cost estimation, analogy-based cost estimation, project clustering, project features.
Martin Auer, Adam Trendowicz, Bernhard Graser, Ernst Haunschmid, Stefan Biffl, "Optimal Project Feature Weights in Analogy-Based Cost Estimation: Improvement and Limitations," IEEE Transactions on Software Engineering, vol. 32, no. 2, pp. 83-92, Feb. 2006, doi:10.1109/TSE.2006.17
Usage of this product signifies your acceptance of the Terms of Use.