The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - May/June (2008 vol.34)
pp: 407-432
Wojciech J. Dzidek , Simula Research Laboratory Simula Research Laboratory
Erik Arisholm , Simula Research Laboratory
Lionel C. Briand , Simula Research Laboratory
ABSTRACT
The Unified Modeling Language (UML) is the de facto standard for object-oriented software analysis and design modeling. However, few empirical studies exist that investigate the costs and evaluate the benefits of using UML in realistic contexts. Such studies are needed so that the software industry can make informed decisions regarding the extent to which they should adopt UML in their development practices. This is the first controlled experiment that investigates the costs of maintaining and the benefits of using UML documentation during the maintenance and evolution of a real, non-trivial system, using professional developers as subjects, working with a state-of-the-art UML tool during an extended period of time. The subjects in the control group had no UML documentation. In this experiment, the subjects in the UML group had on average a practically and statistically significant 54% increase in the functional correctness of changes (p=0.03), and an insignificant 7% overall improvement in design quality (p=0.22) - though a much larger improvement was observed on the first change task (56%) - at the expense of an insignificant 14% increase in development time caused by the overhead of updating the UML documentation (p=0.35).
INDEX TERMS
Maintainability, Distribution, Maintenance, and Enhancement, Software Engineering, Software/Software Engi, Design notations and documentation, Object-Oriented Programming
CITATION
Wojciech J. Dzidek, Erik Arisholm, Lionel C. Briand, "A Realistic Empirical Evaluation of the Costs and Benefits of UML in Software Maintenance", IEEE Transactions on Software Engineering, vol.34, no. 3, pp. 407-432, May/June 2008, doi:10.1109/TSE.2008.15
REFERENCES
[1] B. Anda, K. Hansen, I. Gullesen, and H.K. Thorsen, “Experiences from Using a UML-Based Development Method in a Large Organization,” Empirical Software Eng. J., vol. 11, pp. 555-581, 2006.
[2] K. Beck, Extreme Programming Explained. Addison-Wesley, 2000.
[3] R.S. Pressman, Software Engineering: A Practitioner's Approach, seventh ed. McGraw Hill, 2005.
[4] R. Glass, Facts and Fallacies of Software Engineering. Addison-Wesley, 2002.
[5] A. Kleppe, J. Warmer, and W. Bast, MDA Explained: The Model-Driven Architecture—Practice and Promise. Addison-Wesley, 2003.
[6] E. Arisholm, L.C. Briand, S.E. Hove, and Y. Labiche, “The Impact of UML Documentation on Software Maintenance: An Experimental Evaluation,” IEEE Trans. Software Eng., vol. 32, pp. 365-381, 2006.
[7] T. McGibbon, “Software Reliability Data Summary,” Data Analysis Center for Software, 1996.
[8] F. Shull, V. Basili, B. Boehm, A.W. Brown, P. Costa, M. Lindvall, D. Port, I. Rus, R. Tesoriero, and M. Zelkowitz, “What We Have Learned about Fighting Defects,” Proc. Eighth IEEE Int'l Symp. Software Metrics, 2002.
[9] “Borland Together for Eclipse,” Borland, 2004.
[10] B. Bruegge and A.H. Dutoit, Object-Oriented Software Engineering Using UML, Patterns, and Java, second ed. Prentice Hall, 2004.
[11] J. Holmes, Struts: The Complete Reference. McGraw-Hill, 2004.
[12] “JavaServer Pages 2.0 Specification,” Sun Microsystems, 2003.
[13] J. Gosling, The Java Language Specification, second ed. Addison-Wesley, 2000.
[14] T.A. Powell, HTML: The Complete Reference, third ed. Osborne/McGraw-Hill, 2001.
[15] “Eclipse,” IBM, 2004.
[16] M. Kofler, MySQL. Apress, 2001.
[17] “BESTweb,” Simula Research Laboratory, http://simula.noBESTweb/, 2004.
[18] M. Jørgensen and M. Shepperd, “A Systematic Review of Software Development Cost Estimation Studies,” IEEE Trans. Software Eng., vol. 33, no. 1, pp. 33-53, Jan. 2007.
[19] R. Binder, Testing Object-Oriented Systems: Models, Patterns, and Tools. Addison-Wesley, 2000.
[20] J. Tian, Software Quality Engineering: Testing, Quality Assurance, and Quantifiable Improvement. John Wiley & Sons/IEEE CS Press, 2005.
[21] J.L. Devore and N. Farnum, Applied Statistics for Engineers and Scientists. Duxbury, 1999.
[22] L.V. Garcia, “Escaping the Bonferroni Iron Claw in Ecological Studies,” Oikos, vol. 105, pp. 657-663, 2004.
[23] J. Cohen, Statistical Power Analysis for the Behavioral Sciences, second ed. L. Erlbaum Assoc., 1988.
[24] R.H. Myers, D.C. Montgomery, and G.G. Vining, Generalized Linear Models: With Applications in Engineering and the Sciences. J. Wiley, 2002.
[25] S.E. Hove and B. Anda, “Experiences from Conducting Semi-Structured Interviews in Empirical Software Engineering Research,” Proc. 11th IEEE Int'l Symp. Software Metrics, 2005.
[26] O.R. Holsti, Content Analysis for the Social Sciences and Humanities. Addison-Wesley, 1969.
[27] S.G. Eick, T.L. Graves, A.F. Karr, J.S. Marron, and A. Mockus, “Does Code Decay? Assessing the Evidence from Change Management Data,” IEEE Trans. Software Eng., vol. 27, pp. 1-12, 2001.
[28] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.
[29] J. Bloch, Effective Java Programming Language Guide. Prentice Hall, 2001.
[30] S. Meyers, Effective C++, second ed. Addison-Wesley, 1997.
[31] G.A. Miller, “The Magical Number Seven, Plus or Minus Two: Some Limits on Our Capacity for Processing Information,” The Psychological Rev., vol. 63, pp. 81-97, 1956.
[32] W.R. Shadish, T.D. Cook, and D.T. Campbell, Experimental and Quasi-Experimental Designs for Generalized Causal Inference. Houghton Mifflin, 2002.
[33] M. Fowler and K. Scott, UML Distilled: A Brief Guide to the Standard Object Modeling Language, second ed. Addison-Wesley, 2000.
[34] S. Tilley and S. Huang, “A Qualitative Assessment of the Efficacy of UML Diagrams as a Form of Graphical Documentation in Aiding Program Understanding,” Proc. ACM SIGDOC '03, pp.184-191, 2003.
[35] E. Tryggeseth, “Report from an Experiment: Impact of Documentation on Maintenance,” Empirical Software Eng. J., vol. 2, pp. 201-207, 1997.
[36] T.H. Huxley, “We are All Scientists,” The New Treasury of Science, H.Shapley, S. Rapport, and H. Wright, eds., Collins, 1965.
[37] A. Brooks, M. Roper, M. Wood, J. Daly, and J. Miller, “Replication's Role in Software Engineering,” Guide to Advanced Empirical Software Eng., F. Schull, J. Singer, and D. Sjoberg, eds., pp. 365-379, Springer Science, 2008.
[38] B. Curtis, “Measurement and Experimentation in Software Engineering,” Proc. IEEE, vol. 68, pp. 1144-1157, 1980.
[39] K.R. Popper, The Logic of Scientific Discovery. Hutchinson, 1968.
[40] D.I.K. Sjøberg, J.E. Hannay, O. Hansen, V.B. Kampenes, A. Karahasanovic, N.-K. Liborg, and A.C. Rekdal, “A Survey of Controlled Experiments in Software Engineering,” IEEE Trans. Software Eng., vol. 31, pp. 1-21, 2005.
[41] H.M. Collins, Changing Order Replication and Induction in Scientific Practice. Sage Publications, 1985.
[42] W. Broad and N. Wade, Betrayers of the Truth. Oxford Univ. Press, 1986.
[43] J. Brewer and A. Hunter, Multimethod Research: A Synthesis of Styles. Sage Publications, 1989.
[44] E. Arisholm and D.I.K. Sjøberg, “Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software,” IEEE Trans. Software Eng., vol. 30, pp. 521-534, 2004.
[45] E. Arisholm, H.E. Gallis, T. Dybå, and D.I.K. Sjøberg, “Evaluating Pair Programming with Respect to System Complexity and Programmer Expertise,” IEEE Trans. Software Eng., vol. 33, pp.65-86, 2007.
[46] “TAU,” Telelogic, 2003.
[47] “Visio,” Microsoft, 2002.
[48] L.C. Briand, Y. Labiche, and J. Leduc, “Toward the Reverse Engineering of UML Sequence Diagrams for Distributed Java Software,” IEEE Trans. Software Eng., vol. 32, pp. 642-663, 2006.
[49] “Rational Software Architect,” IBM, 2007.
31 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool