This Article 
 Bibliographic References 
 Add to: 
The Impact of UML Documentation on Software Maintenance: An Experimental Evaluation
June 2006 (vol. 32 no. 6)
pp. 365-381
The Unified Modeling Language (UML) is becoming the de facto standard for software analysis and design modeling. However, there is still significant resistance to model-driven development in many software organizations because it is perceived to be expensive and not necessarily cost-effective. Hence, it is important to investigate the benefits obtained from modeling. As a first step in this direction, this paper reports on controlled experiments, spanning two locations, that investigate the impact of UML documentation on software maintenance. Results show that, for complex tasks and past a certain learning curve, the availability of UML documentation may result in significant improvements in the functional correctness of changes as well as the quality of their design. However, there does not seem to be any saving of time. For simpler tasks, the time needed to update the UML documentation may be substantial compared with the potential benefits, thus motivating the need for UML tools with better support for software maintenance.

[1] B. Anda, K. Hansen, I. Gullesen, and H.K. Thorsen, “Experiences from Using a UML-Based Development Method in a Large Organization,” Empirical Software Eng. J., to appear.
[2] E. Arisholm, L.C. Briand, S.E. Hove, and Y. Labiche, “The Impact of UML Documentation on Software Maintenance: An Experimental Evaluation,” Technical Report 2005-14, Simula Research Laboratory, 2005.
[3] E. Arisholm and D. Sjøberg, “Evaluating the Effect of a Delegated versus Centralized Control Style on the Maintainability of Object-Oriented Software,” IEEE Trans. Software Eng., vol. 30, no. 8, pp. 521-534, Aug. 2004.
[4] E. Arisholm, D. Sjøberg, G.J. Carelius, and Y. Lindsjørn, “A Web-Based Support Environment for Software Engineering Experiments,” Nordic J. Computing, vol. 9, no. 4, pp. 231-247, 2002.
[5] V. Basili, “The Role of Experimentation in Software Engineering: Past, Current, and Future,” Proc. IEEE Int'l Conf. Software Eng., pp. 442-449, 1996.
[6] V. Basili, F. Shull, and F. Lanubile, “Building Knowledge through Families of Experiments,” IEEE Trans. Software Eng., vol. 25, no. 4, pp. 456-473, July/Aug. 1999.
[7] K. Beck, Extreme Programming Explained. Addison Wesley, 2001.
[8] G. Booch, J. Rumbaugh, and I. Jacobson, The Unified Modeling Language User Guide. Addison Wesley, 1999.
[9] Borland: Together, 2003, www.borland.comtogether.
[10] L.C. Briand, “Software Documentation: How Much Is Enough?” Proc. IEEE European Conf. Software Maintenance and Reng., Invited Keynote Address, pp. 13-15, 2003.
[11] L.C. Briand, Y. Labiche, M. Di Penta, and H.-D. Yan-Bondoc, “An Experimental Investigation of Formality in UML-Based Development,” IEEE Trans. Software Eng., vol. 31, no. 10, pp. 833-849, Oct. 2005.
[12] B. Bruegge and A.H. Dutoit, Object-Oriented Software Engineering Using UML, Patterns, and Java, second ed. Prentice Hall, 2004.
[13] J.L. Devore and N. Farnum, Applied Statistics for Engineers and Scientists. Duxbury, 1999.
[14] S.E. Hove and B. Anda, “Experiences from Conducting Semi-Structured Interviews in Empirical Software Engineering Research,” Proc. IEEE Int'l Symp. Software Metrics, pp. 23-32, 2005.
[15] C.M. Judd, E.R. Smith, and L.H. Kidder, Research Methods in Social Relations, sixth ed. Holt, Rinehart, and Winston, 1991.
[16] A. Karahasanovic, B. Anda, E. Arisholm, S.E. Hove, M. Jørgensen, D. Sjøberg, and R. Welland, “Collecting Feedback during Software Engineering Experiments,” Empirical Software Eng. — An Int'l J., vol. 10, no. 2, pp. 113-147, 2005.
[17] A. Kleppe, J. Warmer, and W. Bast, MDA Explained— The Model Driven Architecture: Practice and Promise. Addison-Wesley, 2003.
[18] M. Kutar, C. Britton, and T. Barker, “A Comparison of Empirical Study and Cognitive Dimensions Analysis in the Evaluation of UML Diagrams,” Proc. 14th Psychology of Programming Interest Group, 2002.
[19] L. Kuzniarz, M. Staron, and C. Wohlin, “An Empirical Study on Using Stereotypes to Improve Understanding of UML Models,” Proc. 12th IEEE Intl Workshop Program Comprehension, 2004.
[20] Microsoft: Visio, Version 2002, 2003,
[21] OMG, “UML 1.5 Specification,” Object Management Group, Complete Specification formal/03-03-01, 2003.
[22] A.N. Oppenheim, Questionnaire Design, Interviewing and Attitude Measurement. Pinter Publishers, 1992.
[23] T.J. Ostrand and M.J. Balcer, “The Category-Partition Method for Specifying and Generating Functional Test,” Comm. ACM, vol. 31, no. 6, pp. 676-686, 1988.
[24] M.C. Otero and J.J. Dolado, “An Empirical Comparison of the Dynamic Modeling in OML and UML,” J. Systems and Software, vol. 77, no. 2, pp. 91-102, 2005.
[25] T. Pender, UML Bible. Wiley, 2003.
[26] R.S. Pressman, Software Engineering— A Practitioner's Approach, seventh ed. McGraw Hill, 2005.
[27] H.C. Purchase, L. Colpoys, M. McGill, and D. Carrington, “UML Collaboration Diagram Syntax: An Empirical Study of Comprehension,” Proc. First Int'l Workshop Visualizing Software for Understanding and Analysis, 2002.
[28] H.C. Purchase, L. Colpoys, M. McGill, D. Carrington, and C. Britton, “UML Class Diagram Syntax: An Empirical Study of Comprehension,” Proc. Australian Symp. Information Visualisation, 2001.
[29] QSR: N6, 2004, http:/
[30] I. Reinhartz-Berger and D. Dori, “OPM vs. UML-Experimenting with Comprehension and Construction of Web Application Models,” Empirical Software Engineering— An Int'l J., vol. 10, no. 1, pp. 57-79, 2005.
[31] C.B. Seaman, “Qualitative Methods in Empirical Studies of Software Engineering,” IEEE Trans. Software Eng., vol. 25, no. 4, pp. 557-572, July/Aug. 1999.
[32] E. Soloway, R. Lampert, S. Letowski, D. Littman, and J. Pinto, “Designing Documentation to Compensate for Delocalized Plans,” Comm. ACM, vol. 31, no. 11, pp. 1259-1267, 1988.
[33] Telelogic: TAU, 2003,
[34] S. Tilley and S. Huang, “A Qualitative Assessment of the Efficacy of UML Diagrams as a Form of Graphical Documentation in Aiding Program Understanding,” Proc. 21st Ann. Int'l Conf. Systems Documentation, pp. 184-191, 2003.
[35] E. Tryggeseth, “Report from an Experiment: Impact of Documentation on Maintenance,” Empirical Software Eng.: An Int'l J., vol. 2, no. 2, pp. 201-207, 1997.
[36] J. Warmer and A. Kleppe, The Object Constraint Language, second ed. Addison Wesley, 2003.

Index Terms:
Maintenance, UML, experiment.
Erik Arisholm, Lionel C. Briand, Siw Elisabeth Hove, Yvan Labiche, "The Impact of UML Documentation on Software Maintenance: An Experimental Evaluation," IEEE Transactions on Software Engineering, vol. 32, no. 6, pp. 365-381, June 2006, doi:10.1109/TSE.2006.59
Usage of this product signifies your acceptance of the Terms of Use.