The Community for Technology Leaders
RSS Icon
Issue No.05 - Sept.-Oct. (2012 vol.38)
pp: 1213-1231
Cheng Zhang , Durham University, Durham
David Budgen , Durham University, Durham
Context. Although research in software engineering largely seeks to improve the practices and products of software development, many practices are based upon codification of expert knowledge, often with little or no underpinning from objective empirical evidence. Software design patterns seek to codify expert knowledge to share experience about successful design structures. Objectives. To investigate how extensively the use of software design patterns has been subjected to empirical study and what evidence is available about how and when their use can provide an effective mechanism for knowledge transfer about design. Method. We conducted a systematic literature review in the form of a mapping study, searching the literature up to the end of 2009 to identify relevant primary studies about the use of the 23 patterns catalogued in the widely referenced book by the “Gang of Four.” These studies were then categorized according to the forms of study employed, the patterns that were studied, as well as the context within which the study took place. Results. Our searches identified 611 candidate papers. Applying our inclusion/exclusion criteria resulted in a final set of 10 papers that described 11 instances of “formal” experimental studies of object-oriented design patterns. We augmented our analysis by including seven “experience” reports that described application of patterns using less rigorous observational forms. We report and review the profiles of the empirical evidence for those patterns for which multiple studies exist. Conclusions. We could not identify firm support for any of the claims made for patterns in general, although there was some support for the usefulness of patterns in providing a framework for maintenance, and some qualitative indication that they do not help novices learn about design. For future studies we recommend that researchers use case studies that focus upon some key patterns, and seek to identify the impact that their use can have upon maintenance.
Software engineering, Software design, Systematics, Search engines, Terminology, Maintenance engineering, empirical software engineering, Design patterns, systematic literature review
Cheng Zhang, David Budgen, "What Do We Know about the Effectiveness of Software Design Patterns?", IEEE Transactions on Software Engineering, vol.38, no. 5, pp. 1213-1231, Sept.-Oct. 2012, doi:10.1109/TSE.2011.79
[1] Guide to the Software Engineering Body of Knowledge, A. Abran, J.W. Moore, P. Bourque, and R. Dupuis, eds. IEEE Computer Society, 2004.
[2] B. Adelson and E. Soloway, "The Role of Domain Experience in Software Design," IEEE Trans. Software Eng., vol. 11, no. 11, pp. 1351-1360, Nov. 1985.
[3] C. Alexander, S. Ishikawa, M. Silverstein, M. Jacobson, I. Fiksdahl-King, and S. Angel, A Pattern Language. Oxford Univ. Press, 1977.
[4] L. Aversano, L. Cerulo, and M. Di Penta, "Relationship between Design Patterns Defects and Crosscutting Concern Scattering Degree: An Empirical Study," IET Software, vol. 3, no. 5, pp. 395-409, Oct. 2009.
[5] L. Aversano, G. Canfora, and L. Cerulo, "An Empirical Study on the Evolution of Design Patterns," Proc. Sixth Joint Meeting of the European Software Eng. Conf. and the ACM SIGSOFT Symp. The Foundations of Software Eng., pp. 385-394, 2007.
[6] L. Aversano, L. Cerulo, and M. Di Penta, "Relating the Evolution of Design Patterns and Crosscutting Concerns," Proc. Seventh IEEE Int'l Working Conf. Source Code Analysis and Manipulation, pp. 180-192, 2007.
[7] J.M. Bieman, G. Straw, H. Wang, P.W. Munger, and R.T. Alexander, "Design Patterns and Change Proneness: An Examination of Five Evolving Systems," Proc. Ninth Int'l Software Metrics Symp., pp. 40-49, 2003.
[8] J.M. Bieman, D. Jain, and H.J. Yang, "OO Design Patterns, Design Structure, and Program Changes: An Industrial Case Study," Proc. IEEE Int'l Conf. Software Maintenance, pp. 580-589, 2001.
[9] O.P. Brereton, B.A. Kitchenham, D. Budgen, and Z. Li, "Using a Protocol Template for Case Study Planning," Proc. 12th Int'l Conf. Evaluation and Assessment in Software Eng., 2008.
[10] O.P. Brereton, B.A. Kitchenham, D. Budgen, M. Turner, and M.A. Khalil, "Lessons from Applying the Systematic Literature Review Process within the Software Engineering Domain," J. Systems & Software, vol. 80, no. 4, pp. 571-583, 2007.
[11] D. Budgen, J. Bailey, M. Turner, B. Kitchenham, P. Brereton, and S. Charters, "Cross-Domain Investigation of Empirical Practices," IET Software, EASE special section, vol. 3, no. 5, pp. 410-421, Oct. 2009.
[12] D. Budgen and C. Zhang, "Preliminary Reporting Guidelines for Experience Papers," Proc. 13th Int'l Conf. Evaluation and Assessment in Software Eng., pp. 1-10, 2009.
[13] F. Buschmann, R. Meunier, H. Rohnert, P. Sommerlad, and M. Stal, Pattern-Oriented Software Architecture. Wiley, 1996.
[14] A. Chatzigeorgiou, N. Tsantalis, and I. Deligiannis, "An Empirical Study on Students' Ability to Comprehend Design Patterns," Computers & Education, vol. 51, pp. 1007-1016, 2008.
[15] E. Chung, J. Hong, M. Prabaker, J. Landay, and A. Liu, "Development and Evaluation of Emerging Design Patterns for Ubiquitous Computing," Proc. Conf. Designing Interactive Systems, pp. 233-242, 2004.
[16] M.Ó. Cinnéide and P. Fagan, "Design Patterns: The Devils in the Detail," Proc. Conf. Pattern Languages of Programs, 2006.
[17] F. Détienne, Software Design—Cognitive Aspects, practitioner series. Springer, 2002.
[18] T. Dybå, B.A. Kitchenham, and M. Jørgensen, "Evidence-Based Software Engineering for Practitioners," IEEE Software, vol. 22, no. 1, pp. 58-65, Jan./Feb. 2005.
[19] M. Jørgensen, T. Dybå, and T. Dingsøyr, "Empirical Studies of Agile Software Development: A Systematic Review," Information & Software Technology, vol. 50, pp. 833-859, 2008.
[20] B. Ellis, J. Stylos, and B. Myers, "The Factory Pattern in API Design: A Usability Evaluation," Proc. 29th Int'l Conf. Software Eng., pp. 302-311, 2007.
[21] E. Gamma, R. Helm, R. Johnson, and J. Vlissides, Design Patterns: Elements of Reusable Object-Oriented Software. Addison-Wesley, 1995.
[22] M. Gattrell, S. Counsell, and T. Hall, "Design Patterns and Change Proneness: A Replication Using Proprietary Csharp Software," Proc. 16th Working Conf. Reverse Eng., pp. 160-164, 2009.
[23] R.L. Glass, V. Ramesh, and I. Vessey, "An Analysis of Research in Computing Disciplines," Comm. ACM, vol. 47, pp. 89-94, June 2004.
[24] R.L. Glass, I. Vessey, and V. Ramesh, "Research in Software Engineering: An Analysis of the Literature," Information & Software Technology, vol. 44, pp. 491-506, 2002.
[25] R. Guindon, "Knowledge Exploited by Experts during Software System Design," Int'l J. Man-Machine Studies, vol. 33, pp. 279-304, 1990.
[26] M. Hahsler, "A Quantitative Study of the Adoption of Design Patterns by Open Source Software Developers," Free/Open Source Software Development, S. Koch, ed., Idea Group, Inc., 2004.
[27] G.K. Hanssen, F.O. Bjørnson, and H. Westerheim, "Tailoring and Introduction of the Rational Unified Process," Proc. 14th European Conf. European Systems and Software Process Improvement, pp. 7-18, 2007.
[28] B. Hayes-Roth and F. Hayes-Roth, "A Cognitive Model of Planning," Cognitive Science, vol. 3, no. 4, pp. 275-310, 1979.
[29] M. Höst and P. Runeson, "Checklists for Software Engineering Case Study Research," Proc. First Int'l Symp. Empirical Software Eng. and Measurement, pp. 479-481, 2007.
[30] C. Izurieta and J.M. Bieman, "How Software Designs Decay: A Pilot Study of Pattern Evolution," Proc. First Int'l Symp. Empirical Software Eng. and Measurement, pp. 459-461, 2007.
[31] M. Abdul Jalil and S.A. Mohd Noah, "The Difficulties of Using Design Patterns among Novices: An Exploratory Study," Proc. Fifth Int'l Conf. Computational Science & Applications, pp. 97-103, 2007.
[32] S. Jeanmart, Y.-G. Guéhéneuc, H. Sahraoui, and N. Habra, "Impact of the Visitor Pattern on Program Comprehension and Maintenance," Proc. Third Int'l Symp. Empirical Software Eng. & Measurement, pp. 69-78, 2009.
[33] F. Khomh and Y.-G. Guéhéneuc, "Perception and Reality: What Are Design Patterns Good For?" Proc. 11th ECOOP Workshop Quantitative Approaches in Object Oriented Software Eng., p. 7, 2007.
[34] F. Khomh and Y. Guéhéneuc, "Do Design Patterns Impact Software Quality Positively?" Proc. 12th European Conf. Software Maintenance and Reeng., pp. 274-278, 2008.
[35] B.A. Kitchenham, T. Dybå, and M. Jørgensen, "Evidence-Based Software Engineering," Proc. 26th Int'l Conf. Software Eng., pp. 273-281, 2004.
[36] B. Kitchenham, P. Brereton, D. Budgen, M. Turner, J. Bailey, and S. Linkman, "Systematic Literature Reviews in Software Engineering—A Systematic Literature Review," Information & Software Technology, vol. 51, no. 1, pp. 7-15, 2009.
[37] B. Kitchenham, D. Budgen, P. Brereton, M. Turner, S. Charters, and S. Linkman, "Large-Scale Software Engineering Questions-Expert Opinion or Empirical Evidence?" IET Software, vol. 1, no. 5, pp. 161-171, Oct. 2007.
[38] B. Kitchenham and S. Charters, "Guidelines for Performing Systematic Literature Review in Software Engineering," Technical Report EBSE 2007-001, Keele Univ. and Durham Univ. Joint Report, 2007.
[39] B. Kitchenham, R. Pretorius, D. Budgen, P. Brereton, M. Turner, M. Niazi, and S. Linkman, "Systematic Literature Reviews in Software Engineering—A Tertiary Study," Information & Software Technology, vol. 52, pp. 792-805, 2010.
[40] B.A. Kitchenham, D. Budgen, and O. Pearl Brereton, "Using Mapping Studies as the Basis for Further Research—A Participant-Observer Case Study," Information & Software Technology, special section from EASE, vol. 53, no. 4, pp. 638-651, 2011.
[41] C. Kohls and K. Scheiter, "The Relation between Design Patterns and Schema Theory," Proc. 15th Conf. Pattern Languages of Programs, pp. 1-14, 2008.
[42] R. Murray Lindsay and A.S.C. Ehrenberg, "The Design of Replicated Studies," The Am. Statistician, vol. 47, no. 3, pp. 217-228, 1993.
[43] G. Masuda, N. Sakamoto, and K. Ushijima, "Applying Design Patterns to Decision Tree Learning System," Proc. Sixth ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 111-120, 1998.
[44] P. Mohagheghi and V. Dehlen, "Where Is the Proof?—A Review of Experiences from Applying MDE in Industry," Proc. Fourth European Conf. Model Driven Architecture: Foundations and Applications, pp. 432-443, 2008.
[45] T.H. Ng, S.C Cheung, W.K. Chan, and Y.T. Yu, "Toward Effective Deployment of Design Patterns for Software Extension: A Case Study," Proc. Int'l Workshop Software Quality, pp. 51-56, 2006.
[46] T.H. Ng, S.C. Cheung, W.K. Chan, and Y.T. Yu, "Do Maintainers Utilize Deployed Design Patterns Effectively?" Proc. 29th Int'l Conf. Software Eng., pp. 168-177, 2007.
[47] M. Di Penta, L. Cerulo, Y.-G. Guéhéneuc, and G. Antoniol, "An Empirical Study of the Relationships between Design Patterns Roles and Class Change Proneness," Proc. IEEE Int'l Conf. Software Maintenance, pp. 217-226, 2008.
[48] M. Petticrew and H. Roberts, Systematic Review in the Social Sciences: A Practical Guide. Blackwell Publishing, 2006.
[49] L. Prechelt and B. Unger, "A Series of Controlled Experiments on Design Patterns: Methodology and Results," Proc. Softwaretechnik '98, 1998.
[50] L. Prechelt, B. Unger, and M. Philippsen, "Documenting Design Patterns in Code Eases Program Maintenance," Proc. ICSE Workshop Process Modelling and Empirical Studies of Software Evolution, pp. 72-76, 1997.
[51] L. Prechelt, B. Unger, W.F. Tichy, P. Brossler, and L.G. Votta, "A Controlled Experiment in Maintenance Comparing Design Patterns to Simpler Solutions," IEEE Trans. Software Eng., vol. 27, no. 12, pp. 1133-1144, Dec. 2001.
[52] L. Prechelt, B. Unger-Lamprecht, M. Philippsen, and W.F. Tichy, "Two Controlled Experiments Assessing the Usefulness of Design Pattern Documentation in Program Maintenance," IEEE Trans. Software Eng., vol. 28, no. 6, pp. 595-606, June 2002.
[53] R. Pretorius and D. Budgen, "A Mapping Study on Empirical Evidence Related to the Models and Forms Used in the UML," Proc. ACM/IEEE Second Int'l Symp. Empirical Software Eng. and Measurement, pp. 342-344, 2008.
[54] H.J. Rittel and M.M. Webber, "Planning Problems Are Wicked Problems," Developments in Design Methodology, N. Cross, ed., pp. 135-144, Wiley, 1984.
[55] D. Schmidt, "Using Design Patterns to Develop Reusable Object-Oriented Communication Software," Comm. ACM, vol. 38, no. 10, pp. 65-74, 1995.
[56] I. Sommerville, Software Engineering, eighth ed. Addison-Wesley, 2007.
[57] M. Torchiano, "Documenting Pattern Use in Java Programs," Proc. Int'l Conf. Software Maintenance, pp. 230-233, 2002.
[58] B. Unger and W. Tichy, "Do Design Patterns Improve Communication? An Experiment with Pair Design," Proc. Int'l Workshop Empirical Studies of Software Maintenance, pp. 1-5, 2000.
[59] M. Vocak, W.F Tichy, D.I.K. Sjøberg, E. Arisolm, and M. Aldrin, "A Controlled Experiment Comparing the Maintainability of Programs Designed with and without Design Patterns—A Replication in a Real Programming Environment," Empirical Software Eng., vol. 9, pp. 149-195, 2004.
[60] P. Wendorff, "Assessment of Design Patterns during Software Reengineering: Lessons Learned from a Large Commercial Project," Proc. Fifth European Conf. Software Maintenance and Reeng., pp. 77-84, 2001.
[61] K.N. Whitley, "Visual Programming Languages and the Empirical Evidence For and Against," J. Visual Languages and Computing, vol. 8, pp. 109-142, 1997.
[62] L. Williams and A. Cockburn, "Agile Software Development: It's about Feedback and Change," Computer, vol. 36, no. 4, pp. 39-43, Apr. 2003.
[63] B. Wydaeghe, K. Verschaeve, B. Michiels, B. Van Damme, E. Archens, and V. Jonckers, "Building an OMT-Editor Using Design Patterns: An Experience Report," Proc. Conf. Technology of Object-Oriented Languages, 1998.
[64] R. Yin, Case Study Research: Design & Methods, third ed. Sage Books, 2003.
[65] W. Yuanhong, M. Hong, and S. Weizhong, "Experience Report: Using Design Patterns in the Development of JB System," Proc. Technology of Object Oriented Languages and Systems, pp. 159-165, 1997.
[66] C. Zhang and D. Budgen, "A Survey of Experienced User Perceptions about Design Patterns," submitted for publication, 2011.
22 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool