The Community for Technology Leaders
Green Image
Issue No. 04 - April (2016 vol. 42)
ISSN: 0098-5589
pp: 302-321
Jonathan L. Krein , , Department of Computer Science, Provo, UT
Lutz Prechelt , , Institut für Informatik, Germany
Natalia Juristo , , Computing School, Spain
Aziz Nanthaamornphong , , Department of Information and Communication Technology, Thailand
Jeffrey C. Carver , , Department of Computer Science, Tuscaloosa, AL
Sira Vegas , , Computing School, Spain
Charles D. Knutson , , Department of Computer Science, Provo, UT
Kevin D. Seppi , , Department of Computer Science, Provo, UT
Dennis L. Eggett , , Department of Statistics, Provo, UT
ABSTRACT
Context. Several empirical studies have explored the benefits of software design patterns, but their collective results are highly inconsistent. Resolving the inconsistencies requires investigating moderators—i.e., variables that cause an effect to differ across contexts. Objectives. Replicate a design patterns experiment at multiple sites and identify sufficient moderators to generalize the results across prior studies. Methods. We perform a close replication of an experiment investigating the impact (in terms of time and quality) of design patterns (Decorator and Abstract Factory) on software maintenance. The experiment was replicated once previously, with divergent results. We execute our replication at four universities—spanning two continents and three countries—using a new method for performing distributed replications based on closely coordinated, small-scale instances (“joint replication”). We perform two analyses: 1) a post-hoc analysis of moderators, based on frequentist and Bayesian statistics; 2) an a priori analysis of the original hypotheses, based on frequentist statistics. Results. The main effect differs across the previous instances of the experiment and across the sites in our distributed replication. Our analysis of moderators (including developer experience and pattern knowledge) resolves the differences sufficiently to allow for cross-context (and cross-study) conclusions. The final conclusions represent 126 participants from five universities and 12 software companies, spanning two continents and at least four countries. Conclusions. The Decorator pattern is found to be preferable to a simpler solution during maintenance, as long as the developer has at least some prior knowledge of the pattern. For Abstract Factory, the simpler solution is found to be mostly equivalent to the pattern solution. Abstract Factory is shown to require a higher level of knowledge and/or experience than Decorator for the pattern to be beneficial.
INDEX TERMS
Joints, Production facilities, Portals, Training, Libraries, Industries, Context,controlled experiment, Design patterns, software maintenance, moderator variables, multi-site, joint replication,controlled experiment, Design patterns, software maintenance, moderator variables, multi-site, joint replication
CITATION
Jonathan L. Krein, Lutz Prechelt, Natalia Juristo, Aziz Nanthaamornphong, Jeffrey C. Carver, Sira Vegas, Charles D. Knutson, Kevin D. Seppi, Dennis L. Eggett, "A Multi-Site Joint Replication of a Design Patterns Experiment Using Moderator Variables to Generalize across Contexts", IEEE Transactions on Software Engineering, vol. 42, no. , pp. 302-321, April 2016, doi:10.1109/TSE.2015.2488625
98 ms
(Ver 3.3 (11022016))