Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TSE.2013.44
Lech Madeyski , Wroclaw University of Technology, Wroclaw
Wojciech Orzeszyna , Wroclaw University of Technology, Wroclaw and Blekinge Institute of Technology, Karlskrona
Richard Torkar , Blekinge Institute of Technology, Karlskrona and Chalmers University of Technology | University of Gothenburg, Göteborg
Mariusz Józala , Wroclaw University of Technology, Wroclaw
Context. The equivalent mutant problem (EMP) is one of the crucial problems in mutation testing widely studied over decades. Objectives. The objectives are: to present a systematic literature review (SLR) in the field of EMP; to identify, classify and improve the existing, or implement new, methods which try to overcome EMP and evaluate them. Method. We performed SLR based on the search of digital libraries. We implemented four second order mutation (SOM) strategies, in addition to first order mutation (FOM), and compared them from different perspectives. Results. Our SLR identified 17 relevant techniques (in 22 articles) and three categories of techniques: detecting (DEM); suggesting (SEM); and avoiding equivalent mutant generation (AEMG). The experiment indicated that SOM in general and JudyDiffOp strategy in particular provide the best results in the following areas: total number of mutants generated; the association between the type of mutation strategy and whether the generated mutants were equivalent or not; the number of not killed mutants; mutation testing time; time needed for manual classification. Conclusions. The results in the DEM category are still far from perfect. Thus, the SEM and AEMG categories have been developed. The JudyDiffOp algorithm achieved good results in many areas.
Software Quality/SQA, Testing tools, Testing strategies, Test coverage of code
L. Madeyski, W. Orzeszyna, R. Torkar and M. Józala, "Overcoming the Equivalent Mutant Problem: A Systematic Literature Review and a Comparative Experiment of Second Order Mutation," in IEEE Transactions on Software Engineering.