This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Mutation Testing Cost Reduction Techniques: A Survey
May/June 2010 (vol. 27 no. 3)
pp. 80-86
ASCII Text
x 
 
Macario Polo Usaola, Pedro Reales Mateo, "Mutation Testing Cost Reduction Techniques: A Survey," IEEE Software, vol. 27, no. 3, pp. 80-86, May/June, 2010.
 
 
BibTex
x
 
@article{ 10.1109/MS.2010.79,
author = {Macario Polo Usaola and Pedro Reales Mateo},
title = {Mutation Testing Cost Reduction Techniques: A Survey},
journal ={IEEE Software},
volume = {27},
number = {3},
issn = {0740-7459},
year = {2010},
pages = {80-86},
doi = {http://doi.ieeecomputersociety.org/10.1109/MS.2010.79},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - MGZN
JO - IEEE Software
TI - Mutation Testing Cost Reduction Techniques: A Survey
IS - 3
SN - 0740-7459
SP80
EP86
EPD - 80-86
A1 - Macario Polo Usaola,
A1 - Pedro Reales Mateo,
PY - 2010
KW - testing and debugging
KW - testing strategies
KW - testing tools
KW - software engineering
VL - 27
JA - IEEE Software
ER -
 
Macario Polo Usaola, University of Castilla-La Mancha
Pedro Reales Mateo, University of Castilla-La Mancha
Since the 1970s, researchers have widely used mutation as a testing technique, applying mainly it to validate test suites, as well as to validate test case strategies and test data generation. Mutation today is sufficiently mature for industrial applications. Although mutation's three main steps (mutant generation, test case execution, and result analysis) can be costly, researchers have made it possible to apply all this knowledge in the industry. This article reviews the most significant research in mutation testing cost reduction, which developers must take into account to apply it to software development and, perhaps more important, to develop powerful new mutation-based testing tools.

1. R. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," IEEE Computer, vol. 11, no. 4, 1978, pp. 34–41.
2. A.J. Offutt and J.M. Voas, Subsumption of Condition Coverage Techniques by Mutation Testing, tech. report ISSE-TR-96-01, Dept. of Information and Software Systems Eng., George Mason Univ., 1996.
3. A.J. Offutt et al., "An Experimental Evaluation of Data Flow and Mutation Testing," Software: Practice and Experience, vol. 26, no. 2, 1996, pp. 165–176.
4. M. Polo, M. Piattini, and S. Tendero, "Integrating Techniques and Tools for Testing Automation," Software Testing, Verification and Reliability, vol. 17, no. 1, 2007, pp. 3–39.
5. A.J. Offutt, "A Practical System for Mutation Testing: Help for the Common Programmer," Proc. 12th Int'l Conf. Testing Computer Software (ICST 95), IEEE CS Press, 1995, pp. 99–109.
6. E.S. Mresa and L. Bottaci, "Efficiency of Mutation Operators and Selective Mutation Strategies: An Empirical Study," Software Testing, Verification and Reliability, vol. 9, no. 4, 1999, pp. 205–232.
7. M. Grindal, A.J. Offutt, and S.F. Andler, "Combination Testing Strategies: A Survey," Software Testing, Verification and Reliability, vol. 15, no. 3, 2005, pp. 167–199.
8. R.A. DeMillo and E.H. Spafford, "The Mothra Software Testing Environment," Proc. 11th NASA Software Eng. Laboratory Workshop, Goddard Space Center, 1986.
9. A.T. Acree, "On Mutation," doctoral dissertation, School of Information and Computer Science, Georgia Inst. of Technology, 1980.
10. R.A. DeMillo et al., "An Extended Overview of the Mothra Software Testing Environment," Proc. 2nd Workshop Software Testing, Verification, and Analysis, IEEE CS Press, 1988, pp. 142–151.
11. K.N. King and A.J. Offutt, "A Fortran Language System for Mutation-Based Software Testing," Software: Practice and Experience, vol. 21, no. 7, 1991, pp. 685–718.
12. A.P. Mathur, "Performance, Effectiveness, and Reliability Issues in Software Testing," Proc. 15th Ann. Int'l Computer Software and Applications Conf., IEEE CS Press, 1991, pp. 604–605.
13. A.J. Offutt et al., "An Experimental Determination of Sufficient Mutant Operators," ACM Trans. Software Eng. and Methodology, vol. 5, no. 2, 1996, pp. 99–118.
14. J. Andrews, L. Briand, and Y. Labiche, "Is Mutation an Appropriate Tool for Testing Experiments?" Proc. 2005 Int'l Conf. Software Eng. (ICSE 05), ACM Press, 2005, pp. 402–411.
15. R. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," IEEE Computer, vol. 11, no. 4, 1978, pp. 34–41.
16. A.J. Offut, "Investigations of the Software Testing Coupling Effect," ACM Trans. Software Eng. and Methodology, vol. 1, no. 1, 1992, pp. 15–20.
17. E.F. Barbosa et al., "Toward the Determination of Sufficient Mutant Operators for C," Software Testing, Verification and Reliability, vol. 11, no. 2, 2001, pp. 113–136.
18. Y.-S. Ma, "MuJava: An Automated Class Mutation System," Software Testing, Verification and Reliability, vol. 15, no. 2, 2005, pp. 97–133.
19. M.R. Garey and D.S. Johnson, Computers and Intractability, W.H. Freeman, 1979.
20. D. Jeffrey and N. Gupta, "Test Suite Reduction with Selective Redundancy," Proc. 21st Int'l Conf. Software Maintenance (ICSM 05), IEEE CS Press, 2005, pp. 549–558.
21. R. DeMillo, E. Krauser, and A. Mathur, "Compiler-Integrated Program Mutation," Proc. 15th Ann. Computer Software and Applications Conf. (Compsac 91), pp. 351–356. 1991.
22. W.E. Howden, "Weak Mutation Testing and Completeness of Test Sets," IEEE Trans. Software Eng., vol. 8, no. 4, 1982, pp. 371–379.
23. A.J. Offutt and S.D. Lee, "An Empirical Evaluation of Weak Mutation," IEEE Trans. Software Eng., vol. 20, no. 5, 1994, pp. 337–344.
24. P.G. Frankl, S.N. Weiss, and C. Hu, "All-Uses versus Mutation Testing: An Experimental Comparison of Effectiveness," J. Systems and Software, vol. 38, no. 3, 2007, pp. 235–253.
25. B.J.M. Grün, D. Schuler, and A. Zeller, "The Impact of Equivalent Mutants," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation Workshops (ICST 09), IEEE CS Press, 2009, pp. 192–199.
26. A.J. Offutt and J. Pan, "Automatically Detecting Equivalent Mutants and Infeasible Paths," Software Testing, Verification and Reliability, vol. 7, no. 3, 1997, pp. 165–192.
27. R. Hierons and M. Harman, "Using Program Slicing to Assist in the Detection of Equivalent Mutants," Software Testing, Verification and Reliability, vol. 9, no. 4, 1999, pp. 233–262.
28. M. Polo, M. Piattini, and I. García-Rodríguez, "Decreasing the Cost of Mutation Testing with Second-Order Mutants," Software Testing, Verification and Reliability, vol. 19, no. 2, 2008, pp. 111–131.
1. P.G. Frankl and E.J. Weyuker, "An Applicable Family of Data Flow Testing Criteria," IEEE Trans. Software Eng., vol. 14, no. 10, 1998, pp. 1483–1498.
2. A.J. Offutt and J.M. Voas, Subsumption of Condition Coverage Techniques by Mutation Testing, tech. report ISSE-TR-96-01, Dept. of Information and Software Systems Eng., George Mason Univ., 1996.

Index Terms:
testing and debugging, testing strategies, testing tools, software engineering
Citation:
Macario Polo Usaola, Pedro Reales Mateo, "Mutation Testing Cost Reduction Techniques: A Survey," IEEE Software, vol. 27, no. 3, pp. 80-86, May-June 2010, doi:10.1109/MS.2010.79
Usage of this product signifies your acceptance of the Terms of Use.