The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - May/June (2010 vol.27)
pp: 80-86
Macario Polo Usaola , University of Castilla-La Mancha
Pedro Reales Mateo , University of Castilla-La Mancha
ABSTRACT
Since the 1970s, researchers have widely used mutation as a testing technique, applying mainly it to validate test suites, as well as to validate test case strategies and test data generation. Mutation today is sufficiently mature for industrial applications. Although mutation's three main steps (mutant generation, test case execution, and result analysis) can be costly, researchers have made it possible to apply all this knowledge in the industry. This article reviews the most significant research in mutation testing cost reduction, which developers must take into account to apply it to software development and, perhaps more important, to develop powerful new mutation-based testing tools.
INDEX TERMS
testing and debugging, testing strategies, testing tools, software engineering
CITATION
Macario Polo Usaola, Pedro Reales Mateo, "Mutation Testing Cost Reduction Techniques: A Survey", IEEE Software, vol.27, no. 3, pp. 80-86, May/June 2010, doi:10.1109/MS.2010.79
REFERENCES
1. R. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," IEEE Computer, vol. 11, no. 4, 1978, pp. 34–41.
2. A.J. Offutt and J.M. Voas, Subsumption of Condition Coverage Techniques by Mutation Testing, tech. report ISSE-TR-96-01, Dept. of Information and Software Systems Eng., George Mason Univ., 1996.
3. A.J. Offutt et al., "An Experimental Evaluation of Data Flow and Mutation Testing," Software: Practice and Experience, vol. 26, no. 2, 1996, pp. 165–176.
4. M. Polo, M. Piattini, and S. Tendero, "Integrating Techniques and Tools for Testing Automation," Software Testing, Verification and Reliability, vol. 17, no. 1, 2007, pp. 3–39.
5. A.J. Offutt, "A Practical System for Mutation Testing: Help for the Common Programmer," Proc. 12th Int'l Conf. Testing Computer Software (ICST 95), IEEE CS Press, 1995, pp. 99–109.
6. E.S. Mresa and L. Bottaci, "Efficiency of Mutation Operators and Selective Mutation Strategies: An Empirical Study," Software Testing, Verification and Reliability, vol. 9, no. 4, 1999, pp. 205–232.
7. M. Grindal, A.J. Offutt, and S.F. Andler, "Combination Testing Strategies: A Survey," Software Testing, Verification and Reliability, vol. 15, no. 3, 2005, pp. 167–199.
8. R.A. DeMillo and E.H. Spafford, "The Mothra Software Testing Environment," Proc. 11th NASA Software Eng. Laboratory Workshop, Goddard Space Center, 1986.
9. A.T. Acree, "On Mutation," doctoral dissertation, School of Information and Computer Science, Georgia Inst. of Technology, 1980.
10. R.A. DeMillo et al., "An Extended Overview of the Mothra Software Testing Environment," Proc. 2nd Workshop Software Testing, Verification, and Analysis, IEEE CS Press, 1988, pp. 142–151.
11. K.N. King and A.J. Offutt, "A Fortran Language System for Mutation-Based Software Testing," Software: Practice and Experience, vol. 21, no. 7, 1991, pp. 685–718.
12. A.P. Mathur, "Performance, Effectiveness, and Reliability Issues in Software Testing," Proc. 15th Ann. Int'l Computer Software and Applications Conf., IEEE CS Press, 1991, pp. 604–605.
13. A.J. Offutt et al., "An Experimental Determination of Sufficient Mutant Operators," ACM Trans. Software Eng. and Methodology, vol. 5, no. 2, 1996, pp. 99–118.
14. J. Andrews, L. Briand, and Y. Labiche, "Is Mutation an Appropriate Tool for Testing Experiments?" Proc. 2005 Int'l Conf. Software Eng. (ICSE 05), ACM Press, 2005, pp. 402–411.
15. R. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practicing Programmer," IEEE Computer, vol. 11, no. 4, 1978, pp. 34–41.
16. A.J. Offut, "Investigations of the Software Testing Coupling Effect," ACM Trans. Software Eng. and Methodology, vol. 1, no. 1, 1992, pp. 15–20.
17. E.F. Barbosa et al., "Toward the Determination of Sufficient Mutant Operators for C," Software Testing, Verification and Reliability, vol. 11, no. 2, 2001, pp. 113–136.
18. Y.-S. Ma, "MuJava: An Automated Class Mutation System," Software Testing, Verification and Reliability, vol. 15, no. 2, 2005, pp. 97–133.
19. M.R. Garey and D.S. Johnson, Computers and Intractability, W.H. Freeman, 1979.
20. D. Jeffrey and N. Gupta, "Test Suite Reduction with Selective Redundancy," Proc. 21st Int'l Conf. Software Maintenance (ICSM 05), IEEE CS Press, 2005, pp. 549–558.
21. R. DeMillo, E. Krauser, and A. Mathur, "Compiler-Integrated Program Mutation," Proc. 15th Ann. Computer Software and Applications Conf. (Compsac 91), pp. 351–356. 1991.
22. W.E. Howden, "Weak Mutation Testing and Completeness of Test Sets," IEEE Trans. Software Eng., vol. 8, no. 4, 1982, pp. 371–379.
23. A.J. Offutt and S.D. Lee, "An Empirical Evaluation of Weak Mutation," IEEE Trans. Software Eng., vol. 20, no. 5, 1994, pp. 337–344.
24. P.G. Frankl, S.N. Weiss, and C. Hu, "All-Uses versus Mutation Testing: An Experimental Comparison of Effectiveness," J. Systems and Software, vol. 38, no. 3, 2007, pp. 235–253.
25. B.J.M. Grün, D. Schuler, and A. Zeller, "The Impact of Equivalent Mutants," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation Workshops (ICST 09), IEEE CS Press, 2009, pp. 192–199.
26. A.J. Offutt and J. Pan, "Automatically Detecting Equivalent Mutants and Infeasible Paths," Software Testing, Verification and Reliability, vol. 7, no. 3, 1997, pp. 165–192.
27. R. Hierons and M. Harman, "Using Program Slicing to Assist in the Detection of Equivalent Mutants," Software Testing, Verification and Reliability, vol. 9, no. 4, 1999, pp. 233–262.
28. M. Polo, M. Piattini, and I. García-Rodríguez, "Decreasing the Cost of Mutation Testing with Second-Order Mutants," Software Testing, Verification and Reliability, vol. 19, no. 2, 2008, pp. 111–131.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool