This Article 
 Bibliographic References 
 Add to: 
An Empirical Evaluation of Weak Mutation
May 1994 (vol. 20 no. 5)
pp. 337-344

Mutation testing is a fault-based technique for unit-level software testing. Weak mutation was proposed as a way to reduce the expense of mutation testing. Unfortunately, weak mutation is also expected to provide a weaker test of the software than mutation testing does. This paper presents results from an implementation of weak mutation, which we used to evaluate the effectiveness versus the efficiency of weak mutation. Additionally, we examined several options in an attempt to find the most appropriate way to implement weak mutation. Our results indicate that weak mutation can be applied in a manner that is almost as effective as mutation testing, and with significant computational savings.

[1] A. T. Acree, "On mutation," Ph.D. dissertation, Ga. Inst. of Technol., Atlanta GA, 1980.
[2] A. T. Acree, T. A. Budd, R. A. DeMillo, R. J. Lipton, and F. G. Sayword, "Mutation analysis," Tech. Rep. GIT-ICS-79/08, School of Inform. and Comput. Sci., Georgia Inst. of Technol., Atlanta GA, Sept. 1979.
[3] B. Choi and A. P. Mathur, "High performance mutation testing,"J. Syst. Software, vol. 20, pp. 135-152, Feb. 1993.
[4] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J. Offutt, "An extended overview of the Mothra software testing environment," inProc. Second Workshop Software Testing, Verification and Analysis, Banff, Alberta, July 1988, pp. 142-151.
[5] R. DeMillo, E. Krauser, and A. Mathur, Compiler Integrated Program Mutation,"Proc. Compsac, IEEE CS Press, Los Alamitos, 1991, pp. 351-356.
[6] R. A. Demillo, R. J. Lipton, and F. G. Sayward, "Hints on test data selection: Help for the practicing programmer,"IEEE Comput., vol. 11, no. 4, pp. 34-41, Apr. 1978.
[7] R. A. DeMillo and A. J. Offutt, "Constraint-based automatic test data generation,"IEEE Trans. Software Eng., vol. 17, pp. 900-910, Sept. 1991.
[8] K. Foster, "Error sensitive test case analysis,"IEEE Trans. Software Eng., vol. 6, no. 3, pp. 258-264, May 1980.
[9] M. R. Girgis and M. R. Woodward, "An integrated system for program testing using weak mutation and data flow analysis," inIEEE Proc. 8th Int. Conf. Software Eng., London, England, Aug. 1985, pp. 313-319.
[10] R. G. Hamlet, "Testing programs with the aid of a compiler,"IEEE Trans. Software Eng., vol. 3, no. 4, July 1977.
[11] J. R. Horgan and A. P. Mathur, "Weak mutation is probably strong mutation," Tech. Rep. SERC-TR-83-P, Software Eng. Res. Center, Purdue Univ., W. Laffayette, IN, Dec. 1990.
[12] W. E. Howden, "Algebraic program testing,"Acta Informatica, vol. 10, no. 1, pp. 53-66, 1978.
[13] W. E. Howden, "Weak mutation testing and completeness of test sets,"IEEE Trans. Software Eng., vol. 8, no. 4, pp. 371-379, July 1982.
[14] W. Howden,Functional Program Testing and Analysis. New York: Mc-Graw-Hill, 1987.
[15] K. N. King and A. J. Offutt, "A Fortran language system for mutation-based software testing,"Software Practice and Experience, to be published.
[16] E. W. Krauser, A. P. Mathur, and V. Rego, "High performance testing on SIMD machines," inProc. 2nd Workshop on Software Testing, Verification and Analysis, 1988, pp. 171-177.
[17] E. W. Krauser and A. P. Mathur, "Program testing on a massively parallel transputer based system," inProc. ISMM Int. Symp. Mini and Microcomputers and Their Applications, 1986, pp. 67-71.
[18] S. Lee, "Weak vs. strong: An empirical comparison of mutation variants," M. S. thesis, Dept. of Comput. Sci., Clemson Univ., Clemson SC, 1991.
[19] B. Marick, "The weak mutation hypothesis," inProc. 3rd Symp. Software Testing, Analysis and Verification, 1991, pp. 190-199.
[20] B. Marick,Using Weak Mutation with GCT. Champaign, IL: Testing Foundations, 1993.
[21] A. P. Mathur and E. W. Krauser, "Modeling mutation on a vector processor," inProc. Int. Conf. Software Engineering, Singapore, Apr. 1988.
[22] L. J. Morell, "A theory of error-based testing," Ph.D. dissertation, Dep. Comput. Sci., Univ. Maryland, Tech. Rep. TR-1395, Aug. 1984.
[23] L. J. Morell, "A theory of fault-based testing,"IEEE Trans. Software Eng., vol. 16, pp. 844-857, Aug. 1990.
[24] A. J. Offutt, "Automatic test data generation," Ph.D. thesis, Georgia Instit. Technol., Atlanta, 1988 (Tech. Rep. GIT-ICS 88/28) (also published as Software Eng. Res. Ctr., Purdue Univ., West Lafayette, IN, Tech. Rep. SERC-TR-25-P).
[25] A. J. Offutt, "Investigations of the software testing coupling effect,"ACM Trans. Software Engineering Methodology, vol. 1, pp. 5-20, Jan. 1992.
[26] A. J. Offutt, A. Lee, G. Rothermel, R. Untch, and C. Zapf, "An experimental determination of sufficient mutation operators," Tech. Rep. ISSE-TR-94-100, Dept. of Inform. and Software Syst. Eng., George Mason Univ., Fairfax VA, 1994.
[27] A. J. Offutt, R. Pargas, S. V. Fichter, and P. Khambekar, "Mutation testing of software using a MIMD computer," in1992 Int. Conf. Parallel Processing, 1992, vol. II, pp. 257-266.
[28] A. J. Offutt, G. Rothermel, and C. Zapf, "An experimental evaluation of selective mutation," inProc. 15th Int. Conf. Software Eng., 1993, pp. 100-107.
[29] D. J. Richardson and M. C. Thompson, "The RELAY model of error detection and its application," inProc. Second Workshop Software Testing, Verification, and Analysis. IEEE Computer Society, Los Angeles, CA, 1988.
[30] R. Untch, A. J. Offutt, and M. J. Harrold, "Mutation analysis using program schemata," inProc. 1993 Int. Symp. Software Testing and Analysis, 1993, pp. 139-148.
[31] M.R. Woodward and K. Halewood, "From weak to strong, dead or alive? An analysis of some mutation testing issues," inProc. 2nd Workshop Software Testing, Verification, and Analysis, Banff, AB, Canada, July 1988.

Index Terms:
program testing; software engineering; weak mutation; mutation testing; fault-based technique; unit-level software testing; effectiveness; efficiency; computational savings
A.J. Offutt, S.D. Lee, "An Empirical Evaluation of Weak Mutation," IEEE Transactions on Software Engineering, vol. 20, no. 5, pp. 337-344, May 1994, doi:10.1109/32.286422
Usage of this product signifies your acceptance of the Terms of Use.