This Article 
 Bibliographic References 
 Add to: 
Constraint-Based Automatic Test Data Generation
September 1991 (vol. 17 no. 9)
pp. 900-910

A novel technique for automatically generating test data is presented. The technique is based on mutation analysis and creates test data that approximate relative adequacy. It is a fault-based technique that uses algebraic constraints to describe test cases designed to find particular types of faults. A set of tools (collectively called Godzilla) that automatically generates constraints and solves them to create test cases for unit and module testing has been implemented. Godzilla has been integrated with the Mothratesting system and has been used as an effective way to generate test data that kill program mutants. The authors present an initial list of constraints and discuss some of the problems that have been solved to develop the complete implementation of the technique.

[1] A. T. Acree, T. A. Budd, R. A. DeMillo, R. J. Lipton, and F. G. Sayward, "Mutation analysis," School of Inform. and Comput. Sci., Georgia Instit. Technol., Atlanta, Tech. Rep. GIT-ICS-79/08, Sept. 1979.
[2] D. Baldwin and F. Sayward, "Heuristics for determining equivalence of program mutations," Dept. Comput. Sci., Yale Univ., New Haven, CT, Res. Rep. 276, 1979.
[3] M. S. Bazaraa and J. J. Jarvis,Linear Programming and Network Flows. New York: Wiley, 1977.
[4] T. A. Budd, "Mutation analysis of program test data," Ph.D. thesis, Yale Univ., New Haven CT, 1980.
[5] T. A. Budd and D. Angluin, "Two notions of correctness and their relation to testing,"Acta Informatica, vol. 18, no. 1, pp. 31-45, Nov. 1982.
[6] L. A. Clarke, "A system to generate test data and symbolically execute programs,"IEEE Trans. Software Eng., vol. SE-2, pp. 215-222, Sept. 1976.
[7] L. A. Clarke, A. Podgurski, D. J. Richardson, and S. J. Zeil, "A comparison of data flow path selection criteria," inProc. 8th Inf. Conf. Sofware Eng.(London UK), Aug. 1985. Los Alamitos, CA: IEEE Computer Soc., 1985, pp. 244-251.
[8] L. A. Clarke and D. J. Richardson, "The application of error-sensitive testing strategies to debugging," inSymp. High-Level Debugging, ACM SIGSOFT/SIGPLAN, Mar. 1983, pp. 45-52.
[9] W. M. Craft, "Detecting equivalent mutants using compiler optimization techniques," Master's thesis, Dept. Comput. Sci., Clemson Univ., Clemson SC, 1989.
[10] R. A. DeMillo, D. S. Guindi, K. N. King, W. M. McCracken, and A. J. Offutt, "An extended overview of the Mothra software testing environment," inProc. Second Workshop Software Testing, Verification and Analysis, Banff, Alberta, July 1988, pp. 142-151.
[11] R. A. DeMillo, E. W. Krauser, R. J. Martin, A. J. Offutt, and E. H. Spafford, "The Mothra tool set," inProc. 22nd Hawaii Int. Conf. System SLiences(Kailua-Kona HI), Jan. 1989, pp. 275-284.
[12] R. A. DeMillo, R. J. Lipton, and F. G. Sayward, "Hints on test data selection: Help for the practicing programmer,"IEEE Computer, vol. C-11, pp. 34-41, Apr. 1978.
[13] R. A. DeMillo and A. J. Offutt, "Experimental results of automatically generated adequate test sets," inProc. 6th Ann. Pacific Northwest Software Quality Conf.(Portland OR), Sept. 1988, pp. 209-232.
[14] R. A. DeMillo, F. G. Sayward, and R. J. Lipton, "Program mutation: A new approach to program testing," inInfotech International State of the Art Report: Program-Testing. Infotech Int., 1979, pp. 107-126.
[15] R. A. DeMillo and E. H. Spafford, "The Mothra software testing environment," inProc. 11th NASA Software Eng. Lab. Workshop, Goddard Space Ctr., Dec. 1986.
[16] P. G. Frankl, "The use of data flow information for the selection and evaluation of software test data," Doctoral dissertation, New York Univ., New York, 1987.
[17] M. R. Girgis and M. R. Woodward, "An experimental comparison of the error exposing ability of program testing criteria," inProc. Workshop on Software Test., July 1986. Los Alamitos, CA: IEEE Comput. Soc., 1986, pp. 64-73.
[18] M. J. Harrold, R. Gupta, and M. L. Soffa, "TBM: A testbed management tool," inProc. 7th Int. Conf. Testing Comput. Software, ACM SIGSOFT(San Francisco, CA), June 1990, pp. 47-56.
[19] J. R. Horgan and A. P. Mathur, "Weak mutation is probably strong mutation," Software Eng. Res. Ctr., Purdue Univ., West Lafayette, IN, Tech. Rep. SERC-TR-83-P, Dec. 1990.
[20] W. E. Howden, "Reliability of the path analysis testing strategy,"IEEE Trans. Software Eng., vol. SE-2, pp. 208-215, Sept. 1976.
[21] W. E. Howden, "Weak mutation testine and completeness of test sets,"IEEE Trans. Software Eng., vol. SE-8, pp. 371-379, July 1982.
[22] W. Howden,Functional Program Testing and Analysis. New York: Mc-Graw-Hill, 1987.
[23] J. C. Huang, "An approach to program testing,"ACM Comput. Surveys, vol. 7, no. 3, pp. 113-128, Sept. 1975.
[24] J. C. King, "Symbolic execution and program testing,"Commun. ACM, vol. 19, no. 7, pp. 385-394, 1976.
[25] K. N. King and A. J. Offutt, "A Fortran language system for mutation-based software testing,"Software Practice and Experience, to be published.
[26] Z. Manna,Mathematical Theory of Computation. New York: McGraw-Hill, 1974.
[27] L. J. Morell, "A theory of error-based testing," Ph.D. dissertation, Dep. Comput. Sci., Univ. Maryland, Tech. Rep. TR-1395, Aug. 1984.
[28] L. J. Morell, "Theoretical insights into fault-based testing," inProc. 2nd Workshop on Software Testing, Verification, and Analysis(Banff, AB, Can.), July 1988. Los Alamitos, CA: IEEE Computer Soc., 1988, pp. 45-62.
[29] G. J. Myers,The Art of Software Testing. New York: Wiley, 1979.
[30] A. J. Offutt, "Automatic test data generation," Ph.D. thesis, Georgia Instit. Technol., Atlanta, 1988 (Tech. Rep. GIT-ICS 88/28) (also published as Software Eng. Res. Ctr., Purdue Univ., West Lafayette, IN, Tech. Rep. SERC-TR-25-P).
[31] A. J. Offutt, "An integrated system for automatically generating test data," inProc. 1990 Conf. Systems Integration(Morristown, NJ), Apr. 1990, Los Alamitos, CA: IEEE Computer Soc., 1990, pp. 694-701.
[32] P. Purdom and C. Brown,The Analysis of Algorithms. New York: Holt, Reinhart and Winston, 1985.
[33] C. V. Ramamoorthy, S. F. Ho, and W. T. Chen, "On the automated generation of program test data,"IEEE Trans. Software Eng., vol. SE-2, pp. 293-300, Dec. 1976.
[34] D. J. Richardson and M. C. Thompson, "The RELAY model of error detection and its application," inProc. Second Workshop Software Testing, Verification, and Analysis. IEEE Computer Society, Los Angeles, CA, 1988.
[35] L. J. White and E. I. Cohen, "A domain strategy for computer program testing,"IEEE Trans. Software Eng., vol. SE-6, pp. 247-257, May 1980.

Index Terms:
constraint-based data generation; automatic test data generation; mutation analysis; relative adequacy; fault-based technique; algebraic constraints; Godzilla; module testing; Mothra testing system; computational complexity; program testing
R.A. DeMilli, A.J. Offutt, "Constraint-Based Automatic Test Data Generation," IEEE Transactions on Software Engineering, vol. 17, no. 9, pp. 900-910, Sept. 1991, doi:10.1109/32.92910
Usage of this product signifies your acceptance of the Terms of Use.