This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Interface Mutation: An Approach for Integration Testing
March 2001 (vol. 27 no. 3)
pp. 228-247

Abstract—The need for test adequacy criteria is widely recognized. Several criteria have been proposed for the assessment of adequacy of tests at the unit level. However, there remains a lack of criteria for the assessment of the adequacy of tests generated during integration testing. We present a mutation-based interprocedural criterion, named Interface Mutation (IM), suitable for use during integration testing. A case study to evaluate the proposed criterion is reported. In this study, the UNIX sort utility was seeded with errors and Interface Mutation evaluated by measuring the cost of its application and its error revealing effectiveness. Alternative IM criteria using different sets of Interface Mutation operators were also evaluated. While comparing the error revealing effectiveness of these Interface Mutation-based test sets with same size randomly generated test sets we observed that in most cases Interface Mutation-based test sets are superior. The results suggest that Interface Mutation offers a viable test adequacy criteria for use at the integration level.

[1] H. Agrawal, R.A. DeMillo, R. Hataway, W. Hsu, W. Hsu, E. Krauser, R.J. Martin, A.P. Mathur, and E.H. Spafford, “Design of Mutant Operators for C Programming Language,” Technical Report SERC-TR41-P, Software Eng. Research Center, Purdue Univ. Mar. 1989.
[2] T.A. Budd and D. Angluin, “Two Notions of Correctness and their Relation toTesting,” Acta Informatica, vol. 18, no. 1, pp. 31–45, Nov. 1982.
[3] B.J. Choi, R.A. DeMillo, E.W. Krauser, A.P. Mathur, R.J. Martin, A.J. Offutt, H. Pan, and E.H. Spafford, “The Mothra Toolset,” Proc. 22nd Ann. Hawaii Int'l Conf. System Sciences, Jan. 1989.
[4] M.E. Delamaro, “Mutação de Interface: Um Critério de Adequação Inter-procedimental para o Teste de Integração,” Doctoral dissertation, Physics Inst. São Carlos—University of São Paulo, São Carlos, SP, June 1997.
[5] M.E. Delamaro and J.C. Maldonado, “Proteum—A Tool for the Assesment of Test Adequacy for C, Programs,” Proc. Conf. Performability in Computing Systems (PCS 96), pp. 79–95, July 1996.
[6] M.E. Delamaro, J.C. Maldonado, and A.P. Mathur, “Integration Testing Using Interface Mutation,” Proc. VII Int'l Symp. Software Reliability Engineering (ISSRE '96), pp. 112–121, Nov. 1996.
[7] M.E. Delamaro, J.C. Maldonado, A. Pasquini, and A.P. Mathur, “Interface Mutation Test Adequacy Criterion: An Empirical Evaluation,” Technical Report 83, ICMSC–USP, São Carlos—SP, Jan. 1999.
[8] R.A. DeMillo, R.J. Lipton, and F.G. Sayward, “Hints on Test Data Selection: Help for the Practicing Programmer,” IEEE Computer, vol. 11, no. 4, Apr. 1978.
[9] R.A. DeMillo and A.P. Mathur, “A Grammar Based Fault Classification Scheme and its Application to the Classification of the Errors of TEX,” Technical Report SERC-TR165-P, Software Eng. Research Center, Purdue Univ. Sept. 1995.
[10] R.A. DeMillo and A.J. Offutt, “Constraint Based Automatic Test Data Generation,” IEEE Trans. Software Eng., vol. 17, no. 9, pp. 900–910, Sept. 1991.
[11] S.C.P.F. Fabbri, J.C. Maldonado, P.C. Masiero, and M.E. Delamaro, “Mutation Analysis Testing for Finite State Machines,” Proc. Fifth Int'l Symp. Software Reliability Eng (ISSRE), pp. 220–229, Nov. 1994.
[12] S.C.P.F. Fabbri, J.C. Maldonado, P.C. Masiero, and M.E. Delamaro, “Mutation Analisys Applied to Validate Specifications Based on Petri Nets,” Proc. Eighth IFIP Conf. Formal Descriptions Techniques for Distribute Systems and Communication Protocols, pp. 329–337, Oct. 1995.
[13] P.G. Frankl and E.J. Weyuker,“A formal analysis of the fault-detecting ability of testing methods,” IEEE Transactions on Software Engineering, vol. 19, no. 3, Mar. 1993, pp. 202-213,
[14] A. Haley and S. Zweben, “Development and Application of a White Box Approach to Integration Testing,” The J. Systems and Software, vol. 4, pp. 309–315, 1984.
[15] R.G. Hamlet, “Testing Programs with the Aid of a Compiler,” IEEE Trans. Software Eng., vol. 3, no. 4, July 1977.
[16] M.J. Harrold and M.L. Soffa, “Selecting and Using Data for Integration Testing,” IEEE Software, vol. 8, no. 2, pp. 58–65, Mar. 1991.
[17] Z. Jin and A.J. Offut, “Integration Testing Based on Software Couplings,” Proc. 10th Ann. Conf. Computer Assurance (COMPASS '95), pp. 13–23, Jan. 1995.
[18] U. Linnenkugel and M. Müllerburg, “Test Data Selection Criteria for (Software) IntegrationTesting,” Proc. First Int'l Conf. Systems Integration, pp. 709–717, Apr. 1990.
[19] A.J. Offut, J. Pan, K. Tewary, and T. Zhang, “An Experimental Evaluation of Data Flow and Mutation Testing,” Software Practice and Experience, vol. 26, no. 2, pp. 165–176, Feb. 1996.
[20] A.J. Offutt, A. Lee, G. Rothermel, R. Untch, and C. Zapf, “An Experimental Determination of Sufficient Mutation Operators,” ACM Trans. Software Eng. Methods, vol. 5, no. 2, pp. 99–118, Apr. 1996.
[21] A.J. Offutt, C.Z. Rothermel, and C. Zapf, "An Experimental Evaluation of Selective Mutation," Proc. Int'l Conf. Software Eng., pp. 100-107, 1993.
[22] S. Rapps and E.J. Weyuker, “Selecting Software Test Data Using Data Flow Information,” IEEE Trans. Software Eng., vol. 11, no. 4, pp. 367–375, Apr. 1985.
[23] W.E. Wong,“On mutation and data flow,” PhD thesis, Department of Computer Sciences, Purdue University, West Lafayette, Ind., Dec. 1993 (also available as SERC-TR-149-P, Software EngineeringResearch Center, Purdue University).
[24] W.E. Wong et al., "Effect of Test Size and Block Coverage on Fault Detection Effectiveness," Fifth Int'l Symp. Software Reliability Eng., IEEE CS Press, Los Alamitos, Calif., 1994, pp. 230-238.
[25] W.E. Wong, J.C. Maldonado, M.E. Delamaro, and A.P. Mathur, “Constrained Mutation in C Programs,” Proc. Eighth Brazilian Symp. Software Eng., pp. 439–452, Oct. 1994.
[26] W.E. Wong and A.P. Mathur, “How Strong is Constrained Mutation in Fault Detection?” Proc. Int'l Computer Symp., pp. 515–520, Dec. 1994.
[27] W.E. Wong and A.P. Mathur, “Fault Detection Effectiveness of Mutation and Data Flow Testing,” Software Quality J., vol. 4, no. 1, pp. 69–83, Mar. 1995.
[28] W.E. Wong and A.P. Mathur, “Reducing the Cost of Mutation Testing: An Empirical Study,” The J. Systems and Software, vol. 31, no. 3, pp. 185–196, Dec. 1995.
[29] W.E. Wong, A.P. Mathur, and J.C. Maldonado, “Mutation Versus All-uses: An Empirical Evaluation of Cost, Strength, and Effectiveness,” Proc. Int'l Conf. Software Quality and Productivity, pp. 258–265, Dec. 1994.
[30] M.R. Woodward, “Mutation Testing—its Origin and Evolution,” Information and Software Technology, vol. 35, no. 3, pp. 163–169, Mar. 1993.

Index Terms:
Test adequacy criteria, mutation testing, mutation analysis, integration testing, testing tool.
Citation:
Márcio E. Delamaro, José C. Maldonado, Aditya P. Mathur, "Interface Mutation: An Approach for Integration Testing," IEEE Transactions on Software Engineering, vol. 27, no. 3, pp. 228-247, March 2001, doi:10.1109/32.910859
Usage of this product signifies your acceptance of the Terms of Use.