This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Formal Analysis of the Subsume Relation Between Software Test Adequacy Criteria
April 1996 (vol. 22 no. 4)
pp. 248-255
ASCII Text
x 
 
Hong Zhu, "A Formal Analysis of the Subsume Relation Between Software Test Adequacy Criteria," IEEE Transactions on Software Engineering, vol. 22, no. 4, pp. 248-255, April, 1996.
 
 
BibTex
x
 
@article{ 10.1109/32.491648,
author = {Hong Zhu},
title = {A Formal Analysis of the Subsume Relation Between Software Test Adequacy Criteria},
journal ={IEEE Transactions on Software Engineering},
volume = {22},
number = {4},
issn = {0098-5589},
year = {1996},
pages = {248-255},
doi = {http://doi.ieeecomputersociety.org/10.1109/32.491648},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Software Engineering
TI - A Formal Analysis of the Subsume Relation Between Software Test Adequacy Criteria
IS - 4
SN - 0098-5589
SP248
EP255
EPD - 248-255
A1 - Hong Zhu,
PY - 1996
KW - Software testing
KW - test adequacy criteria
KW - subsume relation
KW - fault detecting ability
KW - testing scenarios.
VL - 22
JA - IEEE Transactions on Software Engineering
ER -
 

Abstract—Software test adequacy criteria are rules to determine whether a software system has been adequately tested. A central question in the study of test adequacy criteria is how they relate to fault detecting ability. In this paper, we identify two idealized software testing scenarios. In the first scenario, which we call prior testing scenario, software testers are provided with an adequacy criterion in addition to the software under test. The knowledge of the adequacy criterion is used to generate test cases. In the second scenario, which we call posterior testing scenario, software testers are not provided with the knowledge of adequacy criterion. The criterion is only used to decide when to stop the generation of test cases. In 1993, Frankl and Weyuker proved that the subsume relation between software test adequacy criteria does not guarantee better fault detecting ability in the prior testing scenario.

In this paper, we will investigate the posterior testing scenario and prove that in this scenario the subsume relation does guarantee a better fault detecting ability. Two measures of fault detecting ability will be used, the probability of detecting faults and the expected number of exposed errors.

[1] V.R. Basili and R.W. Selby, “Comparing the Effectiveness of Software Testing Strategies,” IEEE Trans. Software Eng., vol. 13, pp. 1,278-1,296, 1987.
[2] L.A. Clarke, J. Hassell, and D.J. Richardson, "A Close Look at Domain Testing," IEEE Trans. Software Engineering, vol. 8, no. 4, pp. 380-390, July 1982.
[3] L.A. Clarke, Formal Evaluation of Data Flow Path Selection Criteria IEEE Trans. Software Eng., vol. 15, no. 11, pp. 1318-1332, Nov. 1989.
[4] R.A. DeMillo, R.J. Lipton, and F.G. Sayward, "Hints on Test Data Selection: Help for the Practising Programmer," Computer, vol. 11, pp. 34-41, Apr. 1978.
[5] J.W. Duran and S. Ntafos, "An Evaluation of Random Testing," IEEE Trans. Software Engineering, vol. 10, no. 4, pp. 438-444, July 1984.
[6] P.G. Frankl and S.N Weiss,“An experimental comparison of the effectiveness of branch testing and data flow testing,” IEEE Transactions on Software Engineering, vol. 19, no. 8, Oct. 1993, pp. 774-787.
[7] P. Frankl and E.J. Weyuker, “An Applicable Family of Data Flow Testing Criteria,” IEEE Trans. Software Eng., vol. 14, no. 10, pp. 1,483–1,498, Oct. 1988.
[8] P.G. Frankl and E.J. Weyuker,“A formal analysis of the fault-detecting ability of testing methods,” IEEE Transactions on Software Engineering, vol. 19, no. 3, Mar. 1993, pp. 202-213,
[9] P.G. Frankl and E.J. Weyuker,“Provable improvements on branch testing,” IEEE Transactions on Software Engineering, vol. 19, no. 10, Oct. 1993, pp. 962-975.
[10] A. Gopal and T. Budd, "Program Testing by Specification Mutation," Tech. Report TR 83-17, Univ. of Arizona, Nov. 1983.
[11] J.B. Goodenough and S.L. Gerhart, "Toward a Theory of Test Data Selection," IEEE Trans. Software Engineering, vol. 3, June 1975.
[12] J.A. Gourlay, "A Mathematical Framework for the Investigation of Testing" IEEE Trans. Software Engineering, vol. 9, no. 6, pp. 686-709, Nov. 1983.
[13] W.E. Howden, "Weak Mutation Testing and Completeness of Test Sets," IEEE Trans. Software Engineering, vol. 8, no. 4, pp. 371-379, July 1982.
[14] J. Laski and B. Korel, "A Data Flow Oriented Program Testing Strategy," IEEE Trans. Software Engineering, vol. 9, pp. 33-43, May 1983.
[15] T.J. McCabe, "A Complexity Measure," IEEE Trans. Software Engineering, vol. 2, no. 4, pp. 308-320, 1976.
[16] T.J. McCabe, ed., Structured Testing. IEEE CS Press, 1983.
[17] T.J. McCabe and G.G. Schulmeyer, "System Testing Aided by Structured Analysis: A Practical Experience," IEEE Trans. Software Engineering, vol. 1, no. 9, pp. 917-921, Sept. 1985.
[18] G.J. Myers,The Art of Software Testing.New York: Wiley, 1979.
[19] S.C. Ntafos, "On Required Element Testing," IEEE Trans. Software Engineering, vol. 10, no. 6, pp. 795-803, Nov. 1984.
[20] S. Ntafos, “A Comparison of Some Structural Testing Strategies,” IEEE Trans. Software Eng., vol. 14, no. 6, pp. 868–874, June 1988.
[21] M.R. Paige, "Program Graphs, an Algebra, and Their Implication for Programming," IEEE Trans. Software Engineering, vol. 1, no. 3, pp. 286-291. Sept. 1975.
[22] M.R. Paige, "An Analytical Approach to Software Testing," Proc. COMPSAC'78, pp. 527- 532, 1978.
[23] A. Parrish and S.H. Zweben, "Analysis and Refinement of Software Test Data Adequacy Properties," IEEE Trans. Software Eng., vol. 17, no. 6, pp. 565-581, June 1991.
[24] A.S. Parrish and S.H. Zweben, "Clarifying Some Fundamental Concepts in Software Testing," IEEE Trans. Software Engineering, vol. 19, no. 7, pp. 742-746, July 1993.
[25] S. Rapps and E.J. Weyuker, “Selecting Software Test Data Using Data Flow Information,” IEEE Trans. Software Eng., vol. 11, no. 4, pp. 367–375, Apr. 1985.
[26] M. Tsoukalas, M.J. Duran, and S. Ntafos, “On Some Reliability Estimation Problems in Random and Partition Testing,” IEEE Trans. Software Eng., vol. 19, no. 7, pp. 687-697, July 1993.
[27] L.J. White and E.I. Cohen, "A Domain Strategy for Computer Program Testing," IEEE Trans. Software Engineering, vol. 6, no. 3, pp. 247-257, May 1980.
[28] M.R. Woodward, D. Hedley, and M.A. Hennel, "Experience with Path Analysis and Testing of Programs," IEEE Trans. Software Engineering, vol. 6, no. 5, pp. 278-286, May, 1980.
[29] M.R. Woodward and K. Halewood, "From Weak to Strong, Dead or Alive? An Analysis of Some Mutation Testing Issues," Proc. Second Workshop Software Testing, Verification and Analysis,Banff, Canada, pp. 152-158, July 1988.
[30] P.E. Buis and W.R. Dyksen, "Efficient Vector and Parallel Manipulation of Tensor Products," ACM Trans. Math. Software, vol. 22, no. 1, pp. 18-23, 1996.
[31] H. Zhu and P. Hall, "Test Data Adequacy Measurement," Software Engineering J., vol. 8, no. 1, pp. 21-30, Jan. 1993.
[32] H. Zhu, P.A.V. Hall, and J.H.R. May, "Understanding Software Test Adequac—An Axiomatic and Measurement Theory Approach," Math. of Dependable Systems. Proc. IMA Conf. MDS'93, Mitchell and Stavridou, eds., Oxford Univ. Press, 1995.
[33] H. Zhu, P.A.V. Hall, and J.H.R. May, "Software Test Coverage and Adequacy," Dept. of Computing, The Open Univ., UK, Tech. Report No. 94/15, Aug. 1994. Submitted to ACM Computing Survey.

Index Terms:
Software testing, test adequacy criteria, subsume relation, fault detecting ability, testing scenarios.
Citation:
Hong Zhu, "A Formal Analysis of the Subsume Relation Between Software Test Adequacy Criteria," IEEE Transactions on Software Engineering, vol. 22, no. 4, pp. 248-255, April 1996, doi:10.1109/32.491648
Usage of this product signifies your acceptance of the Terms of Use.