This Article 
 Bibliographic References 
 Add to: 
The Cost of Data Flow Testing: An Empirical Study
February 1990 (vol. 16 no. 2)
pp. 121-128

A family of test data adequacy criteria employing data-flow information was previously proposed, and a theoretical complexity analysis was performed. The author describes an empirical study to determine the actual cost of using these criteria. The aim is to establish the practical usefulness of these criteria in testing software and provide a basis for predicting the amount of testing needed for a given program. The first goal of the study is to confirm the belief that the family of software testing criteria considered is practical to use. An attempt is made to show that even as the program size increases, the amount of testing, expressed in terms of the number of test cases sufficient to satisfy a given criterion, remains modest. Several ways of evaluating this hypothesis are explored. The second goal is to provide the prospective user of these criteria with a way of predicting the number of test cases that will be needed to satisfy a given criterion for a given program. This provides testers with a basis for selecting the most comprehensive criterion that they can expect to satisfy. Several plausible bases for such a prediction are considered.

[1] L. A. Clarke, A. Podgurski, D. J. Richardson, and S. J. Zeil, "A comparison of data flow path selection criteria," inProc. 8th Inf. Conf. Sofware Eng.(London UK), Aug. 1985. Los Alamitos, CA: IEEE Computer Soc., 1985, pp. 244-251.
[2] P. G. Frankl, "The use of data flow information for the selection and evaluation of software test data," Doctoral dissertation, New York Univ., New York, 1987.
[3] P. G. Frankl and E. J. Weyuker, "A data flow testing tool," inProc. IEEE Softfair II, San Francisco, CA, Dec. 1985.
[4] P. G. Frankl and E. J. Weyuker, "Data flow testing in the presence of unexecutable paths," inProc. Workshop Software Testing, Banff, Alta., Canada, July 1986, pp. 4-13.
[5] P. G. Frankl and E. J. Weyuker, "An applicable family of data flow testing criteria,"IEEE Trans. Software Eng., vol. 14, no. 10, pp. 1483-1498, Oct. 1988.
[6] P. G. Frankl, S. N. Weiss, and E. J. Weyuker, "ASSET: A system to select and evaluate tests," inProc. IEEE Conf. Software Tools, New York, Apr. 1985.
[7] P. M. Herman, "A data flow analysis approach to program testing,"Australian Comput. J., vol. 8, no. 3, Nov. 1976.
[8] B. W. Kernighan and P. J. Plauger,Software Tools in Pascal. Reading MA: Addison-Wesley, 1981.
[9] J. W. Laski and B. Korel, "A data flow oriented program testing strategy,"IEEE Trans. Software Eng., vol. SE-9, no. 3, pp. 347- 354, May 1983.
[10] S. Ntafos, "On required element testing,"IEEE Trans. Software Eng., vol. SE-10, no. 6, pp. 795-803, Nov. 1984.
[11] S. Ntafos, "A comparison of some structural testing strategies,"IEEE Trans. Software Eng., vol. 14, no. 6, pp. 868-874, June 1988.
[12] T. J. Ostrand and E. J. Weyuker, "Using data flow analysis for regression testing," inProc. Sixth Annu. Pacific Northwest Software Quality Conf., Portland, OR, Sept. 1988, pp. 233-248.
[13] S. Rapps and E. J. Weyuker, "Data flow analysis techniques for program test data selection," inProc. 6th Int. Conf. Software Eng., Sept. 1982, pp. 272-278.
[14] S. Rapps and E. J. Weyuker, "Selecting software test data using data flow information,"IEEE Trans. Software Eng., vol. SE-11, no. 4, pp. 367-375, Apr. 1985.
[15] T. J. Shimeall, "An experiment in software fault elimination and fault tolerance," Ph.D. dissertation, Univ. California, Irvine, 1989.
[16] T. J. Shimeall and N. G. Leveson, "An empirical comparison of software fault tolerance and fault elimination," inProc. Second Workshop Software Test. Verification Analysis, Banff, Canada, July 1988.
[17] E. J. Weyuker, "The complexity of data flow criteria for test data selection,"Inform. Processing Lett., vol. 19, no. 2, pp. 103-109, Aug. 1984.

Index Terms:
data flow testing; empirical study; test data adequacy criteria; data-flow information; theoretical complexity analysis; cost; software testing criteria; parallel programming; program testing.
E.J. Weyuker, "The Cost of Data Flow Testing: An Empirical Study," IEEE Transactions on Software Engineering, vol. 16, no. 2, pp. 121-128, Feb. 1990, doi:10.1109/32.44376
Usage of this product signifies your acceptance of the Terms of Use.