This Article 
 Bibliographic References 
 Add to: 
A Look at 25 Years of Data
January/February 2009 (vol. 26 no. 1)
pp. 15-17
Natalia Juristo, Universidad Politecnica de Madrid
Ana Moreno, Universidad Politecnica de Madrid
Sira Vegas, Universidad Politecnica de Madrid
Forrest Shull, Fraunhofer Center for Experimental Software Engineering, Maryland
Is 25 years enough time to build up a coherent body of knowledge that can help point to useful principles? As a testbed for helping us answer this question, software testing techniques are a good place to start. Few software practices are as important as testing, and testing techniques are amenable to measurement and reasoning about their effectiveness. Because they're aimed at removing faults, measuring the number and type of such removed faults seems like a natural part of applying these techniques. To make sense of this data, Universidad Politecnica de Madrid researchers have spent some time worrying about how to put 25 years' worth of work together usefully.

1. N. Juristo, A.M. Moreno, and S. Vegas, "Reviewing 25 Years of Testing Technique Experiments," Empirical Software Eng., vol. 9, nos. 1–2, 2004, pp. 7–44.
2. P.G. Frankl and S.N. Weiss, "An Experimental Comparison of the Effectiveness of Branch Testing and Data Flow Testing," IEEE Trans. Software Eng., vol. 19, no. 8, 1993, pp. 774–787.
3. M. Hutchins et al., "Experiments on the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria," Proc. 16th Int'l Conf. Software Eng., IEEE Press, 1994, pp. 191–200.
4. P.G. Frankl and O. Iakounenko, "Further Empirical Studies of Test Effectiveness," Proc. ACM SIGSOFT Int'l Symp. Foundations of Software Eng., ACM Press, 1998, pp. 153–162.
5. G.J. Myers, "A Controlled Experiment in Program Testing and Code Walkthroughs/Inspections," Comm. ACM, vol. 21, no. 9, 1978, pp. 760–768.
6. V.R. Basili and R.W. Selby, "Comparing the Effectiveness of Software Testing Strategies," IEEE Trans. Software Eng., vol. 13, no. 12, 1987, pp. 1278–1296.
7. E. Kamsties and C.M. Lott, "An Empirical Evaluation of Three Defect-Detection Techniques," Proc. 5th European Software Eng. Conf., Springer, 1995, pp. 362–383.
8. M. Wood et al., "Comparing and Combining Software Defect Detection Techniques: A Replicated Empirical Study," Proc. 6th European Software Eng. Conf., Springer, 1997, pp. 262–277.
9. E. Wong and A.P. Mathur, "Fault Detection Effectiveness of Mutation and Data-Flow Testing," Software Quality J., vol. 4, no. 1, 1995, pp. 69–83.
10. P.G. Frankl, S.N. Weiss, and C. Hu, "All-Uses vs. Mutation Testing: An Experimental Comparison of Effectiveness," J. Systems and Software, vol. 38, Sept. 1997, pp. 235–253.

Index Terms:
software testing, data flow testing, control flow testing
Natalia Juristo, Ana Moreno, Sira Vegas, Forrest Shull, "A Look at 25 Years of Data," IEEE Software, vol. 26, no. 1, pp. 15-17, Jan.-Feb. 2009, doi:10.1109/MS.2009.2
Usage of this product signifies your acceptance of the Terms of Use.