Understanding and Measuring the Sources of Variation in the Prioritization of Regression Test Suites
Software Metrics, IEEE International Symposium on (2001)
Apr. 4, 2001 to Apr. 6, 2001
Sebastian Elbaum , University of Nebraska
David Gable , University of Nebraska
Gregg Rothermel , Oregon State University
Test case prioritization techniques let testers order their test cases so that those with higher priority, according to some criterion, are executed earlier than those with lower priority. In previous work, we examined a variety of prioritization techniques to determine their ability to improve the rate of fault detection of test suites. Our studies showed that the rate of fault detection of test suites could be significantly improved by using more powerful prioritization techniques. In addition, they indicated that rate of fault detection was closely associated with the target program. We also observed a large quantity of unexplained variance, indicating that other factors must be affecting prioritization effectiveness. These observations motivate the following research questions: (1) Are there factors other than the target program and the prioritization technique that consistently affect the rate of fault detection of test suites? (2) What metrics are most representative of each factor? (3) Can the consideration of additional factors lead to more efficient prioritization techniques? To address these questions, we performed a series of experiments exploring three factors: program structure, test suite composition, and change characteristics. This paper reports the results and implications of those experiments.
S. Elbaum, D. Gable and G. Rothermel, "Understanding and Measuring the Sources of Variation in the Prioritization of Regression Test Suites," Software Metrics, IEEE International Symposium on(METRICS), London, England, 2001, pp. 169.