Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach
Issue No.05 - Sept.-Oct. (2008 vol.34)
Matthew B. Dwyer , University of Nebraska-Lincoln, Lincoln
Myra B. Cohen , University of Nebraska-Lincoln, Lincoln
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TSE.2008.50
Researchers have explored the application of combinatorial interaction testing (CIT) methods to construct samples to drive systematic testing of software system configurations. Applying CIT to highly-configurable software systems is complicated by the fact that, in many such systems, there are constraints between specific configuration parameters that render certain combinations invalid. Many CIT algorithms lack a mechanism to avoid these. In recent work, automated constraint solving methods have been combined with search-based CIT construction methods to address the constraint problem with promising results. However, these techniques can incur a non-trivial overhead. In this paper, we build upon our previous work to develop a family of greedy CIT sample generation algorithms that exploit calculations made by modern boolean satisfiability (SAT) solvers to prune the search space of the CIT problem. We perform a comparative evaluation of the cost-effectiveness of these algorithms on four real-world highly-configurable software systems and on a population of synthetic examples that share the characteristics of those systems. In combination our techniques reduce the cost of CIT in the presence of constraints to 30\% of the cost of widely-used unconstrained CIT methods without sacrificing the quality of the solutions.
Testing strategies, Testing tools
Matthew B. Dwyer, Myra B. Cohen, "Constructing Interaction Test Suites for Highly-Configurable Systems in the Presence of Constraints: A Greedy Approach", IEEE Transactions on Software Engineering, vol.34, no. 5, pp. 633-650, Sept.-Oct. 2008, doi:10.1109/TSE.2008.50