This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Practical Combinatorial Testing: Beyond Pairwise
May/June 2008 (vol. 10 no. 3)
pp. 19-23
ASCII Text
x 
 
Rick Kuhn, Yu Lei, Raghu Kacker, "Practical Combinatorial Testing: Beyond Pairwise," IT Professional, vol. 10, no. 3, pp. 19-23, May/June, 2008.
 
 
BibTex
x
 
@article{ 10.1109/MITP.2008.54,
author = {Rick Kuhn and Yu Lei and Raghu Kacker},
title = {Practical Combinatorial Testing: Beyond Pairwise},
journal ={IT Professional},
volume = {10},
number = {3},
issn = {1520-9202},
year = {2008},
pages = {19-23},
doi = {http://doi.ieeecomputersociety.org/10.1109/MITP.2008.54},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - MGZN
JO - IT Professional
TI - Practical Combinatorial Testing: Beyond Pairwise
IS - 3
SN - 1520-9202
SP19
EP23
EPD - 19-23
A1 - Rick Kuhn,
A1 - Yu Lei,
A1 - Raghu Kacker,
PY - 2008
KW - all-pairs testing
KW - assurance
KW - combinatorial testing
KW - debugging
KW - faults
KW - pairwise testing
VL - 10
JA - IT Professional
ER -
 
Rick Kuhn, US National Institute of Standards and Technology
Yu Lei, University of Texas, Arlington
Raghu Kacker, US National Institute of Standards and Technology
Many software developers have encountered failures that occur only as the result of an interaction between two components. For example, a Web application might work correctly on Linux/Apache or Windows/IIS platforms but fail when run on a Windows XP box running an Apache server. The application error is triggered by an interaction of two components: the operating system and the Web server. Testers often use pairwise testing—all pairs of parameter values—to detect such interactions. Combinatorial testing beyond pairwise is rarely used because good algorithms for higher strength combinations, such as four-way or more, have not been available. Nevertheless, empirical evidence shows that some errors are triggered only by the interaction of three, four, or more parameters. With new algorithms and tools, developers can apply high-strength combinatorial testing capable of detecting these elusive errors.

1. K. Burr and W. Young, "Combinatorial Test Techniques: Table-Based Automation, Test Generation, and Test Coverage," Proc. Int'l Conf. Software Testing, Analysis, and Review (STAR), 1998; http://aetgweb.argreenhouse.com/papers1998-star.pdf .
2. D.R. Kuhn, D.R. Wallace, and A. Gallo, "Software Fault Interactions and Implications for Software Testing," IEEE Trans. Software Eng., vol. 30, no. 6, 2004, pp. 418–421.
3. R. Bryce and C.J. Colbourn, "A Density-Based Greedy Algorithm for Higher Strength Covering Arrays," to be published in J. Software Testing, Verification, and Reliability; www.egr.unlv.edu/~rbryceresearch.htm.
4. R. Bryce and C.J. Colbourn, "The Density Algorithm for Pairwise Interaction Testing," J. Software Testing, Verification, and Reliability, vol. 17, no. 3, Aug. 2007, pp. 159–182.
5. M. Sutton, A. Greene, and P. Amini, Fuzzing: Brute Force Vulnerability Discovery, Addison-Wesley, 2007.
6. G.T. Leavens, A.L. Baker, and C. Ruby, "JML: A Notation for Detailed Design," H. Kilov, B. Rumpe, and I. Simmonds, eds., Behavioral Specifications of Businesses and Systems, Kluwer, 1999, pp. 175–188.
7. L. du Bousquet et al., "A Case Study in JML-Based Software Validation," Proc. 19th Int'l IEEE Conf. Automated Software Eng. (ASE 04), IEEE CS Press, 2004, pp. 294–297.
8. P. Ammann and P.E. Black, "Abstracting Formal Specifications to Generate Software Tests via Model Checking," Proc. 18th Digital Avionics Systems Conf., vol. 2., IEEE Press, 1999, pp. 10.A.6.1–10.
9. D.R. Kuhn and V. Okun, "Pseudo-Exhaustive Testing for Software," Proc. 30th NASA/IEEE Software Eng. Workshop, 2006, pp. 153–158; http://csrc.nist.gov/actsPID258305.pdf.
10. M.B. Cohen, J. Snyder, and G. Rothermel, "Testing Across Configurations: Implications for Combinatorial Testing," Proc. Workshop on Advances in Model-Based Software Testing, IEEE Press, 2006, pp. 1–9.

Index Terms:
all-pairs testing, assurance, combinatorial testing, debugging, faults, pairwise testing
Citation:
Rick Kuhn, Yu Lei, Raghu Kacker, "Practical Combinatorial Testing: Beyond Pairwise," IT Professional, vol. 10, no. 3, pp. 19-23, May-June 2008, doi:10.1109/MITP.2008.54
Usage of this product signifies your acceptance of the Terms of Use.