The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - May/June (2008 vol.10)
pp: 19-23
Yu Lei , University of Texas, Arlington
Rick Kuhn , US National Institute of Standards and Technology
ABSTRACT
Many software developers have encountered failures that occur only as the result of an interaction between two components. For example, a Web application might work correctly on Linux/Apache or Windows/IIS platforms but fail when run on a Windows XP box running an Apache server. The application error is triggered by an interaction of two components: the operating system and the Web server. Testers often use pairwise testing—all pairs of parameter values—to detect such interactions. Combinatorial testing beyond pairwise is rarely used because good algorithms for higher strength combinations, such as four-way or more, have not been available. Nevertheless, empirical evidence shows that some errors are triggered only by the interaction of three, four, or more parameters. With new algorithms and tools, developers can apply high-strength combinatorial testing capable of detecting these elusive errors.
INDEX TERMS
all-pairs testing, assurance, combinatorial testing, debugging, faults, pairwise testing
CITATION
Yu Lei, Rick Kuhn, "Practical Combinatorial Testing: Beyond Pairwise", IT Professional, vol.10, no. 3, pp. 19-23, May/June 2008, doi:10.1109/MITP.2008.54
REFERENCES
1. K. Burr and W. Young, "Combinatorial Test Techniques: Table-Based Automation, Test Generation, and Test Coverage," Proc. Int'l Conf. Software Testing, Analysis, and Review (STAR), 1998; http://aetgweb.argreenhouse.com/papers1998-star.pdf .
2. D.R. Kuhn, D.R. Wallace, and A. Gallo, "Software Fault Interactions and Implications for Software Testing," IEEE Trans. Software Eng., vol. 30, no. 6, 2004, pp. 418–421.
3. R. Bryce and C.J. Colbourn, "A Density-Based Greedy Algorithm for Higher Strength Covering Arrays," to be published in J. Software Testing, Verification, and Reliability; www.egr.unlv.edu/~rbryceresearch.htm.
4. R. Bryce and C.J. Colbourn, "The Density Algorithm for Pairwise Interaction Testing," J. Software Testing, Verification, and Reliability, vol. 17, no. 3, Aug. 2007, pp. 159–182.
5. M. Sutton, A. Greene, and P. Amini, Fuzzing: Brute Force Vulnerability Discovery, Addison-Wesley, 2007.
6. G.T. Leavens, A.L. Baker, and C. Ruby, "JML: A Notation for Detailed Design," H. Kilov, B. Rumpe, and I. Simmonds, eds., Behavioral Specifications of Businesses and Systems, Kluwer, 1999, pp. 175–188.
7. L. du Bousquet et al., "A Case Study in JML-Based Software Validation," Proc. 19th Int'l IEEE Conf. Automated Software Eng. (ASE 04), IEEE CS Press, 2004, pp. 294–297.
8. P. Ammann and P.E. Black, "Abstracting Formal Specifications to Generate Software Tests via Model Checking," Proc. 18th Digital Avionics Systems Conf., vol. 2., IEEE Press, 1999, pp. 10.A.6.1–10.
9. D.R. Kuhn and V. Okun, "Pseudo-Exhaustive Testing for Software," Proc. 30th NASA/IEEE Software Eng. Workshop, 2006, pp. 153–158; http://csrc.nist.gov/actsPID258305.pdf.
10. M.B. Cohen, J. Snyder, and G. Rothermel, "Testing Across Configurations: Implications for Combinatorial Testing," Proc. Workshop on Advances in Model-Based Software Testing, IEEE Press, 2006, pp. 1–9.
45 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool