This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops
On Use of Coverage Metrics in Assessing Effectiveness of Combinatorial Test Designs
Luxembourg, Luxembourg Luxembourg
March 18-March 22
ISBN: 978-1-4799-1324-4
Combinatorial test suite design is a test generation technique, popular in part due to its ability to achieve coverage and defect finding power approximating that of exhaustive testing while keeping test suite sizes constrained. In recent years, there have been numerous advances in combinatorial test design techniques, in terms of efficiency and usability of methods used to create them as well as in understanding of their benefits and limitations when applied to real world software. Numerous case studies have appeared presenting practical applications of the combinatorial test suite design techniques, often comparing them with manually-created, random, or exhaustive suites. These comparisons are done either in terms of defects found or by applying some code coverage metric. Since many different and valid combinatorial test suites of strength t can be created for a given test domain, the question whether they all have the same coverage properties is a pertinent one.In this paper we explore the stability of size and coverage of combinatorial test suites. We find that in general coverage levels increase and coverage variability decreases with increasing order of combinations t; however we also find exceptions with implications for practitioners. In addition, we explore cases where coverage achieved by combinatorial test suites of order t applied to the same program is not different from test suites of order t-1. Lastly, we discuss these findings in context of the ongoing practice of applying code coverage metrics to measure effectiveness of combinatorial test suites.
Index Terms:
Testing,Measurement,Standards,Software,Histograms,Market research,Conferences,statement and branch coverage,Pairwise testing,combinatorial testing,test case generation,test case design,coverage metrics
Citation:
Jacek Czerwonka, "On Use of Coverage Metrics in Assessing Effectiveness of Combinatorial Test Designs," icstw, pp.257-266, 2013 IEEE Sixth International Conference on Software Testing, Verification and Validation Workshops, 2013
Usage of this product signifies your acceptance of the Terms of Use.