|
| This Article | ||
| ||
| Share | ||
| Bibliographic References | ||
| Add to: | ||
| | ||
| Search | ||
| ||
An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software
June 2013 (vol. 39 no. 6)
pp. 787-805
| ASCII Text | x | ||
| Richard Baker, Ibrahim Habli, "An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software," IEEE Transactions on Software Engineering, vol. 39, no. 6, pp. 787-805, June, 2013. | |||
| BibTex | x | ||
| @article{ 10.1109/TSE.2012.56, author = {Richard Baker and Ibrahim Habli}, title = {An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software}, journal ={IEEE Transactions on Software Engineering}, volume = {39}, number = {6}, issn = {0098-5589}, year = {2013}, pages = {787-805}, doi = {http://doi.ieeecomputersociety.org/10.1109/TSE.2012.56}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, } | |||
| RefWorks Procite/RefMan/Endnote | x | ||
| TY - JOUR JO - IEEE Transactions on Software Engineering TI - An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software IS - 6 SN - 0098-5589 SP787 EP805 EPD - 787-805 A1 - Richard Baker, A1 - Ibrahim Habli, PY - 2013 KW - Testing KW - Certification KW - Software systems KW - Safety KW - Industries KW - Guidelines KW - certification KW - Mutation KW - safety-critical software KW - verification KW - testing VL - 39 JA - IEEE Transactions on Software Engineering ER - | |||
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TSE.2012.56
Testing provides a primary means for assuring software in safety-critical systems. To demonstrate, particularly to a certification authority, that sufficient testing has been performed, it is necessary to achieve the test coverage levels recommended or mandated by safety standards and industry guidelines. Mutation testing provides an alternative or complementary method of measuring test sufficiency, but has not been widely adopted in the safety-critical industry. In this study, we provide an empirical evaluation of the application of mutation testing to airborne software systems which have already satisfied the coverage requirements for certification. Specifically, we apply mutation testing to safety-critical software developed using high-integrity subsets of C and Ada, identify the most effective mutant types, and analyze the root causes of failures in test cases. Our findings show how mutation testing could be effective where traditional structural coverage analysis and manual peer review have failed. They also show that several testing issues have origins beyond the test activity, and this suggests improvements to the requirements definition and coding process. Our study also examines the relationship between program characteristics and mutation survival and considers how program size can provide a means for targeting test areas most likely to have dormant faults. Industry feedback is also provided, particularly on how mutation testing can be integrated into a typical verification life cycle of airborne software.
Index Terms:
Testing,Certification,Software systems,Safety,Industries,Guidelines,certification,Mutation,safety-critical software,verification,testing
Citation:
Richard Baker, Ibrahim Habli, "An Empirical Evaluation of Mutation Testing for Improving the Test Quality of Safety-Critical Software," IEEE Transactions on Software Engineering, vol. 39, no. 6, pp. 787-805, June 2013, doi:10.1109/TSE.2012.56
Usage of this product signifies your acceptance of the Terms of Use.

