This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage
March 2003 (vol. 29 no. 3)
pp. 195-209
Mary Jean Harrold, IEEE Computer Society

Abstract—Software testing is particularly expensive for developers of high-assurance software, such as software that is produced for commercial airborne systems. One reason for this expense is the Federal Aviation Administration's requirement that test suites be modified condition/decision coverage (MC/DC) adequate. Despite its cost, there is evidence that MC/DC is an effective verification technique and can help to uncover safety faults. As the software is modified and new test cases are added to the test suite, the test suite grows and the cost of regression testing increases. To address the test-suite size problem, researchers have investigated the use of test-suite reduction algorithms, which identify a reduced test suite that provides the same coverage of the software according to some criterion as the original test suite, and test-suite prioritization algorithms, which identify an ordering of the test cases in the test suite according to some criteria or goals. Existing test-suite reduction and prioritization techniques, however, may not be effective in reducing or prioritizing MC/DC-adequate test suites because they do not consider the complexity of the criterion. This paper presents new algorithms for test-suite reduction and prioritization that can be tailored effectively for use with MC/DC. The paper also presents the results of empirical studies of these algorithms.

[1] T.Y. Chen and M.F. Lau, “Dividing Strategies for the Optimization of a Test Suite,” Information Processing Letters, vol. 60, no. 3, pp. 135-141, Mar. 1996.
[2] Y. Chen, D. Rosenblum, and K. Vo, “TestTube: A System for Selective Regression Testing,” Proc. 16th Int'l Conf. Software Eng., pp. 211-222, May 1994.
[3] J.J. Chilenski and S.P. Miller, “Applicability of Modified Condition/Decision Coverage to Software Testing,” Software Eng. J., vol. 9, no. 5, pp. 193-200, 1994.
[4] S. Elbaum, A. Malishevsky, and G. Rothermel, “Prioritizing Test Cases for Regression Testing,” Proc. ACM Int'l Symp. Software Testing and Analysis, pp. 102-112, Aug. 2000.
[5] S. Elbaum, A. Malishevsky, and G. Rothermel, “Incorporating Varying Test Costs and Fault Severities into Test Case Prioritization,” Proc. Int'l Conf. Software Eng., pp. 329-338, May 2001.
[6] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness. New York: W.H. Freeman, 1979.
[7] M.J. Harrold, R. Gupta, and M.L. Soffa, “A Methodology for Controlling the Size of a Test Suite,” ACM Trans. Software Eng. and Methods, vol. 2, no. 3, pp. 270-285, July 1993.
[8] M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, “Experiments on the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria,” Proc. 16th Int'l. Conf. Software Eng., pp. 191-200, May 1994.
[9] J. Offutt, J. Pan, and J.M. Voas, “Procedures for Reducing the Size of Coverage-Based Test Sets,” Proc. 12th Int'l Conf. Testing Computer Software, pp. 111-123, June 1995.
[10] G. Rothermel and M.J. Harrold, “A Safe, Efficient Regression Test Selection Technique,” ACM Trans. Software Eng. and Methods, vol. 6, no. 2, pp. 173-210, Apr. 1997.
[11] G. Rothermel, M.J. Harrold, J. Ostrin, and C. Hong, “An Empirical Study of the Effects of Minimization on the Fault Detection Capabilities of Test Suites,” Proc. Int'l Conf. Software Maintenence, pp. 34-43, Nov. 1998.
[12] G. Rothermel, R. Untch, C. Chu, and M.J. Harrold, “Prioritizing Test Cases for Regression Testing,” IEEE Trans. Software Eng., vol. 27, no. 10, pp. 929-948, Oct. 2001.
[13] G. Rothermel, R. Untch, C. Chu, and M.J. Harrold, “Test Case Prioritization: An Empirical Study,” Proc. Int'l Conf. Software Maintenence, pp. 179-188, Sept. 1999.
[14] W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, “Effect of Test Set Minimization on Fault Detection Effectiveness,” Software Practice and Experience, vol. 28, no. 4, pp. 347-369, Apr. 1998.
[15] W.E. Wong, J.R. Horgan, A.P. Mathur, and A. Pasquini, “Test Set Size Minimization and Fault Detection Effectiveness: A Case Study in a Space Application,” Proc. 21st Ann. Int'l Comp. Software and Application Conf., pp. 522-528, Aug. 1997.

Index Terms:
Test-suite reduction, test-suite prioritization, modified condition/decision coverage, testing, critical software.
Citation:
James A. Jones, Mary Jean Harrold, "Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage," IEEE Transactions on Software Engineering, vol. 29, no. 3, pp. 195-209, March 2003, doi:10.1109/TSE.2003.1183927
Usage of this product signifies your acceptance of the Terms of Use.