This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Analyzing Regression Test Selection Techniques
August 1996 (vol. 22 no. 8)
pp. 529-551

Abstract—Regression testing is a necessary but expensive maintenance activity aimed at showing that code has not been adversely affected by changes. Regression test selection techniques reuse tests from an existing test suite to test a modified program. Many regression test selection techniques have been proposed; however, it is difficult to compare and evaluate these techniques because they have different goals. This paper outlines the issues relevant to regression test selection techniques, and uses these issues as the basis for a framework within which to evaluate the techniques. We illustrate the application of our framework by using it to evaluate existing regression test selection techniques. The evaluation reveals the strengths and weaknesses of existing techniques, and highlights some problems that future work in this area should address.

[1] H. Agrawal, J. Horgan, E. Krauser, and S. London, “Incremental Regression Testing,” Proc. Conf. Software Maintenance—1993, pp. 348-357, Sept. 1993.
[2] S. Bates and S. Horwitz, “Incremental Program Testing Using Program Dependence Graphs,” Proc. 20th ACM Symp. Principles of Programming Languages, Jan. 1993.
[3] P. Benedusi, A. Cimitile, and U. De Carlini, “Post-Maintenance Testing Based on Path Change Analysis,” Proc. Conf. Software Maintenance—1988, pp. 352-361, Oct. 1988.
[4] D. Binkley, "Using Semantic Differencing to Reduce the Cost of Regression Testing," Proc. Conf. Software Maintenance—1992, pp. 41-50, Nov. 1992.
[5] D. Binkley, "Reducing the cost of Regression Testing by Semantics Guided Test Case Selection," Proc. Conf. Software Maintenance—1995, pp. 251-260, Oct. 1995.
[6] P.A. Brown and D. Hoffman, "The Application of Module Regression Testing at TRIUMF," Nuclear Instruments and Methods in Physics Research, Section A, vol. .A293, nos. 1-2, pp. 377-381, Aug. 1990.
[7] Y.F. Chen, D.S. Rosenblum, and K.P. Vo, “TestTube: A System for Selective Regression Testing,” Proc. 16th Int'l Conf. Software Eng., pp. 211-222, May 1994.
[8] H. Crowder, E.L. Johnson, and M. Padberg, "Solving Large-Scale Zero-One Linear Programming Problems," Operations Research, vol. 31, no. 5, pp. 803-834, Sept. 1983.
[9] T. Dogsa and I. Rozman, "CAMOTE—Computer Aided Module Testing and Design Environment," Proc. Conf. Software Maintenance—1988, pp. 404-408, Oct. 1988.
[10] K.F. Fischer, "A Test Case Selection Method for the Validation of Software Maintenance Modifications," Proc. COMPSAC '77, pp. 421-426, Nov. 1977.
[11] K.F. Fischer, F. Raji, and A. Chruscicki, "A Methodology for Retesting Modified Software," Proc. Nat'l Telecommunications Conf. B-6-3, pp. 1-6, Nov. 1981.
[12] M.R. Garey and D.S. Johnson, Computers and Intractability: A Guide to the Theory of NP-Completeness.New York: W.H. Freeman, 1979.
[13] R. Gupta, M.J. Harrold, and M.L. Soffa, “An Approach to Regression Testing Using Slicing,” Proc. Conf. Software Maintenance—1992, pp. 299-308, Nov. 1992.
[14] M.J. Harrold, R. Gupta, and M.L. Soffa, "A Methodology for Controlling the Size of a Test Suite," ACM Trans. Software Eng. and Methodology, vol. 2, no. 3, pp. 270-285, July 1993.
[15] M.J. Harrold and M.L. Soffa, “An Incremental Approach to Unit Testing During Maintenance,” Proc. Conf. Software Maintenance—1988, pp. 362-367, Oct. 1988.
[16] M.J. Harrold and M.L. Soffa, "An Incremental Data Flow Testing Tool," Proc. Sixth Int'l Conf. Testing Computer Software, May 1989.
[17] M.J. Harrold and M.L. Soffa, “Interprocedural Data Flow Testing,” Proc. ACM SIGSOFT '89 Third Symp. Software Testing, Analysis, and Verification, pp. 158–167, Dec. 1989.
[18] J. Hartman and D.J. Robson, “Revalidation During the Software Maintenance Phase,” Proc. Conf. Software Maintenance, IEEE CS Press, Los Alamitos, Calif., 1989, pp. 70-79.
[19] J. Hartmann and D.J. Robson, "RETEST—Development of a Selective Revalidation Prototype Environment for Use in Software Maintenance," Proc. 23rd Hawaii Int'l Conf. System Sciences, pp. 92-101, Jan. 1990.
[20] J. Hartmann and D.J. Robson, “Techniques for Selective Revalidation,” IEEE Software, vol. 16, no. 1, pp. 31-38, Jan. 1990.
[21] D. Hoffman, "A CASE Study in Module Testing," Proc. Conf. Software Maintenance—1989, pp. 100-105, Oct. 1989.
[22] D. Hoffman and C. Brealey, “Module Test Case Generation,” Proc. Third Symp. Software Testing, Analysis, and Verification, pp. 97-102, 1989.
[23] S. Horwitz, T. Reps, and D. Binkley, “Interprocedural Slicing Using Dependence Graphs,” ACM Trans. Programming Languages and Systems. vol. 12, no. 1, pp. 26-60, Jan. 1990.
[24] J. Laski and W. Szermer, “Identification of Program Modifications and Its Applications in Software Maintenance,” Proc. Conf. Software Maintenance—1992, pp. 282-290, Nov. 1992.
[25] J.A.N. Lee and X. He, “A Methodology for Test Selection,” The J. Systems and Software, vol. 13, no. 1, pp. 177-185, Sept. 1990.
[26] H.K.N. Leung and L.J. White, “Insights Into Regression Testing,” Proc. Conf. Software Maintenance—1989, pp. 60-69, Oct. 1989.
[27] H.K.N. Leung and L.J. White, "Insights into Testing and Regression Testing Global Variables," J. Software Maintenance, vol. 2, pp. 209-222, Dec. 1990.
[28] H.K.N. Leung and L.J. White, “A Study of Integration Testing and Software Regression at the Integration Level,” Proc. Conf. Software Maintenance—1990, pp. 290-300, Nov. 1990.
[29] H.K.N. Leung and L.J. White, “A Cost Model to Compare Regression Test Strategies,” Proc. Conf. Software Maintenance—1991, pp. 201-208, Oct. 1991.
[30] R. Lewis, D.W. Beck, and J. Hartmann, "Assay—A Tool to Support Regression Testing," Proc. ESEC '89, Second European Software Eng. Conf., pp. 487-496, Sept. 1989.
[31] T.J. Ostrand and E.J. Weyuker, "Using Dataflow Analysis for Regression Testing," Proc. Sixth Ann. Pacific Northwest Software Quality Conf., pp. 233-247, Sept. 1988.
[32] G. Rothermel, "Efficient, Effective Regression Testing Using Safe Test Selection Techniques," PhD dissertation, Clemson Univ., May 1996.
[33] G. Rothermel and M.J. Harrold, "A Safe, Efficient Algorithm for Regression Test Selection," Proc. Conf. Software Maintenance—1993, pp. 358-367, Sept. 1993.
[34] G. Rothermel and M.J. Harrold, ”Selecting Regression Tests for Object-Oriented Software,” Proc. Int'l Conf. Software Maintenance, pp. 14–25, Aug. 1994.
[35] G. Rothermel and M.J. Harrold, "Selecting Tests and Identifying Test Coverage Requirements for Modified Software," Proc. 1994 Int'l Symp. Software Testing and Analysis, pp. 169-184, Aug. 1994.
[36] S. Schach, Software Engineering.Boston: Aksen Assoc., 1992.
[37] B. Sherlund and B. Korel, "Modification Oriented Software Testing," Conf. Proc.: Quality Week 1991, pp. 1-17, 1991.
[38] B. Sherlund and B. Korel, "Logical Modification Oriented Software Testing," Proc. 12th Int'l Conf. Testing Computer Software, June 1995.
[39] A.B. Taha, S.M. Thebaut, and S.S. Liu, “An Approach to Software Fault Localization and Revalidation Based on Incremental Data Flow Analysis,” Proc. 13th Ann. Int'l Computer Software and Applications Conf., pp. 527-534, Sept. 1989.
[40] A. von Mayrhauser, R.T. Mraz, and J. Walls, "Domain Based Regression Testing," Proc. Conf. Software Maintenance—1994, pp. 26-35, Sept. 1994.
[41] L.J. White and H.K.N. Leung, "A Firewall Concept for Both Control-Flow and Data-Flow in Regression Integration Testing," Proc. Conf. Software Maintenance—1992, pp. 262-270, Nov. 1992.
[42] L.J. White, V. Narayanswamy, T. Friedman, M. Kirschenbaum, P. Piwowarski, and M. Oha, "Test Manager: A Regression Testing Tool," Proc. Conf. Software Maintenance—1993, pp. 338-347, Sept. 1993.
[43] W.E. Wong, J.R. Horgan, S. London, and A.P. Mathur, "Effect of Test Set Minimization on Fault Detection Effectiveness," Proc. 17th Int'l Conf. Software Eng., pp. 41-50, IEEE, 1995.
[44] W. Yang, "Identifying Syntactic Differences Between Two Programs," Software—Practice and Experience, vol. 21, no. 7, pp. 739-755, July 1991.
[45] S.S. Yau and Z. Kishimoto, "A Method for Revalidating Modified Programs in the Maintenance Phase," Proc. COMPSAC '87: 11th Ann. Int'l Computer Software and Applications Conf., pp. 272-277, Oct. 1987.
[46] J. Ziegler, J.M. Grasso, and L.G. Burgermeister, "An Ada Based Real-Time Closed-Loop Integration and Regression Test Tool," Proc. Conf. Software Maintenance—1989, pp. 81-90, Oct. 1989.

Index Terms:
Software maintenance, regression testing, selective retest, regression test selection.
Citation:
Gregg Rothermel, Mary Jean Harrold, "Analyzing Regression Test Selection Techniques," IEEE Transactions on Software Engineering, vol. 22, no. 8, pp. 529-551, Aug. 1996, doi:10.1109/32.536955
Usage of this product signifies your acceptance of the Terms of Use.