The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - July/August (2009 vol.35)
pp: 452-469
Pavan Kumar Chittimalli , Tata Research Development & Design Centre, India
Mary Jean Harrold , Georgia Institute of Technology, Atlanta
ABSTRACT
This paper presents a technique that leverages an existing regression test selection algorithm to compute accurate, updated coverage data on a version of the software, P_{i+1}, without rerunning any test cases that do not execute the changes from the previous version of the software, P_i to P_{i+1}. The technique also reduces the cost of running those test cases that are selected by the regression test selection algorithm by performing a selective instrumentation that reduces the number of probes required to monitor the coverage data. Users of our technique can avoid the expense of rerunning the entire test suite on P_{i+1} or the inaccuracy produced by previous approaches that estimate coverage data for P_{i+1} or that reuse outdated coverage data from P_i. This paper also presents a tool, ReCover, that implements our technique, along with a set of empirical studies on a set of subjects that includes several industrial programs, versions, and test cases. The studies show the inaccuracies that can exist when an application—regression test selection—uses estimated or outdated coverage data. The studies also show that the overhead incurred by selective instrumentation used in our technique is negligible and overall our technique provides savings over earlier techniques.
INDEX TERMS
Regression testing, regression test selection, testing, maintenance.
CITATION
Pavan Kumar Chittimalli, Mary Jean Harrold, "Recomputing Coverage Information to Assist Regression Testing", IEEE Transactions on Software Engineering, vol.35, no. 4, pp. 452-469, July/August 2009, doi:10.1109/TSE.2009.4
REFERENCES
[1] C. Kaner, “Improving the Maintainability of Automated Test Suites,” Proc. Quality Week Conf., May 1997.
[2] B. Beizer, Software Testing Techniques. Van Nostrand Reinhold, 1990.
[3] H.K.N. Leung and L.J. White, “Insights into Regression Testing,” Proc. IEEE Conf. Software Maintenance, pp. 60-69, Oct. 1989.
[4] T. Ball, “On the Limit of Control Flow Analysis for Regression Test Selection,” Proc. ACM SIGSOFT Int'l Symp. Software Testing and Analysis, pp. 134-142, Mar. 1998.
[5] Y.F. Chen, D.S. Rosenblum, and K.P. Vo, “Testtube: A System for Selective Regression Testing,” Proc. 16th ACM/IEEE Int'l Conf. Software Eng., pp. 211-222, May 1994.
[6] A. Orso, N. Shi, and M.J. Harrold, “Scaling Regression Testing to Large Software Systems,” Proc. 12th ACM SIGSOFT Symp. Foundations of Software Eng., pp. 241-252, Nov. 2004.
[7] G. Rothermel and M.J. Harrold, “A Safe, Efficient Regression Test Selection Technique,” ACM Trans. Software Eng. and Methodology, vol. 6, no. 2, pp. 173-210, Apr. 1997.
[8] G. Rothermel, M.J. Harrold, and J. Dedhia, “Analyzing Regression Test Selection Techniques,” IEEE Trans. Software Eng., vol. 22, no. 8, pp. 529-551, Aug. 1996.
[9] F. Vokolos and P. Frankl, “Pythia: A Regression Test Selection Tool Based on Text Differencing,” Proc. IEEE Int'l Conf. Reliability, Quality and Safety of Software Intensive Systems, pp. 3-21, June 1997.
[10] M.J. Harrold, J.A. Jones, T. Li, D. Liang, A. Orso, M. Pennings, S. Sinha, S.A. Spoon, and A. Gujarathi, “Regression Test Selection for Java Software,” Proc. ACM Conf. Object-Oriented Programming, Systems, Languages, and Applications, pp. 312-326, Nov. 2001.
[11] S. Elbaum, D. Gable, and G. Rothermel, “The Impact of Software Evolution on Code Coverage Information,” Proc. IEEE Int'l Conf. Software Maintenance, pp. 170-179, Nov. 2001.
[12] T. Apiwattanapong, A. Orso, and M.J. Harrold, “A Differencing Algorithm for Object-Oriented Programs,” Proc. 19th IEEE Int'l Conf. Automated Software Eng., pp. 2-13, Sept. 2004.
[13] A. Srivastava and J. Thiagarajan, “Effectively Prioritizing Tests in Development Environment,” Proc. 2002 ACM SIGSOFT Int'l Symp. Software Testing and Analysis, pp. 97-106, July 2002.
[14] A. Chawla and A. Orso, “A Generic Instrumentation Framework for Collecting Dynamic Information,” Proc. ISSTA Workshop Empirical Research in Software Testing, July 2004.
[15] P.K. Chittimalli, M. Bapat, and R.D. Naik, “ProAX: A Program Analysis and Transformation Framework,” Proc. Theoretical Computer Science Technical Architect's Conf., Dec. 2004.
[16] Z. Wang, K. Pierce, and S. McFarling, “BMAT—A Binary Matching Tools for Stale Profile Propagation,” J. Instruction-Level Parallelism, vol. 2, pp. 1-20, Mar. 2000.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool