This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Developing a Single Model and Test Prioritization Strategies for Event-Driven Software
January/February 2011 (vol. 37 no. 1)
pp. 48-64
Renée C. Bryce, Utah State University, Logan
Sreedevi Sampath, University of Maryland, Baltimore
Atif M. Memon, University of Maryland, College Park
Event-Driven Software (EDS) can change state based on incoming events; common examples are GUI and Web applications. These EDSs pose a challenge to testing because there are a large number of possible event sequences that users can invoke through a user interface. While valuable contributions have been made for testing these two subclasses of EDS, such efforts have been disjoint. This work provides the first single model that is generic enough to study GUI and Web applications together. In this paper, we use the model to define generic prioritization criteria that are applicable to both GUI and Web applications. Our ultimate goal is to evolve the model and use it to develop a unified theory of how all EDS should be tested. An empirical study reveals that the GUI and Web-based applications, when recast using the new model, show similar behavior. For example, a criterion that gives priority to all pairs of event interactions did well for GUI and Web applications; another criterion that gives priority to the smallest number of parameter value settings did poorly for both. These results reinforce our belief that these two subclasses of applications should be modeled and studied together.

[1] A.M. Memon and Q. Xie, "Studying the Fault-Detection Effectiveness of GUI Test Cases for Rapidly Evolving Software," IEEE Trans. Software Eng., vol. 31, no. 10, pp. 884-896, Oct. 2005.
[2] A. Andrews, J. Offutt, and R. Alexander, "Testing Web Applications by Modeling with FSMs," Software and Systems Modeling, vol. 4, no. 3, pp. 326-345, July 2005.
[3] G.D. Lucca, A. Fasolino, F. Faralli, and U.D. Carlini, "Testing Web Applications," Proc. IEEE Int'l Conf. Software Maintenance, pp. 310-319, Oct. 2002.
[4] F. Ricca and P. Tonella, "Analysis and Testing of Web Applications," Proc. Int'l Conf. Software Eng., pp. 25-34, May. 2001.
[5] R.C. Bryce and A.M. Memon, "Test Suite Prioritization by Interaction Coverage," Proc. Workshop Domain-Specific Approaches to Software Test Automation in Conjunction with Sixth Joint Meeting of the European Software Eng. Conf. and ACM SIGSOFT Symp. Foundations of Software Eng., pp. 1-7, Sept. 2007.
[6] S. Sampath, R. Bryce, G. Viswanath, V. Kandimalla, and A.G. Koru, "Prioritizing User-Session-Based Test Cases for Web Application Testing," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation, pp. 141-150, Apr. 2008.
[7] P. Brooks, B. Robinson, and A.M. Memon, "An Initial Characterization of Industrial Graphical User Interface Systems," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation, pp. 11-20, 2009.
[8] L. White, "Regression Testing of GUI Event Interactions," Proc. Int'l Conf. Software Maintenance, pp. 350-358, Nov. 1996.
[9] "Web Site Test Tools and Site Management Tools," http://www.softwareqatest.comqatweb1.html , Apr. 2009.
[10] D.C. Kung, C.-H. Liu, and P. Hsia, "An Object-Oriented Web Test Model for Testing Web Applications," Proc. First Asia-Pacific Conf. Quality Software, pp. 111-120, Oct. 2000.
[11] W. Wang, S. Sampath, Y. Lei, and R. Kacker, "An Interaction-Based Test Sequence Generation Approach for Testing Web Applications," Proc. IEEE Int'l Conf. High Assurance Systems Eng., pp. 209-218, 2008.
[12] W. Halfond and A. Orso, "Improving Test Case Generation for Web Applications Using Automated Interface Discovery," Proc. Sixth Joint Meeting of the European Software Eng. Conf. and ACM SIGSOFT Symp. Foundations of Software Eng., pp. 145-154, Sept. 2007.
[13] S. Artzi, A. Kiezun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M.D. Ernst, "Finding Bugs in Dynamic Web Applications," Proc. Int'l Symp. Software Testing and Analysis, pp. 261-272, July 2008.
[14] N. Alshahwan and M. Harman, "Automated Session Data Repair for Web Application Regression Testing," Proc. IEEE Int'l Conf. Software Testing, Verification, and Validation, pp. 298-307, Apr. 2008.
[15] S. Elbaum, G. Rothermel, S. Karre, and M. FisherII, "Leveraging User Session Data to Support Web Application Testing," IEEE Trans. Software Eng., vol. 31, no. 3, pp. 187-202, May 2005.
[16] S. Sampath, S. Sprenkle, E. Gibson, L. Pollock, and A.S. Greenwald, "Applying Concept Analysis to User-Session-Based Testing of Web Applications," IEEE Trans. Software Eng., vol. 33, no. 10, pp. 643-658, Oct. 2007.
[17] K. Onoma, W.-T. Tsai, M. Poonawala, and H. Suganuma, "Regression Testing in an Industrial Environment," Comm. ACM, vol. 41, no. 5, pp. 81-86, May 1988.
[18] G. Rothermel, R.H. Untch, C. Chu, and M.J. Harrold, "Prioritizing Test Cases for Regression Testing," IEEE Trans. Software Eng., vol. 27, no. 10, pp. 929-948, Oct. 2001.
[19] S. Elbaum, A.G. Malishevsky, and G. Rothermel, "Test Case Prioritization: A Family of Empirical Studies," IEEE Trans. Software Eng., vol. 28, no. 2, pp. 159-182, Feb. 2002.
[20] D. Binkley, "Using Semantic Differencing to Reduce the Cost of Regression Testing," Proc. Int'l Conf. Software Maintenance, pp. 41-50, Nov. 1992.
[21] J.A. Jones and M.J. Harrold, "Test-Suite Reduction and Prioritization for Modified Condition/Decision Coverage," IEEE Trans. Software Eng., vol. 29, no. 3, pp. 195-209, Mar. 2003.
[22] D. Jeffrey and N. Gupta, "Test Case Prioritization Using Relevant Slices," Proc. Int'l Computer Software and Applications Conf., pp. 411-418, Sept. 2006.
[23] J. Lee and X. He, "A Methodology for Test Selection," J. Systems and Software, vol. 13, no. 3, pp. 177-185, Nov. 1990.
[24] J. Offutt, J. Pan, and J.M. Voas, "Procedures for Reducing the Size of Coverage-Based Test Sets," Proc. Int'l Conf. Testing Computer Software, pp. 111-123, June 1995.
[25] S. Sprenkle, L. Pollock, H. Esquivel, B. Hazelwood, and S. Ecott, "Automated Oracle Comparators for Testing Web Applications," Proc. Int'l Symp. Software Reliability Eng., pp. 253-262, Nov. 2007.
[26] M. Grindal, J. Offutt, and S. Andler, "Combination Testing Strategies: A Survey," Software Testing, Verification and Reliability, vol. 15, pp. 167-199, Mar. 2005.
[27] D.R. Kuhn, D.R. Wallace, and A.M. Gallo, "Software Fault Interactions and Implications for Software Testing," IEEE Trans. Software Eng., vol. 30, no. 6, pp. 418-421, Oct. 2004.
[28] C.J. Colbourn, "Combinatorial Aspects of Covering Arrays," Le Matematiche, vol. 58, (Catania), pp. 121-167, 2004.

Index Terms:
Combinatorial interaction testing, covering arrays, event-driven software (EDS), t-way interaction coverage, test suite prioritization, user-session testing, Web application testing, GUI testing.
Citation:
Renée C. Bryce, Sreedevi Sampath, Atif M. Memon, "Developing a Single Model and Test Prioritization Strategies for Event-Driven Software," IEEE Transactions on Software Engineering, vol. 37, no. 1, pp. 48-64, Jan.-Feb. 2011, doi:10.1109/TSE.2010.12
Usage of this product signifies your acceptance of the Terms of Use.