This Article 
 Bibliographic References 
 Add to: 
Leveraging User-Session Data to Support Web Application Testing
March 2005 (vol. 31 no. 3)
pp. 187-202
Marc Fisher II, IEEE Computer Society
Web applications are vital components of the global information infrastructure, and it is important to ensure their dependability. Many techniques and tools for validating Web applications have been created, but few of these have addressed the need to test Web application functionality and none have attempted to leverage data gathered in the operation of Web applications to assist with testing. In this paper, we present several techniques for using user session data gathered as users operate Web applications to help test those applications from a functional standpoint. We report results of an experiment comparing these new techniques to existing white-box techniques for creating test cases for Web applications, assessing both the adequacy of the generated test cases and their ability to detect faults on a point-of-sale Web application. Our results show that user session data can be used to produce test suites more effective overall than those produced by the white-box techniques considered; however, the faults detected by the two classes of techniques differ, suggesting that the techniques are complementary.

[1] Apache-Organization, Apache http server version 2.0 documentation, http://httpd.apache.orgdocs-2.0/, 2004.
[2] B. Beizer, Software Testing Techniques. New York: Van Nostrand Reinhold, 1990.
[3] M. Benedikt, J. Freire, and P. Godefroid, “VeriWeb: Automatically Testing Dynamic Web Sites,” Proc. 11th Int'l WWW Conf., May 2002.
[4] R. Binder, Testing Object-Oriented Systems. Addison Wesley, 2000.
[5] R. Carver and K. Tai, “Deterministic Execution Testing of Concurrent Ada Programs,” Proc. Conf. Tri-Ada, pp. 528-544, Jan. 1989.
[6] D. Chays, S. Dan, P. Frankl, F. Vokolos, and E. Weyuker, “A Framework for Testing Database Applications,” Proc. Int'l Symp. Software Testing and Analysis, pp. 147-157, Aug. 2000.
[7] J. Conallen, Building Web Applications with UML. Addison-Wesley, 2000.
[8] G. DiLucca, A. Fasolino, F. Faralli, and U. Carlini, “Testing Web Applications,” Proc. Int'l Conf. Software Maintenance, pp. 310-319, Nov. 2002.
[9] W. Dickinson, D. Leon, and A. Podgurski, “Finding Failures by Cluster Analysis of Execution Profiles,” Proc. Int'l Conf. Software Eng., pp. 339-348, May 2001.
[10] S. Elbaum, A.G. Malishevsky, and G. Rothermel, “Test Case Prioritization: A Family of Empirical Studies,” IEEE Trans. Software Eng., vol. 28, no. 2, pp. 159-182, Feb. 2002.
[11] Empirix, “Web Testing Solutions,” Testing+Solutions/, 2004.
[12] M. Harrold, R. Gupta, and M. Soffa, “A Methodology for Controlling the Size of a Test Suite,” ACM Trans. Software Eng. and Methodology, vol. 2, no. 3, pp. 270-285, July 1993.
[13] E. Hieatt and R. Mee, “Going Faster: Testing the Web Application,” IEEE Software, pp. 60-65, Mar. 2002.
[14] M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, “Experiments on the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria,” Proc. Int'l Conf. Software Eng., pp. 191-200, May 1994.
[15] Software Research, Inc., “eValid,” http://www.soft.comeValid/, 2004.
[16] E. Kirda, M. Jazayeri, C. Kerer, and M. Schranz, “Experiences in Engineering Flexible Web Services,” IEEE MultiMedia, vol. 8, no. 1, pp. 58-65, Jan. 2001.
[17] P. Koppol and K. Tai, “An Incremental Approach to Structural Testing of Concurrent Software,” Proc. Int'l Symp. Software Testing and Analysis, pp. 14-23, Jan. 1996.
[18] S. Lee and J. Offutt, “Generating Test Cases for XML-Based Web Component Interactions Using Mutation Analysis,” Proc. 12th IEEE Int'l Symp. Software Reliability Eng., pp. 200-209, Nov. 2001.
[19] T. Lee, “World Wide Web Consortium,” http:/, 2004.
[20] C. Liu, D. Kung, P. Hsia, and C. Hsu, “Structural Testing of Web Applications,” Proc. 11th IEEE Int'l Symp. Software Reliability Eng., pp. 84-96, Oct. 2000.
[21] “Developing ColdFusion MX Applications,” Macromedia, Inc., 2003, http:/
[22] S. Manley and M. Seltzer, “Web Facts and Fantasy,” Proc. 1997 Usenix Symp. Internet Technologies and Systems, 1997.
[23] A. Nikora and J. Munson, “Software Evolution and the Fault Process,” Proc. 23rd Ann. Software Eng. Workshop, 1998.
[24] Business Internet Group of San Francisco, “The BIG-SF Report on Government Web Application Integrity,” http://www.tealeaf. com/downloads/news/analyst_report BIG-SF_Report_Gov_ 2003-05.pdf , 2004.
[25] Business Internet Group of San Francisco, “The Black Friday Report on Web Application Integrity,” BIG-SF_BlackFridayReport. pdf , 2004.
[26] Parasoft, “WebKing,”, 2004.
[27] R. Pressman, Software Engineering A Practitioner's Approach. fifth ed. McGraw-Hill, 2001.
[28] Rational-Corporation, “Rational Testing Robot,” http://www., 2004.
[29] Testing a Website: Best practices, http:/, 2004.
[30] F. Ricca and P. Tonella, “Analysis and Testing of Web Applications,” Proc. Int'l Conf. Software Eng., pp. 25-34, May 2001.
[31] S. Elbaum, S. Karre, and G. Rothermel, “Improving Web Application Testing with User Session Data,” Proc. Int'l Conf. Software Eng., pp. 49-59, May 2003.
[32] K.C. Tai and R.H. Carver, “A Specification-Based Methodology for Testing Concurrent Programs,” Proc. Fifth European Software Eng. Conf., pp. 154-172, Sept. 1995.
[33] R. Taylor, D. Levine, and C. Kelly, “Structural Testing of Concurrent Programs,” IEEE Trans. Software Eng., vol. 18, no. 3, pp. 206-215, 1992.
[34] S. Tilley and H. Shihong, “Evaluating the Reverse Engineering Capabilities of Web Tools for Understanding Site Content and Structure: A Case Study,” Proc. Int'l Conf. Software Eng., pp. 514-523, May 2001.
[35] J. Tzay, J. Huang, F. Wang, and W.C. Chu, “Constructing an Object-Oriented Architecture for Web Application Testing,” J. Information Science and Eng., vol. 18, no. 1, pp. 59-84, Jan. 2002.
[36] S. Weiss, “A Formal Framework for Studying Concurrent Program Testing,” Proc. Fourth Symp. Software Testing, Analysis, and Verification, pp. 106-113, July 1988.
[37] E.J. Weyuker, “On Testing Nontestable Programs,” The Computing J., vol. 15, no. 4, pp. 465-470, 1982.
[38] E.J. Weyuker and B. Jeng, “Analyzing Partition Testing Strategies,” IEEE Trans. Software Eng., vol. 17, no. 7, pp. 703-711, July 1991.
[39] W. Wong, J. Horgan, S. London, and A. Mathur, “Effect of Test Set Minimization on Fault Detection Effectiveness,” Proc. 17th Int'l Conf. Software Eng., pp. 41-50, Apr. 1995.
[40] R. Yang and C. Chung, “Path Analysis Testing of Concurrent Programs,” Information and Software Technology, vol. 34, no. 1, pp. 43-56, 1992.

Index Terms:
Software testing, test data generation, Web applications, empirical studies.
Sebastian Elbaum, Gregg Rothermel, Srikanth Karre, Marc Fisher II, "Leveraging User-Session Data to Support Web Application Testing," IEEE Transactions on Software Engineering, vol. 31, no. 3, pp. 187-202, March 2005, doi:10.1109/TSE.2005.36
Usage of this product signifies your acceptance of the Terms of Use.