This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Fault Localization for Dynamic Web Applications
March-April 2012 (vol. 38 no. 2)
pp. 314-335
ASCII Text
x 
 
Shay Artzi, Julian Dolby, Frank Tip, Marco Pistoia, "Fault Localization for Dynamic Web Applications," IEEE Transactions on Software Engineering, vol. 38, no. 2, pp. 314-335, March-April, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TSE.2011.76,
author = {Shay Artzi and Julian Dolby and Frank Tip and Marco Pistoia},
title = {Fault Localization for Dynamic Web Applications},
journal ={IEEE Transactions on Software Engineering},
volume = {38},
number = {2},
issn = {0098-5589},
year = {2012},
pages = {314-335},
doi = {http://doi.ieeecomputersociety.org/10.1109/TSE.2011.76},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Software Engineering
TI - Fault Localization for Dynamic Web Applications
IS - 2
SN - 0098-5589
SP314
EP335
EPD - 314-335
A1 - Shay Artzi,
A1 - Julian Dolby,
A1 - Frank Tip,
A1 - Marco Pistoia,
PY - 2012
KW - Fault localization
KW - statistical debugging
KW - program analysis
KW - web applications
KW - PHP.
VL - 38
JA - IEEE Transactions on Software Engineering
ER -
 
Shay Artzi, IBM Software Group, Littleton
Julian Dolby, IBM Thomas J. Watson Research Center, Yorktown Heights
Frank Tip, IBM Thomas J. Watson Research Center, Yorktown Heights
Marco Pistoia, IBM Thomas J. Watson Research Center, Yorktown Heights
In recent years, there has been significant interest in fault-localization techniques that are based on statistical analysis of program constructs executed by passing and failing executions. This paper shows how the Tarantula, Ochiai, and Jaccard fault-localization algorithms can be enhanced to localize faults effectively in web applications written in PHP by using an extended domain for conditional and function-call statements and by using a source mapping. We also propose several novel test-generation strategies that are geared toward producing test suites that have maximal fault-localization effectiveness. We implemented various fault-localization techniques and test-generation strategies in Apollo, and evaluated them on several open-source PHP applications. Our results indicate that a variant of the Ochiai algorithm that includes all our enhancements localizes 87.8 percent of all faults to within 1 percent of all executed statements, compared to only 37.4 percent for the unenhanced Ochiai algorithm. We also found that all the test-generation strategies that we considered are capable of generating test suites with maximal fault-localization effectiveness when given an infinite time budget for test generation. However, on average, a directed strategy based on path-constraint similarity achieves this maximal effectiveness after generating only 6.5 tests, compared to 46.8 tests for an undirected test-generation strategy.

[1] R. Abreu, P. Zoeteweij, and A.J.C. van Gemund, "An Evaluation of Similarity Coefficients for Software Fault Localization," Proc. 12th Pacific Rim Int'l Symp. Dependable Computing, pp. 39-46, 2006.
[2] R. Abreu, P. Zoeteweij, and A.J. van Gemund, "On the Accuracy of Spectrum-Based Fault Localization," Proc. Testing: Academic and Industry Conf. Practice and Research Techniques, pp. 89-98, Sept. 2007.
[3] R. Abreu, P. Zoeteweij, and A.J.C. van Gemund, "An Evaluation of Similarity Coefficients for Software Fault Localization," Proc. 12th Pacific Rim Int'l Symp. Dependable Computing, pp. 39-46, 2006.
[4] H. Agrawal, J.R. Horgan, S. London, and W.E. Wong, "Fault Localization Using Execution Slices and Dataflow Tests," Proc. Int'l Symp. Software Reliability Eng., pp. 143-151, 1995.
[5] S. Artzi, J. Dolby, S. Jensen, A. Møller, and F. Tip, "A Framework for Automated Testing of Javascript Web Applications," Proc. Int'l Conf. Software Eng., 2011.
[6] S. Artzi, J. Dolby, F. Tip, and M. Pistoia, "Directed Test Generation for Effective Fault Localization," Proc. 19th Int'l Symp. Software Testing and Analysis, pp. 49-60, 2010.
[7] S. Artzi, J. Dolby, F. Tip, and M. Pistoia, "Practical Fault Localization for Dynamic Web Applications," Proc. 32nd ACM/IEEE Int'l Conf. Software Eng., vol. 1, pp. 265-274, 2010.
[8] S. Artzi, A. Kieżun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M.D. Ernst, "Finding Bugs in Dynamic Web Applications," Proc. Int'l Symp. Software Testing and Analysis, pp. 261-272, 2008.
[9] S. Artzi, A. Kieżun, J. Dolby, F. Tip, D. Dig, A. Paradkar, and M.D. Ernst, "Finding Bugs in Web Applications Using Dynamic Test Generation and Explicit State Model Checking," IEEE Trans. Software Eng., vol. 36, no. 4 pp. 474-494, July/Aug. 2010.
[10] P. Arumuga Nainar, T. Chen, J. Rosin, and B. Liblit, "Statistical Debugging Using Compound Boolean Predicates," Proc. Int'l Symp. Software Testing and Analysis, S. Elbaum, ed., July 2007.
[11] P. Arumuga Nainar and B. Liblit, "Adaptive Bug Isolation," Proc. 32nd ACM/IEEE Int'l Conf. Software Eng., pp. 255-264, 2010.
[12] G.K. Baah, A. Podgurski, and M.J. Harrold, "Causal Inference for Statistical Fault Localization," Proc. 19th Int'l Symp. Software Testing and Analysis, pp. 73-84, 2010.
[13] B. Baudry, F. Fleurey, and Y. Le Traon, "Improving Test Suites for Efficient Fault Localization," Proc. 28th Int'l Conf. Software Eng., L.J. Osterweil, H.D. Rombach, and M.L. Soffa, eds., pp. 82-91, 2006.
[14] J. Burnim and K. Sen, "Heuristics for Scalable Dynamic Test Generation," Proc. IEEE/ACM Int'l Conf. Automated Software Eng., pp. 443-446, 2008.
[15] C. Cadar, V. Ganesh, P.M. Pawlowski, D.L. Dill, and D.R. Engler, "EXE: Automatically Generating Inputs of Death," Proc. Conf. Computer and Comm. Security, 2006.
[16] M.Y. Chen, E. Kiciman, E. Fratkin, A. Fox, and E. Brewer, "Pinpoint: Problem Determination in Large, Dynamic Internet Services," Proc. Int'l Conf. Dependable Systems and Networks, pp. 595-604, 2002.
[17] T.M. Chilimbi, B. Liblit, K.K. Mehra, A.V. Nori, and K. Vaswani, "Holmes: Effective Statistical Debugging via Efficient Path Profiling," Proc. 31st Int'l Conf. Software Eng., pp. 34-44, May 2009.
[18] H. Cleve and A. Zeller, "Locating Causes of Program Failures," Proc. Int'l Conf. Software Eng., pp. 342-351, May 2005.
[19] V. Dallmeier, C. Lindig, and A. Zeller, "Lightweight Defect Localization for Java," Proc. European Conf. Object-Oriented Programming, pp. 528-550, 2005.
[20] P. Godefroid, N. Klarlund, and K. Sen, "DART: Directed Automated Random Testing," Proc. ACM SIGPLAN Conf. Programming Language Design and Implementation, 2005.
[21] P. Godefroid, M.Y. Levin, and D. Molnar, "Automated Whitebox Fuzz Testing," Proc. Symp. Network and Distributed System Security, 2008.
[22] S. Horwitz, T. Reps, and D. Binkley, "Interprocedural Slicing Using Dependence Graphs," ACM Trans. Programming Languages and Systems, vol. 12, no. 1 pp. 26-60, 1990.
[23] M. Hutchins, H. Foster, T. Goradia, and T. Ostrand, "Experiments of the Effectiveness of Dataflow- and Controlflow-Based Test Adequacy Criteria," Proc. Int'l Conf. Software Eng., pp. 191-200, 1994.
[24] A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Prentice-Hall, Inc., 1988.
[25] D. Jeffrey, N. Gupta, and R. Gupta, "Fault Localization Using Value Replacement," Proc. ACM/SIGSOFT Int'l Symp. Software Testing and Analysis, B.G. Ryder and A. Zeller, eds., pp. 167-178, 2008.
[26] B. Jiang, Z. Zhang, T. Tse, and T.Y. Chen, "How Well Do Test Case Prioritization Techniques Support Statistical Fault Localization," Proc. 33rd Ann. IEEE Int'l Computer Software and Applications Conf., July 2009.
[27] G. Jin, A. Thakur, B. Liblit, and S. Lu, "Instrumentation and Sampling Strategies for Cooperative Concurrency Bug Isolation," Proc. 25th Ann. ACM SIGPLAN Conf. Object-Oriented Programming, Systems, Languages, and Applications, M. Rinard, ed., Oct. 2010.
[28] J.A. Jones and M.J. Harrold, "Empirical Evaluation of the Tarantula Automatic Fault-Localization Technique," Proc. IEEE/ACM Int'l Conf. Automated Software Eng., pp. 273-282, 2005.
[29] J.A. Jones, M.J. Harrold, and J. Stasko, "Visualization of Test Information to Assist Fault Localization," Proc. Int'l Conf. Software Eng., pp. 467-477, 2002.
[30] B. Liblit, A. Aiken, A.X. Zheng, and M.I. Jordan, "Bug Isolation via Remote Program Sampling," Proc. Conf. Programming Language Design and Implementation, pp. 141-154, 2003.
[31] B. Liblit, M. Naik, A.X. Zheng, A. Aiken, and M.I. Jordan, "Scalable Statistical Bug Isolation," Proc. Conf. Programming Language Design and Implementation, pp. 15-26, 2005.
[32] C. Liu, X. Yan, L. Fei, J. Han, and S.P. Midkiff, "Sober: Statistical Model-Based Bug Localization," Proc. European Software Eng. Conf. Held Jointly with 13th ACM SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 286-295, 2005.
[33] J. Lyle and M. Weiser, "Automatic Bug Location by Program Slicing," Proc. Second Int'l Conf. Computers and Applications, pp. 877-883, 1987.
[34] Y. Minamide, "Static Approximation of Dynamically Generated Web Pages," Proc. Int'l Conf. World Wide Web, 2005.
[35] H. Pan and E.H. Spafford, "Heuristics for Automatic Localization of Software Faults," Technical Report SERC-TR-116-P, Purdue Univ., July 1992.
[36] S. Park, R.W. Vuduc, and M.J. Harrold, "Falcon: Fault Localization in Concurrent Programs," Proc. 32nd ACM/IEEE Int'l Conf. Software Eng., pp. 245-254, 2010.
[37] X. Ren and B.G. Ryder, "Heuristic Ranking of Java Program Edits for Fault Localization," Proc. ACM/SIGSOFT Int'l Symp. Software Testing and Analysis, D.S. Rosenblum and S.G. Elbaum, eds., pp. 239-249, 2007.
[38] M. Renieris and S.P. Reiss, "Fault Localization with Nearest Neighbor Queries," Proc. IEEE Int'l Conf. Automated Software Eng., pp. 30-39, 2003.
[39] R. Santelices, J.A. Jones, Y. Yu, and M.J. Harrold, "Lightweight Fault-Localization Using Multiple Coverage Types," Proc. 31st Int'l Conf. Software Eng., pp. 56-66, May 2009.
[40] K. Sen, D. Marinov, and G. Agha, "CUTE: A Concolic Unit Testing Engine for C," Proc. European Software Eng. Conf. Held Jointly with 13th ACM SIGSOFT Int'l Symp. Foundations of Software Eng., 2005.
[41] M. Stoerzer, B.G. Ryder, X. Ren, and F. Tip, "Finding Failure-Inducing Changes in Java Programs Using Change Classification," Proc. SIGSOFT Int'l Symp. Foundations of Software Eng., pp. 57-68, Nov. 2006.
[42] F. Tip, "A Survey of Program Slicing Techniques," J. Programming Languages, vol. 3, no. 3 pp. 121-189, 1995.
[43] T. Wang and A. Roychoudhury, "Automated Path Generation for Software Fault Localization," Proc. 20th IEEE/ACM Int'l Conf. Automated Software Eng., pp. 347-351, 2005.
[44] G. Wassermann, D. Yu, A. Chander, D. Dhurjati, H. Inamura, and Z. Su, "Dynamic Test Input Generation for Web Applications," Proc. Int'l Symp. Software Testing and Analysis, pp. 249-260, 2008.
[45] C. Yilmaz, A.M. Paradkar, and C. Williams, "Time Will Tell: Fault Localization Using Time Spectra," Proc. 30th Int'l Conf. Software Eng., W. Schäfer, M.B. Dwyer, and V. Gruhn, eds., pp. 81-90, 2008.
[46] Y. Yu, J.A. Jones, and M.J. Harrold, "An Empirical Study of the Effects of Test-Suite Reduction on Fault Localization," Proc. Int'l Conf. Software Eng., pp. 201-210, 2008.
[47] A. Zeller, "Yesterday, My Program Worked. Today, It Does Not. Why?" SIGSOFT Software Eng. Notes, vol. 24, no. 6 pp. 253-267, 1999.
[48] A. Zeller, "Isolating Cause-Effect Chains from Computer Programs," Proc. ACM SIGSOFT Symp. Foundations of Software Eng., pp. 1-10, Nov. 2002.
[49] X. Zhang, N. Gupta, and R. Gupta, "Locating Faults through Automated Predicate Switching," Proc. Int'l Conf. Software Eng., pp. 272-281, 2006.
[50] Z. Zhang, W.K. Chan, T.H. Tse, B. Jiang, and X. Wang, "Capturing Propagation of Infected Program States," Proc. Joint Meeting of the European Software Eng. Conf. and the ACM SIGSOFT Symp. Foundations of Software Eng., pp. 43-52, 2009.

Index Terms:
Fault localization, statistical debugging, program analysis, web applications, PHP.
Citation:
Shay Artzi, Julian Dolby, Frank Tip, Marco Pistoia, "Fault Localization for Dynamic Web Applications," IEEE Transactions on Software Engineering, vol. 38, no. 2, pp. 314-335, March-April 2012, doi:10.1109/TSE.2011.76
Usage of this product signifies your acceptance of the Terms of Use.