This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Transformation-Based Diagnosis of Student Programs for Programming Tutoring Systems
April 2003 (vol. 29 no. 4)
pp. 360-384
Songwen Xu, IEEE Computer Society

Abstract—A robust technology that automates the diagnosis of students' programs is essential for programming tutoring systems. Such technology should be able to determine whether programs coded by a student are correct. If a student's program is incorrect, the system should be able to pinpoint errors in the program as well as explain and correct the errors. Due to the difficulty of this problem, no existing system performs this task entirely satisfactorily, and this problem still hampers the development of programming tutoring systems. This paper describes a transformation-based approach to automate the diagnosis of students' programs for programming tutoring systems. Improved control-flow analysis and data-flow analysis are used in program analysis. Automatic diagnosis of student programs is achieved by comparing the student program with a specimen program at the semantic level after both are standardized. The approach was implemented and tested on 525 real student programs for nine different programming tasks. Test results show that the method satisfies the requirements stated above. Compared to other existing approaches to automatic diagnosis of student programs, the approach developed here is more rigorous and safer in identifying student programming errors. It is also simpler to make use of in practice. Only specimen programs are needed for the diagnosis of student programs. The techniques of program standardization and program comparison developed here may also be useful for research in the fields of program understanding and software maintenance.

[1] D. Allemang, “Using Functional Modes in Automatic Debugging,” IEEE Expert, vol. 6, no. 6, pp. 13-18, 1991.
[2] J.R. Anderson, R. Farrell, and R. Sauers, “Learning to Program in Lisp,” Cognitive Science, vol. 8, pp. 87-129, 1984.
[3] S.K. Abd-El-Hafiz and V.R. Basili, A Knowledge-Based Approach to Program Understanding. Kluwer Academic, 1995.
[4] A. Adam and J. Laurent, “A System to Debug Student Programs,” Artificial Intelligence, vol. 15, no. 1, pp. 75-122, 1980.
[5] M. Elsom-Cook and B. du Boulay, “A Pascal Program Checker,” Artificial Intelligence and Human Learning, J. Self, ed., pp. 361-373, 1988.
[6] J. Ferrante,K.J. Ottenstein,, and J.D. Warren,“The program dependence graph and its use in optimization,” ACM Trans. Programming Languages and Systems, vol. 9, no. 3, pp. 319-349, June 1987.
[7] T.S. Gegg-Harrison, “Exploiting Program Schemata in an Automated Program Debugger,” J. Artificial Intelligence in Education, vol. 5, no. 2, pp. 255-278, 1994.
[8] A. Goldberg and D. Robson, Smalltalk 80: The Language and Its Implementation.Reading, Mass.: Addison Wesley, 1983.
[9] J.S. Hahn and J. Kim, “Automatic Problem Description from Model Program for Knowledge-based Program Tutor,” Automatic Software Eng., vol. 4, pp. 439-461, 1997.
[10] N. Hattori and N. Ishii, “A Method to Remove Variations in Source Codes,” Information and Software Technology, vol. 38, pp. 25-36, 1996.
[11] S. Horwitz, “Identifying the Semantic and Textual Differences Between Two Versions of a Program,“ Proc. SIGPLAN '90 Conf. Programming Language Design and Implementation, pp. 234-244, 1990.
[12] J.E. Hopcroft and J.D. Ullman, Introduction to Automata Theory, Languages and Computation. Addison-Wesley, Apr. 1979.
[13] D. Jackson and D.A. Ladd, “Semantic Diff: A Tool for Summarizing the Effects of Modifications,” Int'l Conf. Software Maintenance, 1994.
[14] W.L. Johnson and E. Soloway, “Proust: Knowledge-Based Program Understanding,” IEEE Trans. Software Eng., vol. 11, no. 3, pp. 11-19, Nov. 1985.
[15] W. Kozaczynski, J. Ning, and A. Engberts, "Program Concept Recognition and Transformation," IEEE Trans. Software Eng., vol. 18, no. 12, pp. 1,065-1,075, Dec. 1992.
[16] E. Lemut, B. du Boulay, and G. Dettori, Cognitive Models and Intelligent Environments for Learning Programming. Springer-Verlag, 1992.
[17] C. Looi, “Automatic Debugging of Prolog Programs in a Prolog Intelligent Teaching System,” Instructional Science, vol. 20, pp. 215-263, 1991.
[18] D. Loveman, “Program Improvement by Source to Source Transformation,” J. ACM, vol. 24, no. 1, pp. 121-145, 1977.
[19] J.D. McGregor, B.A. Malloy, and R.L. Siegmund, “A Comprehensive Program Representation of Object-Oriented Software,” Ann. Software Eng., vol. 2, pp. 51-91, 1996.
[20] S.S. Muchnick, Advanced Compiler Design and Implementation, Morgan Kaufmann, San Francisco, Calif., 1997.
[21] W.R. Murray, Automatic Program Debugging for Intelligent Tutoring Systems, Morgan Kaufmann, 1988.
[22] A. Pettorossi and M. Proietti, “Transformation of Logic Programs: Foundations and Techniques,” J. Logic Programming, vols. 19-20, pp. 261-320, 1994.
[23] C. Rich and L. Wills, "Recognizing a Program's Design: A Graph Parsing Approach," IEEE Software, vol. 7, no. 1, 1990.
[24] E.Y. Shapiro, Algorithmic Program Debugging. MIT Press, 1983.
[25] G. Thorburn and G. Rowe, “PASS: An Automated System for Program Assessment,” Computers and Education, vol. 29, no. 4, pp. 195-206, 1997.
[26] H. Ueno, “Concepts and Methodologies for Knowledge-Based Program Understanding-the ALPUS's Approach,” IEICE Trans. Information and Systems, vol. E78-D, no. 9, pp. 1108-1117, 1995.
[27] W. Yang, S. Horwitz, and T. Reps, “A Program Integration Algorithm that Accommodates Semantics-Preserving Transformations,” ACM Trans. Software Eng. and Methodology, vol. 1, no. 3, pp. 310-354, July 1992.

Index Terms:
Automatic program diagnosis, programming tutoring systems, program representation, program transformation, program comparison, program matching.
Citation:
Songwen Xu, Yam San Chee, "Transformation-Based Diagnosis of Student Programs for Programming Tutoring Systems," IEEE Transactions on Software Engineering, vol. 29, no. 4, pp. 360-384, April 2003, doi:10.1109/TSE.2003.1191799
Usage of this product signifies your acceptance of the Terms of Use.