This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
April-June 2011 (vol. 4 no. 2)
pp. 162-174
M Amelung, Eudemonia Solutions AG, Magdeburg, Germany
K Krieger, Fak. fur Inf./IWS, Otto-von-Guericke-Univ., Magdeburg, Germany
D Rosner, Fak. fur Inf./IWS, Otto-von-Guericke-Univ., Magdeburg, Germany
Assessment is an essential element in learning processes. It is therefore not unsurprising that almost all learning management systems (LMSs) offer support for assessment, e.g., for the creation, execution, and evaluation of multiple choice tests. We have designed and implemented generic support for assessment that is based on assignments that students submit as electronic documents. In addition to assignments that are graded by teachers, we also support assignments that can be automatically tested and evaluated, e.g., assignments in programming languages, or other formal notations. In this paper, we report about the design and implementation of a service-oriented approach for automatic assessment of programming assignments. The most relevant aspects of our “assessment as a service” solution are that on the one hand the advantages of automatic assessment can be used with a multitude of programming languages, as well as other formal notations (as so-called backends); on the other hand, the features of these types of assessment can be easily interfaced with different existing learning management systems (as so called frontends). We also report about the practical use of the implemented software components at our university and other educational institutions.

[1] M. Amelung, P. Forbrig, and D. Rösner, "Towards Generic and Flexible Web Services for E-Assessment," Proc. 13th Ann. Conf. Innovation and Technology in Computer Science Education (ITiCSE '08), pp. 219-224, 2008.
[2] M. Amelung, K. Krieger, and D. Rösner, "Flexibles E-Assessment auf Basis einer Service-orientierten Architektur," Proc. Lernen im Digitalen Zeitalter (DeLFI '09): 7, E-Learning Fachtagung Informatik, A. Schwill and N. Apostolopoulos, eds., pp. 247-258, 2009.
[3] R.J. Light, Making the Most of College: Students Speak Their Minds. Harvard Univ., 2001.
[4] M. Amelung, M. Piotrowski, and D. Rösner, "Educomponents: A Component-Based E-Learning Environment," Proc. 12th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education (ITiCSE '07), p. 352, 2007.
[5] M. Piotrowski, "Document-Oriented E-Learning Components," PhD thesis, Dept. of Computer Science, Otto von Guericke Univ., 2009.
[6] D. Rösner, M. Piotrowski, and M. Amelung, "A Sustainable Learning Environment Based on an Open Source Content Management System," Proc. German E-Science Conf. (GES '07), 2007.
[7] M. Piotrowski, M. Amelung, and D. Rösner, "Tactical, Document-Oriented E-Learning Components," Proc. IADIS Int'l Conf. E-Learning, pp. 171-177, 2007.
[8] P. Black and D. Wiliam, "Inside the Black Box: Raising Standards through Classroom Assessment," Phi Delta Kappan, vol. 80, no. 2, pp. 139-148, 1998.
[9] K.M. Ala-Mutka, "A Survey of Automated Assessment Approaches for Programming Assignments," J. Computer Science Education, vol. 15, no. 2, pp. 83-102, June 2005.
[10] G.E. Forsythe and N. Wirth, "Automatic Grading Programs," Comm. ACM, vol. 8, no. 5, pp. 275-278, 1965.
[11] M. Laakso, T. Salakoski, A. Korhonen, and L. Malmi, "Automatic Assessment of Exercises for Algorithms and Data Structures—A Case Study with TRAKLA2," Proc. Fourth Finnish/Baltic Sea Conf. Computer Science Education, pp. 28-36, Oct. 2004.
[12] R. Saikkonen, L. Malmi, and A. Korhonen, "Fully Automatic Assessment of Programming Exercises," Proc. Sixth Ann. Conf. Innovation and Technology in Computer Science Education (ITiCSE '01), pp. 133-136, 2001.
[13] M.T. Helmick, "Interface-Based Programming Assignments and Automatic Grading of Java Programs," Proc. 12th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education (ITiCSE '07), pp. 63-67, 2007.
[14] M. Striewe, M. Balz, and M. Goedicke, "A Flexible and Modular Software Architecture for Computer Aided Assessments and Automated Marking," Proc. First Int'l Conf. Computer Supported Education, pp. 54-61, 2009.
[15] C.A. Higgins, G. Gray, P. Symeonidis, and A. Tsintsifas, "Automated Assessment and Experiences of Teaching Programming," J. Educational Resources in Computing, vol. 5, no. 3, p. 5, 2005.
[16] M. Joy, N. Griffiths, and R. Boyatt, "The BOSS Online Submission and Assessment System," J. Educational Resources in Computing, vol. 5, no. 3, p. 2, 2005.
[17] C. Beierle, M. Kulaš, and M. Widera, "Automatic Analysis of Programming Assignments," Proc. der 1. E-Learning Fachtagung Informatik (DeLFI '03), A. Bode, J. Desel, S. Ratmayer, and M. Wessner, eds., vol. P-37, pp. 144-153, 2003.
[18] J. Krinke, M. Störzer, and A. Zeller, "Web-Basierte Programmierpraktika Mit Praktomat," Proc. des Workshop Neue Medien in der Informatik-Lehre, Oct. 2002.
[19] O. Gotel, C. Scharff, and A. Wildenberg, "Teaching Software Quality Assurance by Encouraging Student Contributions to an Open Source Web-Based System for the Assessment of Programming Assignments," Proc. 13th Ann. Conf. Innovation and Technology in Computer Science Education (ITiCSE '08), pp. 214-218, 2008.
[20] S.H. Edwards and M. Peréz-Quiñones, "Web-CAT: Automatically Grading Programming Assignments," Proc. 13th Ann. SIGCSE Conf. Innovation and Technology in Computer Science Education (ITiCSE '08), 2008.
[21] A. Hoffmann, A. Quast, and R. Wismüller, "Online-Übungssystem für die Programmierausbildung zur Einführung in die Informatik," Proc. Die 6. E-Learning Fachtagung Informatik (DeLFI '08), 2008.
[22] M. Rahn and J. Waldmann, "The Leipzig Autotool System for Grading Student Homework," Proc. Workshop E-Learning, M. Hanus, S. Krishnamurthi, and S. Thompson, eds., 2002.
[23] Epaile, Automatic Grading of Programs, http://docs.moodle.org/en/Student_projects Automated_grading_of_programs, 2010.
[24] N. Bieberstein, S. Bose, M. Fiammante, K. Jones, and R. Shah, Service-Oriented Architecture (SOA) Compass: Business Value, Planning, and Enterprise Roadmap. IBM, 2005.
[25] M. Bell, SOA Modeling Patterns for Service Oriented Discovery and Analysis. Wiley, 2010.
[26] G.J. Myers, The Art of Software Testing, second ed. Wiley, 2004.
[27] N. Provos, "Improving Host Security with System Call Policies," Proc. 12th USENIX Security Symp., Aug. 2003.
[28] T. Feustel, "Analyse von Texteingaben in einem CAA-Werkzeug zur Elektronischen Einreichung und Auswertung von Aufgaben," Master's thesis, Dept. of Computer Science, Otto von Guericke Univ., 2006.
[29] R. Binder, Testing Object-Oriented Systems: Models, Patterns and Tools, J.C. Shanklin, ed. Addison-Wesley, 2000.
[30] K. Claessen and J. Hughes, "QuickCheck: A Lightweight Tool for Random Testing of Haskell Programs," Proc. Fifth ACM SIGPLAN Int'l Conf. Functional Programming (ICFP '00), pp. 268-279, 2000.
[31] A. Zeller, "Making Students Read and Review Code," Proc. Fifth Ann. SIGCSE/SIGCUE ITiCSE Conf. Innovation and Technology in Computer Science Education (ITiCSE '00), pp. 89-92, July 2000.
[32] M. Otto, "Ontologien zur semantischen Suche in einem Bestand von Dokumenten," Master's thesis, Dept. of Computer Science, Otto von Guericke Univ., 2008.

Index Terms:
educational administrative data processing,computer aided instruction,computer science education,programming assignments,e-assessment,learning process,learning management systems,electronic documents,programming languages,formal notations,service-oriented approach,automatic assessment,Computer languages,Programming profession,Least squares approximation,Education,Automatic testing,eduComponents.,Computer science education,programming,learning systems,learning control systems,modular computer systems,service-oriented architecture,web services,e-learning,computer-aided assessment,e-assessment
Citation:
M Amelung, K Krieger, D Rosner, "E-Assessment as a Service," IEEE Transactions on Learning Technologies, vol. 4, no. 2, pp. 162-174, April-June 2011, doi:10.1109/TLT.2010.24
Usage of this product signifies your acceptance of the Terms of Use.