The Community for Technology Leaders
RSS Icon
Issue No.03 - July-September (2010 vol.3)
pp: 237-249
Christopher J. Harrison , The University of Manchester, Manchester
Raheel Siddiqi , The University of Manchester, Manchester
Automated short-answer marking cannot “guarantee” 100 percent agreement between the marks generated by a software system and marks produced separately by a human. This problem has prevented automated marking systems from being used in high-stake short-answer marking. This paper describes how an automated short-answer marking system, called IndusMarker, can be effectively used to improve teaching and learning in an Object-Oriented Programming course taught at a university. The system is designed for factual answers where there is a clear criterion for answers being right or wrong. The system is based on structure matching, i.e., matching a prespecified structure, developed via a purpose-built structure editor, with the content of the student's answer text.
Automatic assessment tools, self-assessment, classroom feedback systems, computer science education.
Christopher J. Harrison, Raheel Siddiqi, "Improving Teaching and Learning through Automated Short-Answer Marking", IEEE Transactions on Learning Technologies, vol.3, no. 3, pp. 237-249, July-September 2010, doi:10.1109/TLT.2010.4
[1] K.E. Holbert and G.G. Karady, "Strategies, Challenges and Prospects for Active Learning in the Computer-Based Classroom," IEEE Trans. Education, vol. 52, no. 1, pp. 31-38, Feb. 2009.
[2] C. Leacock and M. Chodorow, "C-Rater: Automated Scoring of Short-Answer Question," Computers and the Humanities, vol. 37, no. 4, pp. 389-405, 2003.
[3] J.Z. Sukkarieh and S.G. Pulman, "Automatic Short Answer Marking," Proc. Second Workshop Building Educational Applications Using NLP, pp. 9-16, June 2005.
[4] T. Mitchell, T. Russel, P. Broomhead, and N. Aldridge, "Towards Robust Computerized Marking of Free-Text Responses," Proc. Sixth Int'l Computer Assisted Assessment Conf., 2002.
[5] R. Siddiqi and C.J. Harrison, "A Systematic Approach to the Automated Marking of Short-Answer Questions," Proc. 12th IEEE Int'l Multi Topic Conf. (IEEE INMIC), pp. 329-332, 2008.
[6] M. Farrow and P.J.B. King, "Experiences with Online Programming Examinations," IEEE Trans. Education, vol. 51, no. 2, pp. 251-255, May 2008.
[7] Course Website of the "Object Oriented Programming" Course, , 2009.
[8] Official Website of Bahria Univ., about-bahria why-bahria.htm, 2009.
[9] Official Website of Higher Education Commission Pakistan, http:/, 2010.
[10] Official Website of Pakistan Eng. Council, http:/www.pec., 2010.
[11] J.H. McMillan, Classroom Assessment: Principles and Practice for Effective Instruction, pp. 40 and 117. Allyn and Bacon, 1997.
[12] J.T. Dillon, Questioning and Teaching: A Manual of Practice. Teachers College Press, 1988.
[13] L.R. Gay, "The Comparative Effects of Multiple-Choice versus Short-Answer Tests on Retention," J. Educational Measurement, vol. 17, no. 1, pp. 45-50, Spring 1980.
[14] M.A. McDaniel, J.L. Anderson, M.H. Derbish, and N. Morrisette, "Testing the Testing Effect in the Classroom," European J. Cognitive Psychology, vol. 19, nos. 4/5, pp. 494-513, 2007.
[15] W.J. Hanson, "Creation of Hierarchic Text with a Computer Display," PhD thesis, Stanford Univ., 1971.
[16] S. Holzner, Inside XML, pp. 9-10. New Riders, 2001.
[17] D. Gulbransen, The Complete Idiot's Guide to XML, pp. 131-133. Que, 2000.
[18] S.S. Laurent and E. Cerami, Building XML Applications, p. 46. McGraw-Hill, 1999.
[19] The Stanford Natural Language Processing Group, "The Stanford Parser: A Statistical Parser," , 2009.
[20] i-net Software, "JOrtho—A Java Spell-Checking Library," jortho, 2009.
[21] Wiktionary, "Wiktionary—A Wiki-Based Open Content Dictionary," http:/, 2009.
[22] M.C. de Marneffe, B. MacCartney, and C.D. Manning, "Generating Typed Dependency Parses from Phrase Structure Parses," Proc. Fifth Int'l Conf. Language Resources and Evaluation, pp. 449-454, 2006.
[23] B. McLaughlin, Java and XML, pp. 125-135. O'Reilly & Assoc., 2000.
[24] K. Cagle, M. Corning, J. Diamond, T. Duynstee, O.G. Gudmundsson, M. Mason, J. Pinnock, P. Spencer, J. Tang, and A. Watt, Professional XSL. Wrox Press, 2001.
[25] F. Marton and R. Saljo, "On Qualitative Differences in Learning—I: Outcome and Process," British J. Educational Psychology, vol. 46, pp. 4-11, 1976.
[26] L.S. Ming, "An ICT-Based Technique for Assessment of Short-Answer Question," Proc. Seventh Int'l Conf. Information Technology Based Higher Education and Training, 2006.
[27] R. Siddiqi and C.J. Harrison, "On the Automated Assessment of Short Free-Text Responses," Proc. 34th Ann. Conf. Int'l Assoc. for Educational Assessment (IAEA), 2008.
[28] C. Daly and J.M. Horgan, "An Automated Learning System for Java Programming," IEEE Trans. Education, vol. 47, no. 1, pp. 10-17, Feb. 2004.
24 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool