The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - Third Quarter (2012 vol.5)
pp: 251-263
Ming Liu , Sch. of Electr. & Inf. Eng., Univ. of Sydney, Sydney, NSW, Australia
R. A. Calvo , Sch. of Electr. & Inf. Eng., Univ. of Sydney, Sydney, NSW, Australia
A. Aditomo , Univ. of Sydney, NSW, Australia
L. A. Pizzato , Univ. of Sydney, NSW, Australia
ABSTRACT
In this paper, we present a novel approach for semiautomatic question generation to support academic writing. Our system first extracts key phrases from students' literature review papers. Each key phrase is matched with a Wikipedia article and classified into one of five abstract concept categories: Research Field, Technology, System, Term, and Other. Using the content of the matched Wikipedia article, the system then constructs a conceptual graph structure representation for each key phrase and the questions are then generated based the structure. To evaluate the quality of the computer generated questions, we conducted a version of the Bystander Turing test, which involved 20 research students who had written literature reviews for an IT methods course. The pedagogical values of generated questions were evaluated using a semiautomated process. The results indicate that the students had difficulty distinguishing between computer-generated and supervisor-generated questions. Computer-generated questions were also rated as being as pedagogically useful as supervisor-generated questions, and more useful than generic questions. The findings also suggest that the computer-generated questions were more useful for the first-year students than for second or third-year students.
INDEX TERMS
Decision support systems, Encyclopedias, Electronic publishing, Internet, Writing, Natural language processing,writing support, Decision support systems, Encyclopedias, Electronic publishing, Internet, Writing, Natural language processing, natural language processing, Automatic question generation
CITATION
Ming Liu, R. A. Calvo, A. Aditomo, L. A. Pizzato, "Using Wikipedia and Conceptual Graph Structures to Generate Questions for Academic Writing Support", IEEE Transactions on Learning Technologies, vol.5, no. 3, pp. 251-263, Third Quarter 2012, doi:10.1109/TLT.2012.5
REFERENCES
[1] G. Graswell, Writing for Academic Success: A Postgraduate Guide. SAGE, 2008.
[2] B. Steward, "Writing a Literature Review," The British J. Occupational Therapy, vol. 67, pp. 495-500, 2004.
[3] M. Afolabi, "The Review of Related Literature in Research," Int'l J. Information and Library Research, vol. 4, pp. 59-66, 1992.
[4] D. Taylor, "The Literature Review: A Few Tips on Conducting It," http://www.writing.utoronto.ca/advice/specific-types-of- writing literature-review , July 2011.
[5] V. Rus and A.C. Graesser, "The Question Generation Shared Task and Evaluation Challenge," Proc. Sixth Int'l Natural Language Generation Conf., pp. 251-257, 2009.
[6] R. Mitkov and L.A. Ha, "Computer-Aided Generation of Multiple-Choice Tests," Proc. HLT-NAACL Workshop Building Educational Applications Using Natural Language Processing, pp. 17-22, 2003.
[7] H. Kunichika, T. Katayama, T. Hirashima, and A. Takeuchi, "Automated Question Generation Methods for Intelligent English Learning Systems and Its Evaluation," Proc. Int'l Conf. Computers in Education, pp. 1117-1124, 2002.
[8] M. Heilman and N.A. Smith, "Good Question! Statistical Ranking for Question Generation," Proc. Ann. Conf. North Am. Chapter of the Assoc. for Computational Linguistics - Human Language Technologies, pp. 609-617, 2010.
[9] J. Mostow and W. Chen, "Generating Instruction Automatically for the Reading Strategy of Self-Questioning," Proc. Int'l Conf. Artificial Intelligence in Education, pp. 465-472, 2009.
[10] P. Mannem, R. Prasad, and A. Joshi, "Question Generation from Paragraphs at UPenn: QGSTEC System Description," Proc. Third Workshop Question Generation, pp. 84-91, 2010.
[11] M. Liu, R.A. Calvo, and V. Rus, "Automatic Question Generation for Literature Review Writing Support," Proc. 10th Int'l Conf. Intelligent Tutorial Systems, pp. 45-54, 2010.
[12] M. Liu and R.A. Calvo, "Question Taxonomy and Implications for Automatic Question Generation," Proc. Int'l Conf. Artificial Intelligence in Education, pp. 504-506, 2011.
[13] A.C. Graesser and N.K. Person, "Question Asking during Tutoring," Am. Education Research J., vol. 31, pp. 104-137, 1994.
[14] W. Lehnert, C. Cardie, and E. Riloff, "Analyzing Research Papers Using Citation Sentences," Proc. 12th Ann. Conf. Cognitive Science Soc., pp. 511-518, 1990.
[15] N.K. Person and A.C. Graesser, "Human or Computer? AutoTutor in a Bystander Turing Test," Proc. Sixth Int'l Conf. Intelligent Tutoring Systems (ITS '02), pp. 821-830, 2002.
[16] R. Haswell, "The complexities of Responding to Student Writing; or, Looking for Shortcuts via the Road of Excess," Across the Disciplines, vol. 3, 2006.
[17] M.D. Shermis and J.C. Burstein, Automated Essay Scoring: A Cross-Disciplinary Perspective, vol. 16. MIT, 2003.
[18] R. Williams and H. Dreher, "Automatically Grading Essays with MarkIT©," Proc. Informing Science Conf., 2004.
[19] J. Anderson, Mechanically Inclined: Building Grammar, Usage, and Style into Writer's Workshop. Stenhouse, 2005.
[20] E.C. Thiesmeyer and J.E. Theismeyer, Editor: A System for Checking Usage, Mechanics, Vocabulary, and Structure. Modern Language Assoc., 1990.
[21] M.A. Britt, P. Wiemer-Hastings, A.A. Larson, and C.A. Perfetti, "Using Intelligent Feedback to Improve Sourcing and Integration in Students' Essays," Int'l J. Artificial Intelligence, vol. 14, pp. 359-374, 2004.
[22] T.K. Landauer, D.S. McNamara, S. Dennis, and W. Kintsch, Handbook of Latent Semantic Analysis. Lawrence Erlbaum, 2007.
[23] P. Wiemer-Hastings and A.C. Graesser, "Select-a-Kibitzer: A Computer Tool that Gives Meaningful Feedback on Student Compositions," Interactive Learning Environments, vol. 8, pp. 149-169, 2000.
[24] D. Wade-Stein and E. Kintsch, "Summary Street: Interactive Computer Support for Writing," Cognition and Instruction, vol. 22, pp. 333-362, 2004.
[25] T.K. Landauer, D. Laham, and P.W. Foltz, "The Intelligent Essay Assessor," IEEE Intelligent Systems, vol. 15, no. 5, pp. 27-31, Sept./Oct. 2000.
[26] B. Lemaire and P. Dessus, "A System to Assess the Semantic Content of Student Essays," J. Educational Computing Research, vol. 24, pp. 305-320, 2001.
[27] J. Villalon, P. Kearney, R.A. Calvo, and P. Reimann, "Glosser: Enhanced Feedback for Student Writing Tasks," Proc. IEEE Eighth Int'l Conf. Advanced Learning Technologies (ICALT '08), pp. 454-458, 2008.
[28] R.A. Calvo and R.A. Ellis, "Students' Conceptions of Tutor and Automated Feedback in Professional Writing," J. Eng. Education, pp. 427-438, 2010.
[29] J.H. Wolfe, "Automatic Question Generation from Text - An Aid to Independent Study," SIGCUE Outlook, vol. 10, pp. 104-112, 1976.
[30] D. Coniam, "A Preliminary Inquiry into Using Corpus Word Frequency Data in the Automatic Generation of English Language Cloze Tests," CALICO J., vol. 14, no. 2, pp. 15-33, 1997.
[31] P.D. Turney, "Learning Algorithms for Keyphrase Extraction," Information Retrieval, vol. 2, pp. 303-336, 2000.
[32] E. Frank, G.W. Paynter, I.H. Witten, C. Gutwin, and C.G. Nevill-Manning, "Domain-Specific Keyphrase Extraction," Proc. 16th Int'l Joint Conf. Artificial Intelligence, 1999.
[33] K. Barker and N. Cornacchia, "Using Noun Phrase Heads to Extract Document Keyphrases," Proc. 13th Biennial Conf. Canadian Soc. Computational Studies of Intelligence, 2000.
[34] D.B. Bracewell, F. Ren, and S. Kuriowa, "Multilingual Single Document Keyword Extraction for Information Retrieval," Proc. Int'l Conf. Natural Language Processing and Knowledge Eng., pp. 517-522, 2005.
[35] R. Mihalcea and P. Tarau, "TextRank: Bringing Order into Texts," Proc. Conf. Empirical Methods in Natural Language Processing, 2004.
[36] S. Osinski, J. Stefanowski, and D. Weiss, "Lingo: Search Results Clustering Algorithm Based on Singular Value Decomposition," Proc. Int'l Conf. Intelligent Information Systems, 2004.
[37] R. Levy and G. Andrew, "Tregex and Tsurgeon: Tools for Querying and Manipulating Tree Data Structures," Proc. Fifth Int'l Conf. Language Resources and Evaluation, 2006.
[38] M. Heilman and N.A. Smith, "Extracting Simplified Statements for Factual Question Generation," Proc. Third Workshop Question Generation, 2010.
[39] D. Klein and C.D. Manning, "Fast Exact Inference with a Factored Model for Natural Language Parsing," Proc. Advances in Neural Information Processing Systems 15 (NIPS), pp. 3-10, 2003.
[40] R. Bunescu and M. Pasca, "Using Encyclopedic Knowledge for Named Entity Disambiguation," Proc. 11th Conf. European Chapter of Assoc. for Computational Linguistics, pp. 9-16, 2006.
[41] M. Ruiz-Casado, E. Alfonseca, and P. Castells, "Automatic Assignment of Wikipedia Encyclopedic Entries to WordNet Synsets," Proc. Atlantic Web Intelligence Conf. (AWIC), 2005.
[42] S. Cucerzan, "Large-Scale Named Entity Disambiguation Based on Wikipedia Data," Proc. Joint Conf. Empirical Methods in Natural Language Processing and Computational Natural Language Learning, 2007.
[43] I. Gurevych, C. Müller, and T. Zesch, "What to Be? - Electronic Career Guidance Based on Semantic Relatedness," Proc. 45th Ann. Meeting of the Assoc. of Computational Linguistics, 2007.
[44] D. Milne and I.H. Witten, "An Open-Source Toolkit for Mining Wikipedia," Proc. New Zealand Computer Science Research Student Conf., 2009.
[45] T. Zesch, C. Müller, and I. Gurevych, "Extracting Lexical Semantic Knowledge from Wikipedia and Wiktionary," Proc. Sixth Int'l Conf. Language Resources and Evaluation, 2008.
[46] A.M. Olney, W.L. Cade, and C. Williams, "Generating Concept Map Exercises from Textbook," Proc. ACL HLT Workshop Innovative Use of NLP for Building Educational Applications, 2011.
[47] B. Bloom, Taxonomy of Educational Objectives: The Classification of Educational Goals: Handbook I, Cognitive Domain. Longman, 1956.
[48] N.K. Person and A.C. Graesser, "Human or Computer? AutoTutor in a Bystander Turing Test," Proc. Sixth Int'l Conf. Intelligent Tutoring Systems, 2002.
96 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool