The Community for Technology Leaders
RSS Icon
Issue No.04 - Fourth Quarter (2012 vol.5)
pp: 304-317
S. K. D'Mello , Depts. of Comput. Sci. & Psychol., Univ. of Notre Dame, Notre Dame, IN, USA
A. Graesser , Dept. of Psychol., Univ. of Memphis, Memphis, TN, USA
We explored the possibility of predicting student emotions (boredom, flow/engagement, confusion, and frustration) by analyzing the text of student and tutor dialogues during interactions with an Intelligent Tutoring System (ITS) with conversational dialogues. After completing a learning session with the tutor, student emotions were judged by the students themselves (self-judgments), untrained peers, and trained judges. Transcripts from the tutorial dialogues were analyzed with four methods that included 1) identifying direct expressions of affect, 2) aligning the semantic content of student responses to affective terms, 3) identifying psychological and linguistic terms that are predictive of affect, and 4) assessing cohesion relationships that might reveal student affect. Models constructed by regressing the proportional occurrence of each emotion on textual features derived from these methods yielded large effects (R2 = 38%) for the psychological, linguistic, and cohesion-based methods, but not the direct expression and semantic alignment methods. We discuss the theoretical, methodological, and applied implications of our findings toward text-based emotion detection during tutoring.
Semantics, Education, Pragmatics, Natural language processing, Psychology, Feature extraction, Particle measurements, Emotion recognition, Behavioral science,intelligent tutoring system, Semantics, Education, Pragmatics, Natural language processing, Psychology, Feature extraction, Particle measurements, Emotion recognition, Behavioral science, affect from text, Emotions, affect
S. K. D'Mello, A. Graesser, "Language and Discourse Are Powerful Signals of Student Emotions during Tutoring", IEEE Transactions on Learning Technologies, vol.5, no. 4, pp. 304-317, Fourth Quarter 2012, doi:10.1109/TLT.2012.10
[1] R.A. Calvo and S.K. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 18-37, Jan. 2010, doi:10.1109/T-AFFC.2010.1.
[2] Z. Zeng et al., "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009, doi:10.1109/TPAMI.2008.52.
[3] New Perspectives on Affect and Learning Technologies, R. Calvo and S. D'Mello, eds. Springer, 2011.
[4] Emotion in Education, P. Schutz and R. Pekrun, eds. Academic, 2007.
[5] K. Issroff and T. del Soldato, "Incorporating Motivation into Computer-Supported Collaborative Learning," European Conf. Artificial Intelligence in Education, 1996.
[6] R. Picard, Affective Computing. MIT, 1997.
[7] S.K. D'Mello et al., "Automatic Detection of Learner's Affect from Conversational Cues," User Modeling and User-Adapted Interaction, vol. 18, nos. 1/2, pp. 45-80, 2008.
[8] A. Graesser et al., "AutoTutor: A Tutor with Dialogue in Natural Language," Behavioral Research Methods, Instruments, and Computers, vol. 36, pp. 180-193, 2004.
[9] S. D'Mello and A. Graesser, "Emotions during Learning with AutoTutor," Adaptive Technologies for Training and Education, P. Durlach and A. Lesgold, eds., Cambridge Univ., 2012.
[10] S. D'Mello, "A Meta-Analysis on the Incidence of Emotions during Complex Learning," in review.
[11] A. Graesser et al., "Coh-Metrix: Analysis of Text on Cohesion and Language," Behavior Research Methods, Instruments, and Computers, vol. 36, pp. 193-202, 2004.
[12] J. Pennebaker et al., Linguistic Inquiry and Word Count (LIWC): A Computerized Text Analysis Program. Erlbaum, 2001.
[13] A. Graesser et al., "Detection of Emotions during Learning with AutoTutor," Proc. 28th Ann. Conf. Cognitive Science Soc., pp. 285-290, 2006.
[14] J. Pennebaker et al., "Psychological Aspects of Natural Language Use: Our Words, Our Selves," Ann. Rev. Psychology, vol. 54, pp. 547-577, 2003, doi:10.1146/annurev.psych.54.101601.145041.
[15] B. Pang and L. Lee, "Opinion Mining and Sentiment Analysis," Foundations and Trends in Information Retrieval, vol. 2, nos. 1/2, pp. 1-135, 2008.
[16] A. Ward et al., "Predicting Change in Student Motivation by Measuring Cohesion between Tutor and Student," Proc. Sixth Workshop Innovative Use of NLP for Building Educational Applications (ACL-HLT Workshop), pp. 136-141, 2011.
[17] A. Ward and D. Litman, "Semantic Cohesion and Learning," Proc. Ninth Int'l Conf. Intelligent Tutoring Systems, pp. 459-469, 2008.
[18] J. Wiebe et al., "Annotating Expressions of Opinions and Emotions in Language," Language Resources and Evaluation, vol. 39, no. 2, pp. 165-210, 2005.
[19] D. Litman and K. Forbes-Riley, "Recognizing Student Emotions and Attitudes on the Basis of Utterances in Spoken Tutoring Dialogues with Both Human and Computer Tutors," Speech Comm., vol. 48, no. 5, pp. 559-590, 2006, doi:10.1016/j.specom.2005.09.008.
[20] K. Forbes-Riley and D.J. Litman, "Benefits and Challenges of Real-Time Uncertainty Detection and Adaptation in a Spoken Dialogue Computer Tutor," Speech Comm., vol. 53, nos. 9/10, pp. 1115-1136, 2011, doi:10.1016/j.specom.2011.02.006.
[21] C.E. Osgood et al., Cross-Cultural Universals of Affective Meaning. Univ. of Illi nois, 1975.
[22] A. Samsonovich and G. Ascoli, "Cognitive Map Dimensions of the Human Value System Extracted from Natural Language," Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms, B. Goertzel and P. Wang, eds., pp. 111-124, IOS Press, 2007.
[23] L. Barrett et al., "The Experience of Emotion," Ann. Rev. Psychology, vol. 58, pp. 373-403, 2007, doi:10.1146/annurevpsych. 58.110405.085709.
[24] J. Kahn et al., "Measuring Emotional Expression with the Linguistic Inquiry and Word Count," Am. J. Psychology, vol. 120, no. 2, pp. 263-286, 2007.
[25] C.G. Shields et al., "Emotion Language in Primary care Encounters: Reliability and Validity of an Emotion Word Count Coding System," Patient Education and Counseling, vol. 57, no. 2, pp. 232-238, 2005, doi:10.1016/j.pec.2004.06.005.
[26] C. Chung and J. Pennebaker, "The Psychological Functions of Function Words," Social Comm., K. Fielder, ed., pp. 343-359, Psychology, 2007.
[27] W. Weintraub, Verbal Behavior in Everyday Life. Springer, 1989.
[28] C. Strapparava and A. Valitutti, "WordNet-Affect: An Affective Extension of WordNet," Proc. Int'l Conf. Language Resources and Evaluation, pp. 1083-1086, 2004.
[29] M. Bradley and P. Lang, Affective Norms for English Words (ANEW): Stimuli, Instruction Manual, and Affective Ratings, technical report, Univ. of Florida, 1999.
[30] R. Calvo and M. Kim, "Emotions in Text: Dimensional and Categorical Models," Computational Intelligence, in press.
[31] A. Gill et al., "Identifying Emotional Characteristics from Short Blog Texts," Proc. 30th Ann. Conf. Cognitive Science Soc., pp. 2237-2242, 2008.
[32] Handbook of Latent Semantic Analysis, T. Landauer et al., eds. Erlbaum, 2007.
[33] K. Lund and C. Burgess, "Producing High-Dimensional Semantic Spaces from Lexical Co-Occurrence," Behavior Research Methods Instruments and Computers, vol. 28, pp. 203-208, 1996.
[34] H. Liu et al., "A Model of Textual Affect Sensing Using Real-World Knowledge," Proc. Eighth Int'l Conf. Intelligent User Interfaces, pp. 1225-1132, 2003.
[35] M. Shaikh et al., "Sentiment Assessment of Text by Analyzing Linguistic Features and Contextual Valence Assignment," Applied Artificial Intelligence, vol. 22, no. 6, pp. 558-601, 2008, doi:10.1080/08839510802226801.
[36] J.W. Pennebaker et al., The Development and Psychometric Properties of LIWC, Univ. of Texas at Austin and Univ. of Auckland, 2007.
[37] D.S. McNamara et al., "Coh-Metrix: Capturing Linguistic Features of Cohesion," Discourse Processes, in press.
[38] D. Jurafsky and J.H. Martin, Speech and Language Processing: An Introduction to Natural Language Processing, Computational Linguistics, and Speech Recognition. Prentice-Hall, 2008.
[39] G.A. Miller et al., "Introduction to WordNet: An On-Line Lexical Database," J. Lexiography, vol. 3, pp. 235-244, 1990.
[40] G. Klare, "Assessing Readability," Reading Research Quarterly, vol. 10, no. 1, pp. 62-102, 1974.
[41] K. VanLehn et al., "When are Tutorial Dialogues More Effective Than Reading?" Cognitive Science, vol. 31, no. 1, pp. 3-62, 2007.
[42] E. Rosenberg and P. Ekman, "Coherence between Expressive and Experiential Systems in Emotion," Cognition and Emotion, vol. 8, no. 3, pp. 201-229, 1994.
[43] P. Ekman and W. Friesen, The Facial Action Coding System: A Technique for the Measurement of Facial Movement. Consulting Psychologists, 1978.
[44] S. D'Mello and A. Graesser, "Multimodal Semi-Automated Affect Detection from Conversational Cues, Gross Body Language, and Facial Features," User Modeling and User-Adapted Interaction, vol. 20, no. 2, pp. 147-187, 2010.
[45] J. Ang et al., "Prosody-Based Automatic Detection of Annoyance and Frustration in Human-Computer Dialog," Proc. Int'l Conf. Spoken Language Processing, pp. 2037-2039, 2002.
[46] D. Litman and K. Forbes-Riley, "Predicting Student Emotions in Computer-Human Tutoring Dialogues," Proc. 42nd Ann. Meeting on Assoc. for Computational Linguistics, pp. 352-359, 2004.
[47] S. Afzal and P. Robinson, "Natural Affect Data: Collection and Annotation," New Perspective on Affect and Learning Technologies, R. Calvo and S. D'Mello, eds., pp. 44-70, Springer, 2011.
[48] M.E. Hoque et al., "When Human Coders (and Machines) Disagree on the Meaning of Facial Affect in Spontaneous Videos," Proc. Ninth Int'l Conf. Intelligent Virtual Agents, 2009.
[49] W. Kintsch, Comprehension: A Paradigm for Cognition. Cambridge Univ., 1998.
[50] J. Cohen, "A Power Primer," Psychological Bull., vol. 112, no. 1, pp. 155-159, 1992.
[51] R. Hocking, "Analysis and Selection of Variables in Linear Regression," Biometrics, vol. 32, no. 1, pp. 1-49, 1976.
[52] R.H. Baayen, Analyzing Linguistic Data: A Practical Introduction to Statistics Using R. Cambridge Univ, 2008.
[53] N. Person et al., "Pedagogical and Motivational Dialogue Moves Used by Expert Tutors," Proc. 17th Ann. Meeting of the Soc. for Text and Discourse, 2007.
62 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool