The Community for Technology Leaders
RSS Icon
Issue No.04 - Oct.-Dec. (2013 vol.6)
pp: 378-388
Ramkumar Rajendran , IITB-Monash Res. Acad., Indian Inst. of Technol., Mumbai, Mumbai, India
Sridhar Iyer , Dept. of Comput. Sci. & Eng., Indian Inst. of Technol., Mumbai, Mumbai, India
Sahana Murthy , Educ. Technol., Indian Inst. of Technol., Mumbai, Mumbai, India
Campbell Wilson , Fac. of Inf. Technol., Monash Univ., Melbourne, VIC, Australia
Judithe Sheard , Fac. of Inf. Technol., Monash Univ., Melbourne, VIC, Australia
The importance of affect in learning has led many intelligent tutoring systems (ITS) to include learners' affective states in their student models. The approaches used to identify affective states include human observation, self-reporting, data from physical sensors, modeling affective states, and mining students' data in log files. Among these, data mining and modeling affective states offer the most feasible approach in real-world settings, which may involve a huge number of students. Systems using data mining approaches to predict frustration have reported high accuracy, while systems that predict frustration by modeling affective states, not only predict a student's affective state but also the reason for that state. In our approach, we combine these approaches. We begin with the theoretical definition of frustration, and operationalize it as a linear regression model by selecting and appropriately combining features from log file data. We illustrate our approach by modeling the learners' frustration in Mindspark, a mathematics ITS with large-scale deployment. We validate our model by independent human observation. Our approach shows comparable results to existing data mining approaches and also the clear interpretation of the reasons for the learners' frustration.
Linear regression, Mathematical model, Predictive models, Data models, Learning systems, Sensors,frustration theory, Intelligent tutoring system, affective states, modeling frustration
Ramkumar Rajendran, Sridhar Iyer, Sahana Murthy, Campbell Wilson, Judithe Sheard, "A Theory-Driven Approach to Predict Frustration in an ITS", IEEE Transactions on Learning Technologies, vol.6, no. 4, pp. 378-388, Oct.-Dec. 2013, doi:10.1109/TLT.2013.31
[1] P. Brusilovsky and E. Millán, "User Models for Adaptive Hypermedia and Adaptive Educational Systems," The Adaptive Web: Methods and Strategies of Web Personalization, pp. 3-53, Springer-Verlag, 2007.
[2] B. Woolf, W. Burleson, I. Arroyo, T. Dragon, D. Cooper, and R. Picard, "Affect-Aware Tutors: Recognising and Responding to Student Affect," Int'l J. Learning Technology, vol. 4, nos. 3/4, pp. 129-164, 2009.
[3] S.K. D'Mello, S.D. Craig, J. Sullins, and A.C. Graesser, "Predicting Affective States Expressed through an Emote-Aloud Procedure from Autotutor's Mixed-Initiative Dialogue," Int'l J. Artificial Intelligence in Education, vol. 16, no. 1, pp. 3-28, 2006.
[4] J. Klein, Y. Moon, and R. Picard, "This Computer Responds to User Frustration: Theory, Design, and Results," Interacting with Computers, vol. 14, no. 2, pp. 119-140, 2002.
[5] R.S.J.d. Baker, S.K. D'Mello, M.M.T. Rodrigo, and A.C. Graesser, "Better to Be Frustrated than Bored: The Incidence, Persistence, and Impact of Learners' Cognitive-Affective States during Interactions with Three Different Computer-Based Learning Environments," Int'l J. Human-Computer Studies, vol. 68, no. 4, pp. 223-241, 2010.
[6] R.A. Calvo and S.K. D'Mello, "Affect Detection: An Interdisciplinary Review of Models, Methods, and Their Applications," IEEE Trans. Affective Computing, vol. 1, no. 1, pp. 18-37, Jan. 2010.
[7] Z. Zeng, M. Pantic, G.I. Roisman, and T.S. Huang, "A Survey of Affect Recognition Methods: Audio, Visual, and Spontaneous Expressions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 1, pp. 39-58, Jan. 2009.
[8] M.M.T. Rodrigo, R.S.J.d. Baker, M.C.V. Lagud, S.A.L. Lim, A.F. Macapanpan, S.A.M.S. Pascua, J.Q. Santillano, L.R.S. Sevilla, J.O. Sugay, S. Tep, and N.J.B. Viehland, "Affect and Usage Choices in Simulation Problem-Solving Environments," Proc. Int'l Conf. Artificial Intelligence in Education, pp. 145-152, 2007.
[9] M.M.T. Rodrigo, R.S.J.d. Baker, J. Agapito, J. Nabos, M.C. Repalam, S.S. Reyes, and M.O.C.Z.S. Pedro, "The Effects of an Interactive Software Agent on Student Affective Dynamics While Using an Intelligent Tutoring System," IEEE Trans. Affective Computing, vol. 3, no. 2, pp. 224-236, Apr.-June 2012.
[10] D. Cooper, I. Arroyo, B. Woolf, K. Muldner, W. Burleson, and R. Christopherson, "Sensors Model Student Self Concept in the Classroom," Proc. Int'l Conf. User Modeling, Adaptation, and Personalization, pp. 30-41, 2009.
[11] C. Conati and H. Maclaren, "Empirically Building and Evaluating a Probabilistic Model of User Affect," User Modeling and User-Adapted Interaction, vol. 19, no. 3, pp. 267-303, 2009.
[12] S.K. D'Mello, S.D. Craig, A. Witherspoon, B. Mcdaniel, and A.C. Graesser, "Automatic Detection of Learner's Affect from Conversational Cues," User Modeling and User-Adapted Interaction, vol. 18, nos. 1/2, pp. 45-80, 2008.
[13] M.M.T. Rodrigo and R.S.J.d. Baker, "Coarse-Grained Detection of Student Frustration in an Introductory Programming Course," Proc. Int'l Workshop Computing Education Research, pp. 75-80, 2009.
[14] J. Sabourin, B. Mott, and J.C. Lester, "Modeling Learner Affect with Theoretically Grounded Dynamic Bayesian Networks," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, pp. 286-295, 2011.
[15] S.K. D'Mello, S.D. Craig, B. Gholson, and S. Franklin, "Integrating Affect Sensors in an Intelligent Tutoring System," Proc. Int'l Conf. Intelligent User Interfaces, pp. 7-13, 2005.
[16] T. Dragon, I. Arroyo, B. Woolf, W. Burleson, R. Kaliouby, and H. Eydgahi, "Viewing Student Affect and Learning through Classroom Observation and Physical Sensors," Proc. Ninth Int'l Conf. Intelligent Tutoring Systems, pp. 29-39, 2008.
[17] K. Brawner and B. Goldberg, "Real-Time Monitoring of ECG and GSR Signals during Computer-Based Training," Proc. 11th Int'l Conf. Intelligent Tutoring Systems, pp. 72-77, 2012.
[18] M. Hussain, H. Monkaresi, and R. Calvo, "Categorical vs. Dimensional Representations in Multimodal Affect Detection during Learning," Proc. Int'l Conf. Intelligent Tutoring Systems, pp. 78-83, 2012.
[19] S. Srinivas, M. Bagadia, and A. Gupta, "Mining Information from Tutor Data to Improve Pedagogical Content Knowledge," Proc. Educational Data Mining, pp. 275-276, 2010.
[20] M.S. Hussain, O. AlZoubi, R.A. Calvo, and S.K. D'Mello, "Affect Detection from Multichannel Physiology during Learning Sessions with Autotutor," Proc. Int'l Conf. Artificial Intelligence in Education, pp. 131-138, 2011.
[21] M. Hall, E. Frank, G. Holmes, B. Pfahringer, P. Reutemann, and I.H. Witten, "The Weka Data Mining Software: An Update," SIGKDD Explorations, vol. 11, no. 1, pp. 10-18, 2009.
[22] S.W. Mcquiggan, S. Lee, and J.C. Lester, "Early Prediction of Student Frustration," Proc. Int'l Conf. Affective Computing and Intelligent Interaction, pp. 698-709, 2007.
[23] A. Ortony, G.L. Clore, and A. Collins, The Cognitive Structure of Emotions. Cambridge Univ. Press, 1990.
[24] R. Lawson, Frustration: The Development of a Scientific Concept, Critical Issues in Psychology Series. Macmillan, 1965.
[25] C.T. Morgan, R.A. King, J.R. Weisz, and J. Schopler, Introduction to Psychology, seventh ed. McGraw-Hill, 1986.
[26] A. Amsel, "Arousal, Suppression, and Persistence: Frustration Theory, Attention, and Its Disorders," Cognition & Emotion, vol. 4, no. 3, pp. 239-268, 1990.
[27] N.L. Stein and L.J. Levine, "Making Sense Out of Emotion," Memories, Thoughts, and Emotions: Essays in Honor of George Mandler, W. Kessen, A. Ortony, and F. Kraik, eds., pp. 295-322, Erlbaum, 1991.
[28] J.P. Gee, "What Video Games Have to Teach Us About Learning and Literacy," Computers in Entertainment, vol. 1, no. 1, pp. 20-23, 2003.
[29] J.P. Gee, "Learning and Games," The Ecology of Games: Connecting Youth, Games, and Learning, vol. 3, pp. 21-40, MIT Press, 2008.
[30] A. Kapoor, W. Burleson, and R.W. Picard, "Automatic Prediction of Frustration," Int'l J. Human-Computer Studies, vol. 65, pp. 724-736, 2007.
[31] J. Lazar, A. Jones, M. Hackley, and B. Shneiderman, "Severity and Impact of Computer User Frustration: A Comparison of Student and Workplace Users," Interacting with Computers, vol. 18, no. 2, pp. 187-207, 2006.
[32] T. Kanade, Y. Tian, and J.F. Cohn, "Comprehensive Database for Facial Expression Analysis," Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 46-53, 2000.
[33] Y. Tong, W. Liao, and Q. Ji, "Inferring Facial Action Units with Causal Relations," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 1623-1630, 2006.
[34] P. Ekman, "Strong Evidence for Universals in Facial Expressions: A Reply to Russell's Mistaken Critique," Psychology Bull., vol. 115, no. 2, pp. 268-287, 1994.
[35] H.A. Elfenbein and N. Ambady, "On the Universality and Cultural Specificity of Emotion Recognition: A Meta-Analysis," Psychology Bull., vol. 128, no. 2, pp. 203-235, 2002.
[36] M.K. Mandal, M.P. Bryden, and M.B. Bulman-Fleming, "Similarities and Variations in Facial Expressions of Emotions: Cross-Cultural Evidence," Int'l J. Psychology, vol. 31, no. 1, pp. 49-58, 1996.
[37] S. Geisser, Predictive Inference: An Introduction, Monographs on Statistics and Applied Probability. Chapman & Hall, 1993.
[38] G. Salton, Automatic Text Processing: The Transformation, Analysis, and Retrieval of Information by Computer. Addison-Wesley, 1989.
[39] Y. Yang, "An Evaluation of Statistical Approaches to Text Categorization," Information Retrieval, vol. 1, no. 1, pp. 69-90, 1999.
[40] D. Michie, D.J. Spiegelhalter, C.C. Taylor, and J. Campbell, Machine Learning, Neural and Statistical Classification. Ellis Horwood, 1994.
87 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool