This Article 
 Bibliographic References 
 Add to: 
Suleyman Cetintas, Purdue University, West Lafayette
Luo Si, Purdue University, West Lafayette
Yan Ping Xin, Purdue University, West Lafayette
Casey Hord, Purdue University, West Lafayette
Identifying off-task behaviors in intelligent tutoring systems is a practical and challenging research topic. This paper proposes a machine learning model that can automatically detect students' off-task behaviors. The proposed model only utilizes the data available from the log files that record students' actions within the system. The model utilizes a set of time features, performance features, and mouse movement features, and is compared to 1) a model that only utilizes time features and 2) a model that uses time and performance features. Different students have different types of behaviors; therefore, personalized version of the proposed model is constructed and compared to the corresponding nonpersonalized version. In order to address data sparseness problem, a robust Ridge Regression algorithm is utilized to estimate model parameters. An extensive set of experiment results demonstrates the power of using multiple types of evidence, the personalized model, and the robust Ridge Regression algorithm.

[1] R. Baeza-Yates and B. Ribeiro-Neto, Modern Information Retrieval, pp. 75-82. Addison Wesley, 1999.
[2] R.S. Baker, "Modeling and Understanding Students' Off-Task Behavior in Intelligent Tutoring Systems," Proc. SIGCHI Conf. Human Factors in Computing Systems, pp. 1059-1068, 2007.
[3] R.S. Baker, A.T. Corbett, K.R. Koedinger, and A.Z. Wagner, "Off-Task Behavior in the Cognitive Tutor Classroom: When Students 'Game the System,'" Proc. SIGCHI Conf. Human Factors in Computing Systems (CHI '04), pp. 383-390, 2004.
[4] R.S. Baker, I. Roll, A.T. Corbett, and K.R. Koedinger, "Do Performance Goals Lead Students to Game the System," Proc. 12th Int'l Conf. Artificial Intelligence and Education (AIED '05), pp. 57-64, 2005.
[5] C. Beal, B.P. Woolf, J. Beck, I. Arroyo, K. Schultz, and D.M. Hart, "Gaining Confidence in Mathematics: Instructional Technology for Girls," Proc. Int'l Conf. Math./Science Education and Technology, 2000.
[6] J.E. Beck and B.P. Woolf, "High Level Student Modeling with Machine Learning," Proc. Intelligent Tutoring Systems Conf., pp. 584-593, 2000.
[7] J. Beck, "Engagement Tracing: Using Response Times to Model Student Disengagement," Proc. 12th Int'l Conf. Artificial Intelligence in Education (AIED '05), pp. 88-95, 2005.
[8] C.M. Bishop, Pattern Recognition and Machine Learning, pp. 6-10, 144-145. Springer, 2006.
[9] S. Cetintas, L. Si, Y.P. Xin, and C. Hord, "Predicting Correctness of Problem Solving from Low-Level Log Data in Intelligent Tutoring Systems," Proc. Second Int'l Conf. Educational Data Mining (EDM '09), pp. 230-239, 2009.
[10] S. Cetintas, L. Si, Y.P. Xin, C. Hord, and D. Zhang, "Learning to Identify Students' Off-Task Behavior in Intelligent Tutoring Systems," Proc. 14th Int'l Conf. Artificial Intelligence and Education (AIED '09), pp. 701-703, 2009.
[11] T. Dalton, R.C. Martella, and N.E. Marchand-Martella, "The Effects of a Self-Management Program in Reducing Off-Task Behavior," J. Behavioral Education, vol. 9, nos. 3/4, pp. 157-176, 1999.
[12] F. Davis, La Comunicación No Verbal, vol. 616, El Libro de Bolsillo, Alianza ed., translated by L. Mourglier from Inside Intuition— What We Knew about Non-Verbal Communication, McGraw-Hill Book Company, 1976.
[13] A. De Vicente and H. Pain, "Informing the Detection of the Students' Motivational State: An Empirical Study," Proc. Intelligent Tutoring Systems, pp. 933-943, 2002.
[14] T. Hastie, R. Tibshirani, and J. Friedman, The Elements of Statistical Learning, pp. 61-68. Springer, 2001.
[15] N. Karweit and R.E. Slavin, "Time-on-Task: Issues of Timing, Sampling, and Definition," J. Experimental Psychology, vol. 74, no. 6, pp. 844-851, 1982.
[16] K.R. Koedinger and J.R. Anderson, "Intelligent Tutoring Goes to School in the Big City," Int'l J. Artificial Intelligence in Education, vol. 8, pp. 30-43, 1997.
[17] H.M. Lahaderne, "Attitudinal and Intellectual Correlates of Attention: A Study of Four Sixth-Grade Classrooms," J. Educational Psychology, vol. 59, no. 5, pp. 320-324, 1968.
[18] S.W. Lee, K.E. Kelly, and J.E. Nyre, "Preliminary Report on the Relation of Students' On-Task Behavior with Completion of School Work," Psychological Reports, vol. 84, pp. 267-272, 1999.
[19] D.J. Litman and K. Forbes-Riley, "Predicting Student Emotions in Computer-Human Tutoring Dialogues," Proc. 42nd Ann. Meeting on Assoc. for Computational Linguistics, 2004.
[20] E.M. Maletsky et al., Harcourt Math, Indiana ed. Harcourt, 2004.
[21] C. Merten and C. Conati, "Eye-Tracking to Model and Adapt to User Meta-Cognition in Intelligent Learning Environments," Proc. 11th Int'l Conf. Intelligent User Interfaces, pp. 39-46, 2006.
[22] R.C. Murray and K. VanLehn, "Effects of Dissuading Unnecessary Help Requests While Providing Proactive Help," Proc. 12th Int'l Conf. Artificial Intelligence in Education (AIED '05), pp. 887-889, 2005.
[23] N. Person, B. Klettke, K. Link, and R. Kreuz, "The Integration of Affective Responses into Autotutor," Proc. Int'l Workshop Affect in Interactions. Towards a New Generation of Interfaces, 1999.
[24] C.J. van Rijsbergen, Information Retrieval, second ed. Univ. of Glasgow, 1979.
[25] J.W. Schofield, Computers and Classroom Culture. Cambridge Univ. Press, 1995.
[26] L. Suchman, Plans and Situated Actions: The Problem of Human-Machine Communication. Cambridge Univ. Press, 1987.
[27] J.A. Walonoski and N.T. Heffernan, "Prevention of Off-Task Gaming Behavior in Intelligent Tutoring Systems," Intelligent Tutoring Systems, pp. 722-724, Springer, 2006.
[28] T.R. Ziemek, "Two-D or Not Two-D: Gender Implications of Visual Cognition in Electronic Games," Proc. Symp. Interactive 3D Graphics and Games, pp. 183-190, 2006.

Index Terms:
Computer uses in education, adaptive and intelligent educational systems.
Suleyman Cetintas, Luo Si, Yan Ping Xin, Casey Hord, "Automatic Detection of Off-Task Behaviors in Intelligent Tutoring Systems with Machine Learning Techniques," IEEE Transactions on Learning Technologies, vol. 3, no. 3, pp. 228-236, July-Sept. 2010, doi:10.1109/TLT.2009.44
Usage of this product signifies your acceptance of the Terms of Use.