The Community for Technology Leaders
Green Image
Issue No. 01 - Jan.-March (2018 vol. 11)
ISSN: 1939-1382
pp: 5-12
Mike Holmes , Manchester, Lancashire, United Kingdom
Annabel Latham , Manchester, Lancashire, United Kingdom
Keeley Crockett , Manchester, Lancashire, United Kingdom
James D. OShea , Manchester, Lancashire, United Kingdom
Comprehension is an important cognitive state for learning. Human tutors recognize comprehension and non-comprehension states by interpreting learner non-verbal behavior (NVB). Experienced tutors adapt pedagogy, materials, and instruction to provide additional learning scaffold in the context of perceived learner comprehension. Near real-time assessment for e-learner comprehension of on-screen information could provide a powerful tool for both adaptation within intelligent e-learning platforms and appraisal of tutorial content for learning analytics. However, literature suggests that no existing method for automatic classification of learner comprehension by analysis of NVB can provide a practical solution in an e-learning, on-screen, context. This paper presents design, development, and evaluation of COMPASS, a novel near real-time comprehension classification system for use in detecting learner comprehension of on-screen information during e-learning activities. COMPASS uses a novel descriptive analysis of learner behavior, image processing techniques, and artificial neural networks to model and classify authentic comprehension indicative non-verbal behavior. This paper presents a study in which 44 undergraduate students answered on-screen multiple choice questions relating to computer programming. Using a front-facing USB web camera the behavior of the learner is recorded during reading and appraisal of on-screen information. The resultant dataset of non-verbal behavior and question-answer scores has been used to train artificial neural network (ANN) to classify comprehension and non-comprehension states in near real-time. The trained comprehension classifier achieved normalized classification accuracy of 75.8 percent.
Real-time systems, Compass, Electronic learning, Cameras, Neural networks, Image processing, Computational modeling

M. Holmes, A. Latham, K. Crockett and J. D. OShea, "Near Real-Time Comprehension Classification with Artificial Neural Networks: Decoding e-Learner Non-Verbal Behavior," in IEEE Transactions on Learning Technologies, vol. 11, no. 1, pp. 5-12, 2018.
167 ms
(Ver 3.3 (11022016))