This Article 
 Bibliographic References 
 Add to: 
Monitoring Online Tests through Data Visualization
June 2009 (vol. 21 no. 6)
pp. 773-784
Gennaro Costagliola, Università di Salerno, Fisciano, Italy
Vittorio Fuccella, Università di Salerno, Fisciano, Italy
Massimiliano Giordano, Università di Salerno, Fisciano, Italy
Giuseppe Polese, Università di Salerno, Fisciano, Italy
We present an approach and a system to let tutors monitor several important aspects related to online tests, such as learner behavior and test quality. The approach includes the logging of important data related to learner interaction with the system during the execution of online tests and exploits data visualization to highlight information useful to let tutors review and improve the whole assessment process. We have focused on the discovery of behavioral patterns of learners and conceptual relationships among test items. Furthermore, we have led several experiments in our faculty in order to assess the whole approach. In particular, by analyzing the data visualization charts, we have detected several previously unknown test strategies used by the learners. Last, we have detected several correlations among questions, which gave us useful feedbacks on the test quality.

[1] J. Bath, “Answer-Changing Behaviour on Objective Examinations,” J. Educational Research, no. 61, pp. 105-107, 1967.
[2] J.B. Best, “Item Difficulty and Answer Changing,” Teaching of Psychology, vol. 6, no. 4, pp. 228-240, 1979.
[3] J. Johnston, “Exam Taking Speed and Grades,” Teaching of Psychology, no. 4, pp. 148-149, 1977.
[4] C.A. Paul and J.S. Rosenkoetter, “The Relationship between the Time Taken to Complete an Examination and the Test Score Received,” Teaching of Psychology, no. 7, pp. 108-109, 1980.
[5] L. McClain, “Behavior During Examinations: A Comparison of ‘A,’ ‘C,’ and ‘F’ Students,” Teaching of Psychology, vol. 10, no. 2, pp.69-71, 1983.
[6] D. Hand, H. Mannila, and P. Smyth, Principles of Data Mining, Adaptive Computation and Machine Learning Series, A Bradford Book, MIT Press, 2001.
[7] N. Ye, “Introduction,” The Handbook of Data Mining. Lawrence Erlbaum Assoc., 2003.
[8] C. Plaisant and B. Shneiderman, “Show Me! Guidelines for Producing Recorded Demonstrations,” Proc. IEEE Symp. Visual Languages and Human-Centric Computing (VLHCC '05), pp. 171-178, 2005.
[9] R. Mazza and V. Dimitrova, “Student Tracking and Personalization: Visualising Student Tracking Data to Support Instructors in Web-Based Distance Education,” Proc. 13th Int'l World Wide Web Conf. Alternate Track Papers and Posters, pp. 154-161, 2004.
[10] R.S. Baker, A.T. Corbett, K.R. Koedinger, and A.Z. Wagner, “Off-Task Behavior in the Cognitive Tutor Classroom: When Students ‘Game the System’,” Proc. ACM SIGCHI Conf. Human Factors in Computing Systems (CHI '04), pp. 383-390, 2004.
[11] Asynchronous JavaScript Technology and XML (Ajax) with the Java Platform, J2EEAJAX/, 2007.
[12] G. Costagliola, F. Ferrucci, V. Fuccella, and F. Gioviale, “A Web Based Tool for Assessment and Self-Assessment,” Proc. Second Int'l Conf. Information Technology: Research and Education (ITRE '04), pp. 131-135, 2004.
[13] U. Demšar, “Data Mining of Geospatial Data: Combining Visual and Automatic Methods,” PhD dissertation, Dept. of Urban Planning and Environment, School of Architecture and the Built Environment, Royal Inst. of Technology (KTH), 2006.
[14] U. Fayyad and G. Grinstein, “Introduction,” Information Visualisation in Data Mining and Knowledge Discovery, Morgan Kaufmann, 2002.
[15] D.A. Keim and M. Ward, “Visualization,” Intelligent Data Analysis, M. Berthold and D.J. Hand, eds., second ed. Springer, 2003.
[16] D.A. Keim, Visual Exploration of Large Data Sets, second ed. Springer, 2003.
[17] G. Grinstein and M. Ward, “Introduction to Data Visualization,” Information Visualisation in Data Mining and Knowledge Discovery, Morgan Kaufmann, 2002.
[18] P.E. Hoffman and G.G. Grinstein, “A Survey of Visualizations for High-Dimensional Data Mining,” Information Visualization in Data Mining and Knowledge Discovery, pp. 47-82, 2002.
[19] M. Ankerst, “Visual Data Mining,” PhD dissertation, Ludwig Maximilians Universitat, Munchen, Germany, 2000.
[20] D.A. Keim, W. Müller, and H. Schumann, “Visual Data Mining,” STAR Proc. Eurographics '02, D. Fellner and R. Scopigno, eds., Sept. 2002.
[21] D.A. Keim, “Information Visualization and Visual Data Mining,” IEEE Trans. Visualization and Computer Graphics, vol. 8, no. 1, pp.1-8, Jan.-Mar. 2002.
[22] D.A. Keim, C. Panse, M. Sips, and S.C. North, “Pixel Based Visual Data Mining of Geo-Spatial Data,” Computers & Graphics, vol. 28, no. 3, pp. 327-344, 2004.
[23] K. Cox, S. Eick, and G. Wills, “Brief Application Description— Visual Data Mining: Recognizing Telephone Calling Fraud,” Data Mining and Knowledge Discovery, vol. 1, pp. 225-231, 1997.
[24] A. Inselberg, “Visualization and Data Mining of High-Dimensional Data,” Chemometrics and Intelligent Laboratory Systems, vol. 60, pp.147-159, 2002.
[25] L. Chittaro, C. Combi, and G. Trapasso, “Data Mining on Temporal Data: A Visual Approach and Its Clinical Application to Hemodialysis,” J. Visual Languages and Computing, vol. 14, pp.591-620, 2003.
[26] H. Miller and J. Han, “An Overview,” Geographic Data Mining and Knowledge Discovery, pp. 3-32, Taylor and Francis, 2001.
[27] I. Kopanakis and B. Theodoulidis, “Visual Data Mining Modeling Techniques for the Visualization of Mining Outcomes,” J. Visual Languages and Computing, no. 14, pp. 543-589, 2003.
[28] M. Kreuseler and H. Schumann, “A Flexible Approach for Visual Data Mining,” IEEE Trans. Visualization and Computer Graphics, vol. 8, no. 1, pp. 39-51, Jan.-Mar. 2002.
[29] G. Manco, C. Pizzuti, and D. Talia, “Eureka!: An Interactive and Visual Knowledge Discovery Tool,” J. Visual Languages and Computing, vol. 15, pp. 1-35, 2004.
[30] S. Kimani, S. Lodi, T. Catarci, G. Santucci, and C. Sartori, “Vidamine: A Visual Data Mining Environment,” J. Visual Languages and Computing, vol. 15, pp. 37-67, 2004.
[31] I. Kopanakis, N. Pelekis, H. Karanikas, and T. Mavroudkis, Visual Techniques for the Interpretation of Data Mining Outcomes, pp. 25-35. Springer, 2005.
[32] P. Buono and M. Costabile, “Visualizing Association Rules in a Framework for Visual Data Mining,” From Integrated Publication and Information Systems to Virtual Information and Knowledge Environments, Essays Dedicated to Erich J. Neuhold on the Occasion of His 65th Birthday, pp. 221-231, Springer, 2005.
[33] U. Fayyad, G. Piatetsky-Shapiro, and P. Smyth, “From Data Mining to Knowledge Discovery in Databases,” AI Magazine, pp.37-54, 1996.
[34] M.C. Chen, J.R. Anderson, and M.H. Sohn, “What Can a Mouse Cursor Tell Us More?: Correlation of Eye/Mouse Movements on Web Browsing,” Proc. CHI '01 Extended Abstracts on Human Factors in Computing Systems, pp. 281-282, 2001.
[35] Hibernate, Hibernate Framework, http:/, 2007.
[36] ECMAScript, Ecmascript Language Specification, files/ECMA-STEcma-262.pdf, 2008.
[37] Xquery 1.0: An XML Query Language, World Wide Web Consortium (W3C) Recommendation,, Jan. 2007.
[38] Xquery API for JAVATM (XQJ),, Nov. 2007.
[39] M. Dick, J. Sheard, C. Bareiss, J. Carter, D. Joyce, T. Harding, and C. Laxer, “Addressing Student Cheating: Definitions and Solutions,” SIGCSE Bull., vol. 35, no. 2, pp. 172-184, 2003.
[40] T.S. Harding, D.D. Carpenter, S.M. Montgomery, and N. Steneck, “The Current State of Research on Academic Dishonesty among Engineering Students,” Proc. 31st Ann. Frontiers in Education Conf. (FIE '01), vol. 3, pp. 13-18, 2001.
[41] S. Mulvenon, R.C. Turner, and S. Thomas, “Techniques for Detection of Cheating on Standardized Tests Using SAS,” Proc. 26th Ann. SAS Users Group Int'l Conf. (SUGI '01), pp. 1-6, 2001.
[42] H. Shao, H. Zhao, and G.-R. Chang, “Applying Data Mining to Detect Fraud Behavior in Customs Declaration,” Proc. Int'l Conf. Machine Learning and Cybernetics (ICMLC '02), vol. 3, pp. 1241-1244, 2002.
[43] M. May, S. George, and P. Prévôt, “Tracking, Analyzing and Visualizing Learners' Activities on Discussion Forums,” Proc. Sixth IASTED Int'l Conf. Web Based Education (WBE '07), vol. 2, pp.649-656, 2007.
[44] T. Mochizuki, H. Kato, K. Yaegashi, T. Nagata, T. Nishimori, S. Hisamatsu, S. Fujitani, J. Nakahara, and M. Suzuki, “Promotion of Self-Assessment for Learners in Online Discussion Using the Visualization Software,” Proc. Conf. Computer Support for Collaborative Learning (CSCL '05), pp. 440-449, 2005.
[45] J. Hardy, S. Bates, J. Hill, and M. Antonioletti, “Tracking and Visualization of Student Use of Online Learning Materials in a Large Undergraduate Course,” Proc. Sixth Int'l Conf. Web-Based Learning (ICWL '07), pp. 280-287, 2007.
[46] M. Sasakura and S. Yamasaki, “A Framework for Adaptive E-Learning Systems in Higher Education with Information Visualization,” Proc. 11th Int'l Conf. Information Visualization (IV'07), pp. 819-824, 2007.
[47] G.K.L. Tam, R.W.H. Lau, and J. Zhao, “A 3D Geometry Search Engine in Support of Learning,” Proc. Sixth Int'l Conf. Web-Based Learning (ICWL '07), pp. 248-255, 2007.
[48] Q.V. Nguyen, M.L. Huang, and I. Hawryszkiewycz, “A New Visualization Approach for Supporting Knowledge Management and Collaboration in e-Learning,” Proc. Eighth Int'l Conf. Information Visualisation (IV '04), pp. 693-700, 2004.
[49] C.G. da Silva and H. da Rocha, “Learning Management Systems' Database Exploration by Means of Information Visualization-Based Query Tools,” Proc. Seventh IEEE Int'l Conf. Advanced Learning Technologies (ICALT '07), pp. 543-545, 2007.

Index Terms:
Distance learning, data visualization, interactive data exploration and knowledge discovery.
Gennaro Costagliola, Vittorio Fuccella, Massimiliano Giordano, Giuseppe Polese, "Monitoring Online Tests through Data Visualization," IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 6, pp. 773-784, June 2009, doi:10.1109/TKDE.2008.133
Usage of this product signifies your acceptance of the Terms of Use.