The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan.-March (2012 vol.5)
pp: 2-10
A. Sanna , Dipt. di Autom. e Inf., Politec. di Torino, Torino, Italy
F. Lamberti , Dipt. di Autom. e Inf., Politec. di Torino, Torino, Italy
G. Paravati , Dipt. di Autom. e Inf., Politec. di Torino, Torino, Italy
C. Demartini , Dipt. di Autom. e Inf., Politec. di Torino, Torino, Italy
ABSTRACT
Computer-based assessment of exams provides teachers and students with two main benefits: fairness and effectiveness in the evaluation process. This paper proposes a fully automatic evaluation tool for the Graphic and Virtual Design (GVD) curriculum at the First School of Architecture of the Politecnico di Torino, Italy. In particular, the tool is designed for the 3D modeling course, taught during the second year, where students have to prove their ability to model static scenes using the open source modeler Blender. During the final exam, students are required to create a 3D model as similar as possible to a reference object proposed by the teacher and shown through a set of 2D views; the similarity of the images is judged according to both model shape and materials. The traditional assessment process is particularly slow and strongly based on teachers subjective evaluation; the proposed solution efficiently implements an objective assessment mechanism that exploits computer vision and image analysis algorithms to automatically extract similarity indices. These indices are related to partial evaluation grades, which are then combined to obtain the final mark. A comparison with the traditional assessment process shows robustness and trustworthiness of the designed approach.
INDEX TERMS
solid modelling, computer aided instruction, computer science education, computer vision, educational courses, educational institutions, further education, public domain software, shape similarity criteria, 3D modeling exam assessment, computer-based exam assessment, evaluation process effectiveness, evaluation process fairness, automatic evaluation tool, graphic and virtual design curriculum, First School of Architecture, Politecnico di Torino, Italy, 3D modeling course, static scene model, Blender open source modeler, objective assessment mechanism, computer vision, image analysis algorithm, similarity index extraction, partial evaluation grade, material similarity criteria, Solid modeling, Three dimensional displays, Materials, Education, Computer graphics, Computational modeling, Shape, computer graphics, solid modelling, computer aided instruction, computer science education, computer vision, educational courses, educational institutions, further education, public domain software, shape similarity criteria, 3D modeling exam assessment, computer-based exam assessment, evaluation process effectiveness, evaluation process fairness, automatic evaluation tool, graphic and virtual design curriculum, First School of Architecture, Politecnico di Torino, Italy, 3D modeling course, static scene model, Blender open source modeler, objective assessment mechanism, computer vision, image analysis algorithm, similarity index extraction, partial evaluation grade, material similarity criteria, Solid modeling, Three dimensional displays, Materials, Education, Computer graphics, Computational modeling, Shape, 3D modeling., Evaluation methodologies, teaching and learning strategies, higher education
CITATION
A. Sanna, F. Lamberti, G. Paravati, C. Demartini, "Automatic Assessment of 3D Modeling Exams", IEEE Transactions on Learning Technologies, vol.5, no. 1, pp. 2-10, Jan.-March 2012, doi:10.1109/TLT.2011.4
REFERENCES
[1] European Commission, "European Credit Transfer and Accumulation System (ECTS)," http://ec.europa.eu/education/lifelong-learning-policy doc48_en.htm, July 2006.
[2] J.D. Foley, S.K. Feiner, and J.F. Hughes, Computer Graphics: Principles and Practice. Addison-Wesley, 1990.
[3] Blender Website, http:/www.blender.org, July 2010.
[4] "Shared 'Dublin' Descriptors for Short Cycle, First Cycle, Second Cycle and Third Cycle Awards," working document, Joint Quality Initiative Informal Group, Oct. 2004.
[5] A.I. González-Tablas Ferreres, K. Wouters, B. Ramos Alvarez, and A. Ribagorda Garnacho, "EVAWEB: A Web-Based Assessment System to Learn X.509/PKIX-Based Digital Signatures," IEEE Trans. Education, vol. 50, no. 2, pp. 112-117, May 2007.
[6] M.S. Pérez, P. Herrero, F.M. Sánchez, and V. Robles, "Are Web Self-Assessment Tools Useful for Training?" IEEE Trans. Education, vol. 48, no. 4, pp. 757-763, Nov. 2005.
[7] A. Tartaglia and E. Tresso, "An Automatic Evaluation System for Technical Education at the University Level," IEEE Trans. Education, vol. 45, no. 3, pp. 268-274, Aug. 2002.
[8] G. Conole and B. Warburton, "A Review of Computer Assisted Assessment," ALT-J Research in Learning Technology, vol. 13, no. 2, pp. 17-31, Mar. 2005.
[9] P. Lira, M. Bronfman, and J. Eyzaguirre, "MULTITEST II: A Program for the Generation, Correction, and Analysis of Multiple Choice Tests," IEEE Trans. Education, vol. 33, no. 2, pp. 320-325, Nov. 1990.
[10] G.J. Hwang, "A Test-Sheet-Generating Algorithm for Multiple Assessment Requirements," IEEE Trans. Education, vol. 56, no. 2, pp. 329-337, Aug. 2003.
[11] H. Wainer, Computerized Adaptive Testing: A Primer. Hillsdale, 1990.
[12] E. Guzmán and R. Conejo, "Self-Assessment in a Feasible, Adaptive Web-Based Testing System," IEEE Trans. Education, vol. 48, no. 2, pp. 688-695, Nov. 2005.
[13] J. Whittington and K.J. Nankivell, "Teaching Strategies and Assessment Measures for Rapidly Changing Technology Programs," Proc. Int'l Conf. Computer Graphics and Interactive Techniques, 2006.
[14] M. Al-Smadi, C. Guetl, and D. Helic, "Towards a Standardized E-Assessment System: Motivations, Challenges and First Findings," Int'l J. Emerging Technologies in Learning, vol. 4, no. 2, pp. 6-12, 2009.
[15] L. Indrayanti, T. Usagawa, Y. Chisaki, and T. Dutono, "Evaluation of Pronunciation by Means of Automatic Speech Recognition System for Computer Aided Indonesian Language Learning," Proc. Seventh Int'l Conf. Information Technology Based Higher Education and Training, pp. 553-556, 2006.
[16] H.C. Wang, C.Y. Chang, and T.Y. Li, "Assessing Creative Problem-Solving with Automated Text Grading," Computers and Education, vol. 51, pp. 1450-1466, 2008.
[17] T. Kakkonen and E. Sutinen, "Automatic Assessment of the Content of Essays Based on Course Material," Proc. Second Int'l Conf. Information Technology, pp. 126-130, 2004.
[18] C. Douce, D. Livingstone, and J. Orwell, "Automatic Test-Based Assessment of Programming: A Review," J. Educational Resources in Computing, vol. 5, no. 3, pp. 1-13, 2005.
[19] K.A. Ala-Mutkaa, "A Survey of Automated Assessment Approaches for Programming Assigments," Computer Science Education, vol. 15, no. 2, pp. 83-102, June 2005.
[20] Z. Shukur and N.F. Mohamed, "The Design of ADAT: A Tool for Assessing Automata-Based Assigments," J. Computer Science, vol. 4, no. 5, pp. 415-420, 2008.
[21] Z. Shukur, E. Burke, and E. Foxley, "The Automatic Assessment of Formal Specification Coursework," J. Computing in Higher Education, vol. 11, no. 1, pp. 86-119, 1999.
[22] P. Thomas, K. Waugh, and N. Smith, "Using Patterns in the Automatic Marking of ER Diagrams," Proc. 11th Ann. Conf. Innovation and Technology in Computer Science Education, pp. 83-87, 2006.
[23] H. Ke, G. Zhang, and H. Yan, "Automatic Grading System on SQL Programming," Proc. Int'l Conf. Scalable Computing and Comm.; Eighth Int'l Conf. Embedded Computing, pp. 537-540, 2009.
[24] E. Gutiérrez, M.A. Trenas, J. Ramos, F. Corbera, and S. Romero, "A New Moodle Module Supporting Automatic Verification of VHDL-Based Assigments," Computers and Education, vol. 54, pp. 562-577, 2010.
[25] S. Isotani and L. de Oliverira Brandao, "An Algorithm for Automatic Checking of Exercises in a Dynamic Geometry System: iGeom," Computers and Education, vol. 51, pp. 1283-1303, 2008.
[26] V. Petridis, S. Kazarlis, and V.G. Kabrlasos, "ACES: An Interactive Software Platform for Self-Instruction and Self-Evaluation in Automatic Control Systems," IEEE Trans. Education, vol. 46, no. 2, pp. 102-110, Feb. 2003.
[27] Z. Doulgeri and T. Matiakis, "A Web Telerobotic System to Teach Industrial Robot Planning and Control," IEEE Trans. Education, vol. 29, no. 2, pp. 263-270, May 2006.
[28] L.C. Jacobs and C.I. Chase, Developing and Using Tests Effectively: A Guide for Faculty. Jossey-Bass, 1992.
[29] W.K. Chen and Y.C. Cheng, "Teaching Object-Oriented Programming Laboratory with Computer Game Programming," IEEE Trans. Education, vol. 50, no. 2, pp. 197-203, Aug. 2007.
[30] M.D. Koretsky, D. Amatore, C. Barnes, and S. Kimura, "Enhancement of Student Learning in Experimental Design Using a Virtual Laboratory," IEEE Trans. Education, vol. 51, no. 2, pp. 76-85, Feb. 2008.
[31] H. Delingette, X. Pennec, L. Soler, J. Marescaux, and N. Ayache, "Computational Models for Image-Guided Robot-Assisted and Simulated Medical Interventions," Proc. IEEE, vol. 94, no. 2, pp. 1678-1688, Sept. 2006.
[32] A. Johnson, T. Moher, Y.J. Cho, Y.J. Lin, D. Haas, and J. Kim, "Augmenting Elementary School Education with VR," IEEE Computer Graphics and Applications, vol. 22, no. 2, pp. 6-9, Mar./Apr. 2002.
[33] A.C. Graesser, P. Chipman, B.C. Haynes, and A.O.A., "AutoTutor: An Intelligent Tutoring System with Mixed-Initiative Dialogue," IEEE Trans. Education, vol. 48, no. 2, pp. 612-618, Nov. 2005.
[34] P. Fournier-Viger, R. Nkambou, and A. Mayers, "Evaluating Spatial Representations and Skills in a Simulator-Based Tutoring System," IEEE Trans. Learning Technologies, vol. 1, no. 2, pp. 63-74, Jan.-Mar. 2008.
[35] I. Gustavsson, K. Nilsson, J. Zackrisson, J. Garcia-Zubia, U. Hernandez-Jayo, A. Nafalski, Z. Nedic, Ö. Goöl, J. Machotka, M.I. Pettersson, T. Lagö, and L. Håkansson, "On Objectives of Instructional Laboratories, Individual Assessment, and Use of Collaborative Remote Laboratories," IEEE Trans. Learning Technologies, vol. 2, no. 2, pp. 263-274, Oct.-Dec 2009.
[36] H.Q. Jiang, L. Zhang, and W.Q. Ye, "The Automatic Evaluation Strategies and Methods of Multimedia Work Assigments," Proc. Int'l Conf. Computational Intelligence and Software Eng., pp. 1-5, 2009.
[37] T.J. Branoff, "Constraint-Based Modeling in the Engineering Graphics Curriculum: Laboratory Activities and Evaluation Strategies," Proc. Midyear Conf. Eng. Design Graphics Division of the Am. Soc. for Eng. Education, pp. 132-138, 2004.
[38] J.L. Colwell, J. Whittington, and J. Higley, "Assessment Measures and Outcomes for Computer Graphics Programs," Eng. Design Graphics J., vol. 69, no. 3, pp. 24-33, 2005.
[39] D.H. Baxter and M.J. Guerci, "Automating an Introductory Computer Aided Design Course to Improve Student Evaluation," Proc. Am. Soc. for Eng. Education Ann. Conf., 2003.
[40] D.H. Baxter, "Evaluating an Automatic Grading System for an Introductory Computer Aided Design Course," Proc. 58th Ann. Eng. Design Graphics Midyear Meeting, pp. 39-44, 2003.
[41] D.H. Baxter, "Evaluating Performance in a Freshman Graphics Course to Provide Early Intervention for Students with Visualization and/or Design Intent Difficulties," Proc. Am. Soc. for Eng. Education Ann. Conf., 2002.
[42] Z. Shukur, Y. Away, and M.A. Dawari, "Computer-Aided Marking System for Engineering Drawing," Proc. Soc. for Information Technology and Teacher Education Int'l Conf., pp. 1852-1857, 2004.
[43] D.Y. Chen, X.P. Tian, Y.T. Shen, and M. Ouhyoung, "On Visual Similarity Based 3D Model Retrieval," Computer Graphics Forum, vol. 22, no. 2, pp. 223-232, Sept. 2003.
[44] A.R. Smith, "Color Gamut Transform Pairs," Proc. Fifth Ann. Conf. Computer Graphics and Interactive Techniques, pp. 12-19, 1978.
15 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool