This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
ASCII Text
x 
 
Elizabeth J. Brown, Timothy J. Brailsford, Tony Fisher, Adam Moore, "Evaluating Learning Style Personalization in Adaptive Systems: Quantitative Methods and Approaches," IEEE Transactions on Learning Technologies, vol. 2, no. 1, pp. 10-22, January-March, 2009.
 
 
BibTex
x
 
@article{ 10.1109/TLT.2009.11,
author = {Elizabeth J. Brown and Timothy J. Brailsford and Tony Fisher and Adam Moore},
title = {Evaluating Learning Style Personalization in Adaptive Systems: Quantitative Methods and Approaches},
journal ={IEEE Transactions on Learning Technologies},
volume = {2},
number = {1},
issn = {1939-1382},
year = {2009},
pages = {10-22},
doi = {http://doi.ieeecomputersociety.org/10.1109/TLT.2009.11},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Learning Technologies
TI - Evaluating Learning Style Personalization in Adaptive Systems: Quantitative Methods and Approaches
IS - 1
SN - 1939-1382
SP10
EP22
EPD - 10-22
A1 - Elizabeth J. Brown,
A1 - Timothy J. Brailsford,
A1 - Tony Fisher,
A1 - Adam Moore,
PY - 2009
KW - Evaluation/methodology
KW - Computer-assisted instruction
KW - Human information processing
KW - Adaptive hypermedia
KW - Miscellaneous
KW - Artificial Intelligence
KW - Computing Methodologies
KW - User issues
KW - Hypertext/Hypermedia
KW - Information Interfaces and Representation (HCI)
KW - Information Technologies
VL - 2
JA - IEEE Transactions on Learning Technologies
ER -
 
Elizabeth J. Brown, University of Nottingham, Nottingham
Timothy J. Brailsford, University of Nottingham, Nottingham
Tony Fisher, University of Nottingham, Nottingham
Adam Moore, University of Nottingham, Nottingham
It is a widely held assumption that learning style is a useful model for quantifying user characteristics for effective personalized learning. We set out to challenge this assumption by discussing the current state of the art, in relation to quantitative evaluations of such systems and also the methodologies that should be employed in such evaluations. We present two case studies that provide rigorous and quantitative evaluations of learning-style-adapted e-learning environments. We believe that the null results of both these studies indicate a limited usefulness in terms of learning styles for user modeling and suggest that alternative characteristics or techniques might provide a more beneficial experience to users.

[1] S.E. Ainsworth, “DeFT: A Conceptual Framework for Learning with Multiple Representations,” Learning and Instruction, vol. 16, pp.183-198, 2006.
[2] N. Bajraktarevic, W. Hall, and P. Fullick, “ILASH: Incorporating Learning Strategies in Hypermedia,” Proc. 14th ACM Conf. Hypertext and Hypermedia (HT '03), 2003.
[3] N. Bajraktarevic, W. Hall, and P. Fullick, “Incorporating Learning Styles in Hypermedia Environment: Empirical Evaluation,” Proc. 14th ACM Conf. Hypertext and Hypermedia (HT '03), 2003.
[4] I. Beaumont, “User Modelling in the Interactive Anatomy Tutoring System Anatom-Tutor,” User Modeling and User-Adapted Interaction, vol. 4, pp.21-45, 1994.
[5] C. Boyle and A.O. Encarnacion, “Metadoc: An Adaptive Hypertext Reading System,” User Modeling and User-Adapted Interaction, vol. 4, pp. 1-19, 1994.
[6] E. Brown, T. Brailsford, T. Fisher, and C. van der Eijk, “Revealing the Hidden Rationality of User Browsing Behaviour,” Proc. 18th ACM Conf. Hypertext and Hypermedia (Hypertext '07), 2007.
[7] E. Brown, T. Brailsford, T. Fisher, A. Moore, and H. Ashman, “Reappraising Cognitive Styles in Adaptive Web Applications,” Proc. 15th Int'l World Wide Web Conf. (WWW '06), 2006.
[8] E. Brown, T. Fisher, and T. Brailsford, “Real Users, Real Results: Examining the Limitations of Learning Styles within AEH,” Proc. 18th ACM Conf. Hypertext and Hypermedia (Hypertext '07), 2007.
[9] P. Brusilovsky, “Adaptive Hypermedia,” User Modeling and User Adapted Interaction, Ten Year Anniversary Issue, A. Kobsa, ed., vol. 11, pp.87-110, 2001.
[10] P. Brusilovsky, “Efficient Techniques for Adaptive Hypermedia,” Intelligent Hypertext, pp.12-30, Springer, 1997.
[11] P. Brusilovsky, “Intelligent Tutoring Systems for World-Wide Web,” Proc. Third Int'l World Wide Web Conf. (WWW '95), 1995.
[12] P. Brusilovsky, “Methods and Techniques of Adaptive Hypermedia,” User Modeling and User Adapted Interaction, vol. 6, pp.87-129, 1996.
[13] P. Brusilovsky and J. Eklund, “A Study of User Model Based Link Annotation in Educational Hypermedia,” J. Universal Computer Science, vol. 4, pp.429-448, 1998.
[14] P. Brusilovsky, C. Karagiannidis, and D. Sampson, “The Benefits of Layered Evaluation of Adaptive Applications and Services,” Proc. Workshop Empirical Evaluations of Adaptive Systems at the Eighth Int'l Conf. User Modeling (UM '01), 2001.
[15] P. Brusilovsky and L. Pesin, “ISIS-Tutor: An Adaptive Hypertext Learning Environment,” Proc. Japan-CIS Symp. Knowledge-Based Software Eng. (JCKBSE '94), 1994.
[16] P. Brusilovsky and L. Pesin, “Visual Annotation of Links in Adaptive Hypermedia,” Proc. Conf. Human Factors in Computing Systems (CHI '95), 1995.
[17] P. Brusilovsky, L. Pesin, and M. Zyryanov, “Towards an Adaptive Hypermedia Component for an Intelligent Learning Environment,” Human-Computer Interaction, L.J. Bass, J. Gornostaev, and C. Unger, eds., pp.348-358, Springer-Verlag, 1993.
[18] P. Brusilovsky, E. Schwarz, and G. Weber, “Elm-Art: An Intelligent Tutoring System on World Wide Web,” Intelligent Tutoring Systems, C. Frasson, G. Gauthier, and A. Lesgold, eds., pp.261-269, Springer Verlag, 1996.
[19] P. Brusilovsky, E. Schwarz, and G. Weber, “A Tool for Developing Adaptive Electronic Textbooks on WWW,” Proc. World Conf. Web Soc. (WebNet '96), 1996.
[20] R.M. Carro, E. Pulido, and P. Rodríguez, “TANGOW: Task-Based Adaptive Learner Guidance on the WWW,” Proc. Second Workshop Adaptive Systems and User Modeling on the World Wide Web, 1999.
[21] C.A. Carver, R.A. Howard, and W.D. Lane, “Addressing Different Learning Styles through Course Hypermedia,” IEEE Trans. Education, vol. 42, pp.33-38, 1999.
[22] D.N. Chin, “Empirical Evaluation of User Models and User-Adapted Systems,” User Modeling and User-Adapted Interaction, vol. 11, pp.181-194, 2001.
[23] F. Coffield, D. Moseley, E. Hall, and K. Ecclestone, Learning Styles and Pedagogy in Post-16 Learning: A Systematic and Critical Review. Learning & Skills Research Centre, 2004.
[24] F. Coffield, D. Moseley, E. Hall, and K. Ecclestone, Should We Be Using Learning Styles? What Research Has to Say to Practice. Learning & Skills Research Centre, 2004.
[25] O. Conlan, D. Dagger, and V. Wade, “Towards a Standards-Based Approach to E-Learning Personalization Using Reusable Learning Objects,” Proc. World Conf. E-Learning in Corporate, Gov't, Healthcare, and Higher Education (E-Learn '02), 2002.
[26] O. Conlan, V. Wade, C. Bruen, and M. Gargan, “Multi-Model, Metadata Driven Approach to Adaptive Hypermedia Services for Personalized Elearning,” Proc. Second Int'l Conf. Adaptive Hypermedia and Adaptive Web-Based Systems, 2002.
[27] C.E. Cornett, What You Should Know About Learning Styles. Phi Delta Kappa, 1983.
[28] D. Dagger, O. Conlan, and V. Wade, “Towards ‘Anytime, Anywhere’ Learning: The Role and Realization of Dynamic Terminal Personalization in Adaptive Elearning,” Proc. 15th World Conf. Educational Multimedia, Hypermedia, and Telecomm. (ED-MEDIA 2003), 2003.
[29] P. De Bra and L. Calvi, “Aha! An Open Adaptive Hypermedia Architecture,” New Rev. Hypermedia and Multimedia, vol. 4, pp.115-139, 1998.
[30] J. Debes, “Some Foundations of Visual Literacy,” Audio Visual Instruction, vol. 13, pp.961-964, 1968.
[31] W. Doise, G. Mugny, and A.-N. Perret-Clermont, “Social Interaction and the Development of Cognitive Operations,” European J. Social Psychology, vol. 5, pp.367-383, 1975.
[32] R. Dunn, “The Dunn and Dunn Learning Style Model and Its Theoretical Cornerstone,” Synthesis of the Dunn and Dunn Learning Styles Model Research: Who, What, When, Where and So What—The Dunn and Dunn Learning Styles Model and Its Theoretical Cornerstone, R. Dunn and S. Griggs, eds., St John's Univ., 2003.
[33] R.M. Felder and B.A. Soloman, “Index of Learning Styles,” http://www.ncsu.edu/felder-publicILSpage.html , 2009.
[34] S. Ferrandino, A. Negro, and V. Scarano, “Cheops: Adaptive Hypermedia on the World Wide Web,” Proc. European Workshop Interactive Distributed Multimedia Systems and Telecomm. Services (IDMS '97), 1997.
[35] A.W.P. Fok and H.H.S. Ip, “Personalized Education: An Exploratory Study of Learning Pedagogies in Relation to Personalization Technologies,” Advances in Web-Based Learning: ICWL 2004, W. Liu, L. Qing, and Y. Shi, eds., pp.407-415, Springer-Verlag, 2004.
[36] P. Germanakos, N. Tsianos, Z. Lekkas, C. Mourlas, M. Belk, and G. Samaras, “A Semantic Approach of an Adaptive and Personalized Web-Based Learning Content—The Case of Adaptive Web,” Proc. Second Int'l Workshop Semantic Media Adaptation and Personalization (SMAP '07), 2007.
[37] J.E. Gilbert and C.Y. Han, “Adapting Instruction in Search of ‘A Significant Difference,’” J. Network and Computer Applications, vol. 22, pp.149-160, 1999.
[38] R. Gillespie, Manufacturing Knowledge: A History of the Hawthorne Experiments. Cambridge Univ. Press, 1991.
[39] C. Giorgis, N.J. Johnson, A. Bonomo, C. Colbert, A. Conner, G. Kauffman, and D. Kulesza, “Visual Literacy,” Reading Teacher, vol. 53, pp.146-153, 1999.
[40] M. Grigoriadou, K. Papanikolaou, H. Kornilakis, and G. Magoulas, “Inspire: An Intelligent System for Personalized Instruction in a Remote Environment,” Proc. Eighth Int'l Conf. User Modeling (UM'01), 2001.
[41] D. Hargreaves, About Learning: Report of the Learning Working Group. DEMOS, 1995.
[42] R.C. Hilborn, “Sea Gulls, Butterflies, and Grasshoppers: A Brief History of the Butterfly Effect in Nonlinear Dynamics,” Am. J. Physics, vol. 72, pp.425-427, 2004.
[43] H. Hohl, H.-D. Böcker, and R. Gunzenhäuser, “Hypadapter: An Adaptive Hypertext System for Exploratory Learning and Programming,” User Models and User Adapted Interaction, vol. 6, pp.131-156, 1996.
[44] P. Honey and A. Mumford, The Manual of Learning Styles. Peter Honey Publications, 1992.
[45] J. Kay and B. Kummerfeld, “Adaptive Hypertext for Individualised Instruction,” Proc. Workshop Adaptive Hypertext and Hypermedia at the Fourth Int'l Conf. User Modeling (UM '94), 1994.
[46] J.W. Keefe, “Learning Style: An Overview,” Student Learning Styles: Diagnosing and Prescribing Programs, J.W. Keefe, ed., NASSP, 1979.
[47] D. Kelly and B. Tangney, “Adapting to Intelligence Profile in an Adaptive Educational System,” Interacting with Computers, vol. 18, pp.385-409, 2006.
[48] M. Kozhevnikov, M. Hegarty, and R.E. Mayer, “Revising the Visualiser-Verbalizer Dimension: Evidence for Two Types of Visualizers,” Cognition and Instruction, vol. 20, pp.47-77, 2002.
[49] G. Kress, Literacy in the New Media Age. Routledge, 2003.
[50] E. Melis and R. Monthienvichienchai, “They Call It Learning Style but It's So Much More,” Proc. World Conf. E-Learning in Corporate, Gov't, Healthcare, and Higher Education (E-Learn '04), 2004.
[51] D.M. Mertens, Research Methods in Education and Psychology: Integrating Diversity with Quantitative and Qualitative Approaches. Sage, 1998.
[52] A. Micarelli and F. Sciarrone, “A Case-Based Toolbox for Guided Hypermedia Navigation,” Proc. Fifth Int'l Conf. User Modeling (UM'96), 1996.
[53] R. Moreno and R.E. Mayer, “Cognitive Principles of Multimedia Learning: The Role of Modality and Contiguity,” J. Educational Psychology, vol. 91, pp.358-368, 1999.
[54] K. Morrison, School Leadership and Complexity Theory. RoutledgeFalmer, 2002.
[55] J.O. Nunnally, Psychometric Theory. McGraw-Hill, 1978.
[56] D.W. Oard and J. Kim, “Implicit Feedback for Recommender Systems,” Proc. AAAI Workshop Recommender Systems, 1998.
[57] A. Ortigosa and R.M. Carro, “The Continuous Empirical Evaluation Approach: Evaluating Adaptive Web-Based Courses,” User Modeling 2003, pp.163-168, Springer, 2003.
[58] A. Paivio, Mental Representations. Oxford Univ. Press, 1986.
[59] J. Pallant, SPSS Survival Manual: A Step by Step Guide to Data Analysis Using SPSS for Windows (Version 12), second ed., Open Univ. Press (McGraw-Hill Education), 2005.
[60] K.A. Papanikolaou, M. Grigoriadou, H. Kornilakis, and G.D. Magoulas, “Personalizing the Interaction in a Web-Based Educational Hypermedia System: The Case of Inspire,” User Modeling and User-Adapted Interaction, vol. 13, pp.213-267, 2003.
[61] K.A. Papanikolaou, A. Mabbott, S. Bull, and M. Grigoriadou, “Designing Learner-Controlled Educational Interactions Based on Learning/Cognitive Style and Learner Behaviour,” Interacting with Computers, vol. 18, pp.356-384, 2006.
[62] P. Paredes and P. Rodriguez, “A Mixed Approach to Modelling Learning Styles in Adaptive Educational Hypermedia,” Proc. Third IASTED Conf. Web-Based Education (WBE '04), 2004.
[63] T. Pérez, P. Lopistéguy, J. Gutiérrez, and I. Usandizaga, “Hypertutor: From Hypermedia to Intelligent Adaptive Hypermedia,” Proc. Seventh World Conf. Educational Multimedia and Hypermedia (ED-MEDIA '95), 1995.
[64] J.K. Pinto, M.A. Geiger, and E.J. Boyle, “A Three Year Longitudinal Study of Changes in Student Learning Styles,” J. College Student Development, vol. 35, pp.113-119, 1994.
[65] J.L. Plass, D.M. Chun, R.E. Mayer, and D. Leutner, “Supporting Visual and Verbal Learning Preferences in a Second-Language Multimedia Learning Environment,” J. Educational Psychology, vol. 90, pp.25-36, 1998.
[66] C. Plua and A. Jameson, “Collaborative Preference Elicitation in a Group Travel Recommender System,” Proc. AH 2002 Workshop Recommendation and Personalization in eCommerce, 2002.
[67] R.J. Riding, School Learning and Cognitive Style. David Fulton, 2002.
[68] H. Rittel and M. Webber, Dilemmas in a General Theory of Planning, vol. 4. Elsevier Scientific Publishing Company, Inc., 1973.
[69] C. Robson, Real World Research: A Resource for Social Scientists and Practitioner-Researchers. Blackwell Publishing, 2002.
[70] G. Salomon and D.N. Perkins, “Individual and Social Aspects of Learning,” Rev. Research in Education, vol. 23, pp.1-24, 1998.
[71] M. Scaife and Y. Rogers, “External Cognition: How Do Graphical Representations Work?” Int'l J. Human-Computer Studies, vol. 45, pp.185-213, 1996.
[72] J.B. Schafer, J. Konstan, and J. Riedi, “Recommender Systems in E-Commerce,” Proc. First ACM Conf. Electronic Commerce, 1999.
[73] C. Stangor, Research Methods for the Behavioral Sciences. Houghton Mifflin, 1998.
[74] N. Stash, A. Cristea, and P. De Bra, “Adaptation to Learning Styles in E-Learning: Approach Evaluation,” Proc. World Conf. E-Learning in Corporate, Gov't, Healthcare & Higher Education (E-Learn '06), 2006.
[75] N. Stash, A. Cristea, and P. De Bra, “Learning Styles Adaptation Language for Adaptive Hypermedia,” Adaptive Hypermedia and Adaptive Web-Based Systems, pp.323-327, Springer, 2006.
[76] N. Stash, A. Cristea, and P. De Bra, “Authoring of Learning Styles in Adaptive Hypermedia: Problems and Solutions,” Proc. 13th Int'l World Wide Web Conf., 2004.
[77] M.K. Stern and B.P. Woolf, “Adaptive Content in an Online Lecture System,” Proc. Int'l Conf. Adaptive Hypermedia and Adaptive Web-Based Systems, 2000.
[78] J. Summerville, “Role of Awareness of Cognitive Style in Hypermedia,” Int'l J. Educational Technology, vol. 1, 1999.
[79] E. Triantafillou, A. Pomportsis, and S. Demetriadis, “The Design and the Formative Evaluation of an Adaptive Educational System Based on Cognitive Styles,” Computers and Education, vol. 41, pp.87-103, 2003.
[80] E. Triantafillou, A. Pomportsis, and E. Georgiadou, “AES-CS: Adaptive Educational System Based on Cognitive Styles,” Proc. Second Int'l Conf. Adaptive Hypermedia and Adaptive Web-Based Systems (AH '02), 2002.
[81] S.P. Tyerman, P.J. Woods, and J.R. Warren, “Loop Tutor and Hypertutor: Experiences with Adaptive Tutoring Systems,” Proc. Australian and New Zealand Conf. Intelligent Information Systems, 1996.
[82] D. Wang, Y. Bao, G. Yu, and G. Wang, “Using Page Classification and Association Rule Mining for Personalized Recommendation in Distance Learning,” Advances in Web-Based Learning: ICWL 2002, J. Fong, C.T. Cheung, H.V. Leong, and Q. Li, eds., pp.363-377, Springer-Verlag, 2002.
[83] A. Weber, “The Hawthorne Works,” Assembly Magazine, 2002.
[84] G. Weber, H.-C. Kuhl, and S. Weibelzahl, “Developing Adaptive Internet Based Courses with the Authoring System Netcoach,” Hypermedia: Openness, Structural Awareness, and Adaptivity, S. Reich, M. Tzagarakis, and P. De Bra, eds., pp.226-238, Springer, 2001.
[85] G. Weber and M. Specht, “User Modeling and Adaptive Navigation Support in WWW-Based Tutoring Systems,” Proc. Sixth Int'l Conf. User Modeling, 1997.
[86] S. Weibelzahl, “Problems and Pitfalls in the Evaluation of Adaptive Systems,” Adaptable and Adaptive Hypermedia Systems, S.Y. Chen and G.D. Magoulas, eds., pp.285-299, IRM Press, 2005.
[87] S. Weibelzahl, S. Lippitsch, and G. Weber, “Advantages, Opportunities, and Limits of Empirical Evaluations: Evaluating Adaptive Systems,” Künstliche Intelligenz, vol. 3, pp.17-20, 2002.
[88] S. Weibelzahl and G. Weber, “Evaluating the Inference Mechanism of Adaptive Learning Systems,” User Modeling 2003, p. 148, Springer, 2003.
[89] H.A. Witkin and D.R. Goodenough, Cognitive Style—Essence and Origins: Field Dependence and Field Independence. Int'l Univ., 1981.
[90] C. Wolf, “Construction of an Adaptive E-Learning Environment to Address Learning Styles and an Investigation of the Effect of Media Choice,” School of Education, p. 225, RMIT Univ., 2007.
[91] M.R. Zakaria, A. Moore, H. Ashman, C. Stewart, and T. Brailsford, “The Hybrid Model for Adaptive Educational Hypermedia,” Proc. Second Int'l Conf. Adaptive Hypermedia and Adaptive Web-Based Systems (AH '02), 2002.
[92] R. Zeiliger, “Adaptive Testing: Contribution of the Shiva Model,” Item Banking: Interactive Testing and Self-Assessment, D. Leclercq and J. Bruno, eds., NATO ASF Series F, vol.112, pp.54-65, Springer-Verlag, 1993.

Index Terms:
Evaluation/methodology, Computer-assisted instruction, Human information processing, Adaptive hypermedia, Miscellaneous, Artificial Intelligence, Computing Methodologies, User issues, Hypertext/Hypermedia, Information Interfaces and Representation (HCI), Information Technologies
Citation:
Elizabeth J. Brown, Timothy J. Brailsford, Tony Fisher, Adam Moore, "Evaluating Learning Style Personalization in Adaptive Systems: Quantitative Methods and Approaches," IEEE Transactions on Learning Technologies, vol. 2, no. 1, pp. 10-22, Jan.-March 2009, doi:10.1109/TLT.2009.11
Usage of this product signifies your acceptance of the Terms of Use.