This Article 
 Bibliographic References 
 Add to: 
Jan.-March 2011 (vol. 4 no. 1)
pp. 35-46
Chung Hsien Lan, Nanya Institute of Technology, ChungLi
Sabine Graf, Athabasca University, Athabasca
K. Robert Lai, Yuan Ze University, ChungLi
Kinshuk, Athabasca University, Athabasca
This study presents a conceptual framework for providing intelligent supports through agent negotiation and fuzzy constraints to enhance the effectiveness of peer assessment. By using fuzzy constraints, it not only provides a flexible marking scheme to deal with the imprecision and uncertainty for the representation of assessment but also provides a computational framework to incorporate student's personal characteristics into the process for the reduction of assessment bias. Additionally, a fuzzy constraint-based negotiation mechanism is employed to coordinate the cognitive differences between students. Through iterative agent negotiation, students can reconcile the differences and reach an agreement on the assessment results. Thus, the proposed framework allows students to provide more detailed, informed, and less biased assessments for their peers' work. To demonstrate the usefulness and effectiveness of the proposed approach, a negotiation-based peer assessment system, NePAS, has been built and used in classroom. Experimental results suggested that students were more willing to accept the assessment results and able to acquire more useful information to reflect upon and revise their work. Instructors can also observe students' participation and performance to appropriately adjust instructional strategies.

[1] K.A. Van Lehn, M.T. Chi, W. Baggett, and R.C. Murray, “Progress Report: Towards a Theory of Learning during Tutoring,” Learning Research and Development Center, Univ. of Pittsburgh, 1995.
[2] E.Z. Liu, S.S. Lin, C.H. Chiu, and S.M. Yuan, “Student Participation in Computer Sciences Courses via the Network Peer Assessment System,” Advanced Research in Computers and Communications in Education, vol. 2, pp. 744-747, IOS Press, 1999.
[3] J. Sitthiworachart and M. Joy, “Web-Based Peer Assessment in Learning Computer Programming,” Proc. Int'l Conf. Advanced Learning Technologies, pp. 180-184, 2003.
[4] S.S. Lin, E.Z.F. Liu, C.H. Chiu, and S.M. Yuan, “Web-Based Peer Assessment: Attitude and Achievement,” IEEE Trans. Education, vol. 44, no. 2, p. 211, May 2001.
[5] R. Davis and T. Berrow, “An Evaluation of the Use of Computer Supported Peer Review for Developing Higher-Level Skills,” Computers and Education, vol. 30, pp. 111-115, 1998.
[6] P. Davies, “Using Agent Reflective Self-Assessment for Awarding Degree Classifications,” Innovations in Education and Teaching, vol. 39, no. 4, pp. 307-319, 2002.
[7] E.Z.F. Liu, “Networked Knowledge Management and Evaluation System for CSCL: A Case Study,” Proc. Hawaii Int'l Conf. Business, 2003.
[8] L. Norton and J. Brunas-Wagstaff, “Students' Perceptions of the Fairness of Assessment,” Proc. Inst. for Law and Technology (ILT) Ann. Conf., 2000.
[9] S.S. Lin, E.Z. Liu, and S.M. Yuan, “Web-Based Peer Assessment Feedback for Students with Various Thinking Styles,” J. Computer-Assisted Learning, vol. 17, no. 4, pp. 420-432, 2001.
[10] G.L. May and L. Gueldenzoph, “The Effect of Social Style on Peer Evaluation Ratings in Project Teams,” J. Business Comm., vol. 43, pp. 4-20, 2006.
[11] D. Sadler, “Indeterminacy in the Use of Preset Criteria for Assessment and Grading in Higher Education,” Assessment and Evaluation in Higher Education, vol. 34, pp. 159-179, 2009.
[12] S.S. Lin, E.Z. Liu, C.H. Chiu, and S.M. Yuan, “Web-Based Peer Assessment: Attitude and Achievement,” IEEE Trans. Education, vol. 44, no. 2, pp. 13-13, 2001.
[13] M. Peterson, “Learner Interaction in an Avatar-Based Virtual Environment: A Preliminary Study,” PacCALL J., vol. 1, no. 1, pp. 29-40, 2005.
[14] M. Searby and T. Ewers, “An Evaluation of the Use of Peer Assessment in Higher Education: A Case Study in the School of Music, Kingston University,” Assessment and Evaluation in Higher Education, vol. 22, pp. 371-383, 1997.
[15] S. Hanrahan and G. Isaacs, “Assessing Self- and Peer Assessment: The Students' Views,” Higher Education Research and Development, vol. 20, no. 1, pp. 53-70, 2001.
[16] N.V. Hattum-Janssen and J.M. Lourenco, “Explicitness of Criteria in Peer Assessment Processes for First-Year Engineering Students,” European J. Eng. Education, vol. 31, no. 6, pp. 683-691, 2006.
[17] R. Ballantyne, K. Hughes, and A. Mylonas, “Developing Procedures for Implementing Peer Assessment in Large Classes Using an Action Research Process,” Assessment and Evaluation in Higher Education, vol. 27, pp. 427-441, 2002.
[18] A. Bhalerao and A. Ward, “Towards Electronically Assisted Peer Assessment: A Case Study,” Assoc. for Learning Technology J., vol. 9, no. 1, pp. 26-37, 2001.
[19] L. McDowell and G. Mowl, Improving Student Learning through Assessment and Evaluation, pp. 131-147, Oxford Centre for Staff Development, 1996.
[20] L. Arnold, “Use of Peer Evaluation in the Assessment of Medical Students,” J. Medical Education, pp. 35-42, 1981.
[21] K.R. Lai and M.W. Lin, “Modeling Agent Negotiation via Fuzzy Constraints in E-Business,” Computational Intelligence, vol. 20, pp.624-642, 2004.
[22] K.R. Lai, “Fuzzy Constraint Processing,” PhD thesis, NCSU, 1992.
[23] L. Zadeh, “Fuzzy Sets,” Information and Control, vol. 8, no. 3, pp. 338-353, 1965.
[24] M. Ichino, “A Nonparametric Multiclass Pattern Classifier,” IEEE Trans. Systems, Man and Cybernetics, vol. 9, no. 6, pp. 345-352, June 1979.
[25] M. Sugeno and T. Yasukawa, “A Fuzzy-Logic-Based Approach to Qualitative Modeling,” IEEE Trans. Fuzzy Systems, vol. 1, no. 1, pp. 7-31, Feb. 1993.
[26] A. Foroughi, “A Survey of the Use of Computer Support for Negotiation,” J. Applied Business Research, vol. 11, no. 4, pp. 121-134, 1995.
[27] D.G. Pruitt, Negotiation Behavior. Academic, 1981.
[28] M.W. Lin, “Agent Negotiation as Fuzzy Constraint,” PhD thesis, Dept. of Computer Science, Yuan Ze Univ., 2004.
[29] P. Faratin, C. Sierra, and N.R. Jennings, “Using Similarity Criteria to Make Trade-Off in Automated Negotiation,” Artificial Intelligence, vol. 142, no. 2, pp. 205-237, 2002.
[30] R.L. Bangert-Drowns, C.L. Kulick, J.A. Kulick, and M.T. Morgan, “The Instructional Effect of Feedback in Test-Like Events,” Rev. of Educational Research, vol. 61, pp. 213-238, 1991.
[31] W. Van Leekwijck and E.E. Kerre, “Defuzzification: Criteria and Classification,” Fuzzy Sets and Systems, vol. 108, pp. 159-178, 1999.
[32] Reinken Information Solutions Web Site Development—Marketing, Data Systems, http://reinken.comweb_eval.html, 2000.
[33] R.M. Felder and L.K. Silverman, “Learning and Teaching Styles in Engineering Education,” Eng. Education, vol. 78, pp. 674-681, 1988.
[34] C.A. Carver, R.A. Howard, and W.D. Lane, “Addressing Different Learning Styles through Course Hypermedia,” IEEE Trans. Education, vol. 42, no. 1, pp. 33-38, Feb. 1999.
[35] J. Kuljis and F. Liu, “A Comparison of Learning Style Theories on the Suitability for Elearning,” Proc. Int'l Assoc. of Science and Technology for Development (IASTED) Conf. Web Technologies, Applications, and Services, pp. 191-197, 2005.
[36] W.R. Sherrard and F. Raafat, “An Empirical Study of Peer Bias in Evaluations: Students Rating Students,” J. Education for Business, vol. 70, no. 1, pp. 43-47, 1994.
[37] A. Tziner, K.R. Aharon, J.N. Murphy, and J.N. Cleveland, “Does Conscientiousness Moderate the Relationship between Attitudes and Beliefs Regarding Performance Appraisal and Rating Behavior?” Int'l J. Selection and Assessment, vol. 10, no. 3, pp. 218-224, 2002.

Index Terms:
Peer assessment, assessment bias, agent negotiation, fuzzy constraints.
Chung Hsien Lan, Sabine Graf, K. Robert Lai, Kinshuk, "Enrichment of Peer Assessment with Agent Negotiation," IEEE Transactions on Learning Technologies, vol. 4, no. 1, pp. 35-46, Jan.-March 2011, doi:10.1109/TLT.2010.30
Usage of this product signifies your acceptance of the Terms of Use.