1. Wiki (WIM),
2. communication (COM),
3. tracking and modeling (TAM), and
4. group formation (GFM).
1. Active use—the actions of a student that push information onto his or her group's Wiki and change the content of that Wiki, e.g., the number of words: a) added, b) deleted, and c) rearranged.
2. Passive use—student activities in ClassroomWiki that pull information from his or her group's Wiki and do not result in a change in the contents of that Wiki. For example, the number of times a student views: a) the revision history of their group's Wiki, b) the topics posted by other group members, and c) the messages in his or her posted topics.
3. Interaction—a student's interactions with his or her group members while collaborating, e.g., the total number of topics created, the total number of replies posted, the size of his or her messages in words, and the average number of other group members who replied to a student's posted topic.
4. Survey response—a student's responses to the various surveys or questionnaires posted by the teacher. These surveys can be designed to capture a student's opinion about the effectiveness of his or her group, peers, or the ClassroomWiki itself. For example, a student may be asked to evaluate the contribution of another group member toward their group's Wiki.
5. Evaluation—the evaluation scores received by a student for all Wiki-related activities, e.g., a teacher's evaluation of a student for his or her contribution for the group Wiki.
To provide better assessment of individual student contribution—the five categories of student activity information stored in the student model help the teacher compare the effort or contribution of a student toward his or her group's Wiki against that student's group members' (in accordance to the Tracking and Modeling principle in Section 2.1). For example, a teacher may compare the total number of words that is added by a student with that of the average number of words added by his or her group members to estimate a student's contribution toward his or her group. This ability to compare the contributions can alleviate the free riding phenomenon since the students can be held accountable for not contributing to their group's Wiki. However, there could be scenarios where a student could try to game the ClassroomWiki system by, say, adding a large number of useless, trivial words. To counter this, the survey component comes into play. That is, the peer rating survey results are combined with the quantitative contribution assessment of the students such that a student who tries to game the system would receive a low peer rating from the group members when they observe the “ unnecessary word-adding” activity. This issue of improving the assessment of individual contributions, and thus, having more precise accountability, is further discussed in our future work.
The improved assessment of a student's individual contribution toward his or her group can also be used to prepare a detailed but summarized view of the members of each student group for the teacher so that he or she is able to provide specific and precise intervention if needed. For example, if there are free riding students in a student group, the information extracted from their models would allow the teacher to conveniently identify (even when classroom size is large), intervene, and motivate those students.
To improve group formation—the student models would allow the GFM to form student groups that contain a heterogeneous mix of students with varying levels of performances (as an individual (7) and as a group member (8)), i.e., implement the Heterogeneity principle discussed in Section 2.1. Furthermore, since a student's model is continuously updated, those models will capture the changes in the students' performances (as an individual (7) or as a group member (8)) while he or she progresses through the Wiki assignments. So, when the GFM forms student groups using the student models, the formed groups will reflect the changes in the students' behaviors, thereby implementing the Adaptation principle. For example, if during a Wiki assignment, a student improves his or her knowledge and contribution level toward his or her group, the student's model (i.e., in (7), (8)) would capture that change in terms of improved evaluation scores and increased contribution (forum, revision, etc.). The GFM would then use that changed model to assign the improved student to a different group with a more appropriate level of heterogeneity in future rounds.
Proposition: In the proposition step, the proposer agent chooses other agents ( is the minimum group size) and proposes a group that includes the students assigned to those chosen agents. The proposal from an agent to agent is , where is a set of models (not the ids) of the students in the proposed group are the expected current task and future task rewards (9), (10) for the task calculated from the perspective of agent .
Consideration: In the consideration stage, the proposed-to agent first compares its model stored by the proposer agent with its own model of student . If that model is not updated, in other words, if agent is unaware of the recent changes in the model of the student , the responding agent rejects the proposal and sends the updated model of to the proposer. Note that this notification from the responding agent allows a proposer to have updated view of the other potential members during the coalition formation round. This update procedure is important since each agent is assigned to a single student and may be unaware of the changes in the models of other students. If the proposer has the updated view of the responding agent's assigned student, the responding agent compares the expected current task and future task reward values of the proposed group to its current group. The responding agent leaves its current group to join the proposed group if the weighted sum of the current task and future task rewards is larger for the proposed group , i.e.,
Here, in (11), the and values are calculated by the proposer using the functions and (9), (10), respectively. So, in this negotiation process, an agent's decision regarding whether to join a group is determined by the value of these functions. Note that the number of times this entire negotiation process is run depends on the number of negotiation rounds, which is set as a multiple of the number of agents so that each agent is able to act as a proposer multiple times. Furthermore, to ensure that there is always an agreement among the agents, we set the current task and future task reward values (11) to zero if an agent cannot join a coalition (i.e., there is no agreement). Since (11) yields a nonzero value for any group, there will always be an agreement among the agents since it is better to be in any group than to be in no group.
Notification: If all of the chosen agents agree to join the proposed group, the proposer sends out a confirmation message to them notifying that they are now in the newly formed group. Otherwise, if any one of the responding agents disagrees, the proposer stops the negotiation process and waits for some other agent's proposal or its next turn to join a group.
Improving the assessment of the qualitative aspect of student contributions in ClassroomWiki by estimating the quality of their edits and messages using natural language processing techniques (LingPipe tool—alias-i.com/lingpipe) such as:
- content-related phrase identification,
- sentence detection,
- stemming [ 25], and
- common words detection.
Implementing a self-reported and group-reported (i.e., signed off by the group) contribution assessment method. This is to better validate the impact of our current method of student assessment on the free riding of students.
Improving the MHCF group formation by incorporating a Bayesian Network to enable the agents to learn the current and future task reward functions that map the student models—and learner characteristics—to students' collaborative learning outcomes.
Obtaining more detailed results for our improved ClassroomWiki by running a more comprehensive, semester-long experiment with a large set of students for multiple collaborative writing assignments. In this experiment, we plan to:
- compare MHCF group formation with VALCAM [ 26]—another group formation method to provide a stronger comparative baseline,
- investigate the impact of MHCF on student performance when MHCF is able to utilize the student model built on a more detailed history of student activities, and
- collect more data to obtain results with higher statistical significance, and to further evaluate the impact of the three design principles.
N. Khandaker is with the Department of Computer Science and Engineering, University of Nebraska, 122F Avery Hall, Lincoln, NE 68588-0115. E-mail: email@example.com.
L.-K. Soh is with the Department of Computer Science and Engineering, University of Nebraska, 122E Avery Hall, Lincoln, NE 68588-0115. E-mail: firstname.lastname@example.org.
Manuscript received 29 July 2009; revised 3 Sept. 2009; accepted 3 Nov. 2009; published online 12 Nov. 2009.
For information on obtaining reprints of this article, please send e-mail to: email@example.com, and reference IEEECS Log Number TLT-2009-07-0129.
Digital Object Identifier no. 10.1109/TLT.2009.50.
Nobel Khandaker received the BS degree (with honors) in physics from the University of Dhaka, Bangladesh, and the MS degree in computer science from the University of Nebraska Lincoln. He is currently working toward the doctorate degree in the Department of Computer Science and Engineering at the University of Nebraska Lincoln. His primary research interests include teamwork and coalition formation for human participants, multiagent coalition formation in uncertain environments, multiagent learning, computer-supported collaborative learning systems, and agent-based simulation. He is a member of the ACM and the AAAI and a student member of the IEEE.
Leen-Kiat Soh received the BS (with highest distinction), MS, and PhD degrees with honors in electrical engineering from the University of Kansas. He is currently an associate professor in the Department of Computer Science and Engineering at the University of Nebraska. His primary research interests are in multiagent systems and intelligent agents, especially in coalition formation and multiagent learning. He has applied his research to computer-aided education, intelligent decision support, and distributed GIS. He is a member of the IEEE, the ACM, and the AAAI.