Pages: pp. 5-20
Abstract—This study critically reviews the recently published scientific literature on the design and impact of adaptive and intelligent systems for collaborative learning support (AICLS) systems. The focus is threefold: 1) analyze critical design issues of AICLS systems and organize them under a unifying classification scheme, 2) present research evidence on the impact of these systems on student learning, and 3) identify current trends and open research questions in the field. After systematically searching online bibliographic databases, 105 articles were included in the review with 70 of them reporting concrete evaluation data on the learning impact of AICLS systems. Systems design analysis led us to propose a classification scheme with five dimensions: pedagogical objective, target of adaptation, modeling, technology, and design space. The reviewed articles indicate that AICLS systems increasingly introduce Artificial Intelligence and Web 2.0 techniques to support pretask interventions, in-task peer interactions, and learning domain-specific activities. Findings also suggest that AICLS systems may improve both learners' domain knowledge and collaboration skills. However, these benefits are subject to the learning design and the capability of AICLS to adapt and intervene in an unobtrusive way. Finally, providing peer interaction support seems to motivate students and improve collaboration and learning.
Index Terms—Adaptive collaborative learning support, intelligent support systems, adaptive hypermedia, collaborative learning.
The purpose of this work is to review the design and learning impact of adaptive and intelligent systems which are developed to support collaborative learning.
Although “adaptivity” and “intelligence” have built a long tradition in technology systems for individual learning (e.g., [ 1]), it is only relatively recently that these ideas have entered the domain of computer-supported collaborative learning (CSCL). Several research groups have reported on the design and evaluation of adaptive and/or intelligent systems that aim to support collaborative learning (e.g., [ 2]) (a detailed analysis of the design and impact of intelligent tutors to support individual and group learners is given in Woolf [ 3]).
After providing some initial background information, we shall discuss the method we followed in order to review the area. Results stemming from this method, organized under a specific classification scheme are presented next. This classification scheme
We also present research efforts at the level of a proposed architecture or framework and major trends in the field of AICLSs. We finally conclude with a discussion identifying open research questions.
In the context of technology-enhanced learning, system designers have tried to systematically exploit the modeling potential of computers and develop systems that support learners through adaptive or intelligent operation. Adaptive and Intelligent systems are both model-based systems although they have different purposes in supporting learning.
An adaptive educational system (AES) is mainly a system that aims at adapting some of its key functional characteristics (for example, content presentation and/or navigation support) to the learner needs and preferences [ 1]). Thus, an adaptive system operates differently for different learners, taking into account information accumulated in the individual or group learner models.
Respectively, an intelligent tutoring system (ITS) aims to provide learner-tailored support during the problem-solving process, as a human tutor would do [ 1]. To achieve this, ITS designers apply techniques from the broader field of Artificial Intelligence (AI) and implement extensive modeling of the problem-solving process in the specific domain of application.
Introducing adaptive characteristics gave birth to the strand of Adaptive Hypermedia Systems (AHS), a significant subset of which is Adaptive Educational Systems with systems like AHA, InterBook, and WebCOBALT [ 1]. Respectively, the strand of the ITSs appeared with systems like ELM-ART, KBS-Hyperbook, and SQL-Tutor [ 1]. According to Brusilovsky and Peylo [ 1], ITS traditionally focused on Curriculum Sequencing, Intelligent Solution Analysis, and Problem-Solving Support, while AES focused strongly on Adaptive Presentation and Navigation Support.
The above approaches aim principally at helping the individual learner. Recently, research efforts have focused on introducing adaptivity and intelligence in the context of CSCL bringing together AESs and ITSs on one hand and CSCL systems on the other. There is strong evidence that adaptation advances the learning effects of CSCL (e.g., [ 4], [ 5], [ 6]).
Dillenbourg [ 7] suggests that “collaborative” concerns essentially four aspects of learning:
Computer-Supported Collaborative Learning emerges when introducing technology in collaborative learning settings (a historical perspective of CSCL is presented by Stahl et al. [ 8]). Lipponen [ 9] emphasizes that CSCL is focused on how collaborative learning supported by technology can enhance peer interaction and work in groups, and how collaboration and technology facilitate the sharing and distribution of knowledge and expertise among community members.
The pedagogical roots of collaborative learning are to be found in Vygotsky's work [ 10] who extended Piaget's constructivist perspective toward the social field. That is, the dialogue between learners who interact developing a shared understanding of a problem and its solution process. So, collaborative learning combines social and construction elements of the learning process and CSCL aims at efficiently introducing technologies capable of supporting both components [ 6], [ 11]. However, CSCL is not simply implying the use of technology for communication purposes. Successful CSCL applications aim to capture and model information and knowledge of group activity and use it to achieve a more effective group monitoring and support [ 12], thus leading to the development of AICLS systems. Developing this type of systems instantiates two key notions of the CSCL conceptual framework, namely, distributed cognition and the zone of proximal development [ 13]. In relation to the former, AICLS systems aim to capture the complexity of interactions in the collaborative learning setting and transform it to understandable and useful representations, thus offloading the teacher and the learners from respective cognitive overload. In relation to the second, the ambition of system designers is that their tools exhibit a supportive behavior similar to that of a helpful experienced partner who intervenes unobtrusively and “just in time” to support group learners in achieving a productive level of interaction and therefore in accomplishing their task.
Computationally supported adaptive and intelligent operations are increasingly integrated in the design of CSCL systems in an effort to maximize the user-tailored support provided to group learners, focusing both on improved domain learning and development of collaboration skills. In general, creating adaptive/intelligent systems for CSCL is considered to be more demanding than creating respective systems for individual study, since one must also take into account aspects associated with social relations and group dynamics, apart from the pedagogical ones [ 6].
As a result of this endeavor, two specific research areas have emerged within the wider area of CSCL systems: 1)Adaptive Collaborative Learning Support (ACLS) systems, and 2) Intelligent Collaborative Learning Support (ICLS) systems. However, these subsets also have a significant common section; there are systems that combine characteristics of both the intelligent and adaptive design approach. This intersection is large and the borders between “intelligent” and “nonintelligent” are not clear-cut, so both groups are certainly of interest for the “AI in Education” (AI-Ed) and the CSCL communities.
In our approach, we use the acronym “AICLS” system as a general term to denote the broader research area of adaptive and/or intelligent systems that aim to support the collaborative learning activity. In this way, we emphasize the commonalities of the two approaches within the perspective of technology systems for personalized learner support.
The focus of our work is threefold: 1) to analyze critical design issues of AICLS systems and organize them under a unifying classification scheme, which can offer deeper insight on the design characteristics and trends in the area, 2) to present research evidence on the impact of these systems on student learning focusing strongly on studies that present student-based evaluation data, and 3) to identify and discuss current trends and open research questions concerning the design of AICLS systems.
For the purposes of this study, a literature search was undertaken in December 2009, in the following international online bibliographic databases:
The search string used was: (“intelligent” OR “adapt” OR “adaptation” OR “adaptive”) AND (“collaboration” OR “collaborative”) AND (“CSCL” OR “learning”) and was performed mainly in the Abstract and Titles of the candidate articles. Searches were limited to articles published in journals and conference proceedings, in English, from 1998 onwards. The latter constraint was posed due to the rapid changes in ICT, in general, and in CSCL technologies, in particular in the last decade. Furthermore, a number of journal proceedings articles and workshop contributions that were located during searches in the afore-mentioned databases were examined (applying the same search criteria as above). Those relevant to adaptivity/intelligence and CSCL systems were also considered.
Hundreds of articles resulted from the search engines initially. Given that the study focused on the examination of adaptivity/intelligence as potential CSCL systems enhancement, articles located through the database searches that did not fall within that focus were excluded from consideration. Articles excluded were those that: 1) did not clearly focused on collaborative learning, or 2) presented CSCL systems with no real adaptation focus, for example, articles which simply identified that adaptation is a necessity but did not deal with it as an automatic (or even semiautomatic) system operation neither analyzed the need for adaptivity in learning environments.
Overall, this research and filtering process resulted to 216 articles candidates for review. We filtered these 216articles further and finally included in our review the following three groups of articles:
Overall, 105 articles in the three groups above were thoroughly reviewed and analyzed. It is important to mention that a key criterion for including an article in the review was whether it presented student-based evaluation data on the learning impact of the system (group 1). However, other articles with different type of evaluation data (for example, expert-based evaluations applied in system architectures) were included in groups 2 and 3.
We have also chosen to cite only one paper from the multitude of papers that a research group usually publishes, either the most recent one or the one presenting the most significant evaluation data of the system developed, for reasons of space economy. This reduced the group-1 articles cited in this review to 46 (out of 70 reviewed), those that appear in Table 1.
The comparative analysis of the selected literature works resulted in a classification scheme, which is proposed here as a comprehensive tool for analyzing and interconnecting the major design principles applied in the area of AICLS systems. We propose that AICLS systems can be classified and their design and operation can be analyzed in the following dimensions:
The above classification scheme emerged from a systematic analysis of the reviewed works, focusing on the key features of an AICLS system, as discussed or otherwise emphasized by the authors/designers in their articles. Following a bottom-up approach, we reached a generalization regarding the main operational dimensions of a system like this. In essence, an AICLS system is a group learning supportive tool that utilizes some kind of technological tools and methods (technology) to implement a specific strategy on how to support learners (pedagogical objective), by externalizing an adaptive or intelligence-based form of support (target of intervention), depending on certain machine-based representations of key aspects of the learning setting (modeling). The above four distinct aspects comprise, we argue, a basic ontology of an AICLS system and so are directly represented in the proposed scheme. Additionally, we decided to include one more dimension: “design space.” Although this dimension appears in only one work (Walker et al. [ 14]), we believe that it provides a worth exploring alternative in designing the adaptive feedback of an AICLS system.
This first dimension provides an essential linking to the broader area of CSCL systems. According to Soller et al. [ 2], depending on its pedagogical objective, a CSCL system can be characterized as “mirroring,” or “metacognitive,” or “guiding.”
Simply speaking, a mirroring system aims to “mirror” the collaborative activity by presenting to learners the value of selected activity parameters (e.g., [ 5]). A system that presents the number of posts submitted by group members during an online asynchronous discussion is a mirroring system. Next, a metacognitive system presents additionally (as compared to a mirroring system) information on what productive group behavior might be (e.g., [ 15], [ 16]). For example, a metacognitive system would inform teammates that essential engagement of all partners in the activity would be represented by a balanced post contribution by all group members. Finally, a guiding system would additionally guide (or advice) the peers on what to do in order to improve collaboration. In our example, a CSCL guiding system would advice (may be even compel) lagging behind students to increase their participation by sending more posts.
All AICLS systems reviewed are in essence guiding systems in the PO dimension. They are designed to implement a strategy of providing explicit support (guidance) to the collaborating students as they not only monitor (neither provide only metacognitive hints) but take over the entire regulation process and propose remedial actions based on a computational modeling of the learner/group characteristics and/or diagnosis of peer interactions. For example, the COMET's [ 17] clinical reasoning strategy is to directly guide learners through instructions/hints in a problem-based learning setting. Also, the objective of NUCLEO [ 5] is to help forming groups of learners based on a specific model of their learning styles.
This dimension refers to what is being adapted (for an adaptive system) or what is the focus of the intelligence-based support (in the case of intelligent systems). We propose that this dimension can be further analyzed in the following subdimensions:
These subdimensions clearly represent the core CSCL supportive objectives, as articulated in the prescriptions encountered usually in the CSCL literature (see, for example Dillenbourg in [ 16]) on how to increase the probability that productive peer interactions will occur during collaboration. First, one needs to optimize the group formation process (in general, to set up favorable initial conditions), then domain-specific support is required in order to help peers improve their domain knowledge understanding through interaction (i.e., “collaborate to learn”). Finally, peer interaction support is also essential to structure and guide interaction among group members (i.e., help peers to “learn to collaborate”).
Therefore, a system that would provide adaptive/intelligent support to collaborating students would eventually target to one (or more) of the above subdimensions.
One way to increase the probability that certain types of peer interaction will occur is to carefully design the situation. All efforts aiming to this point, deal with group formation issues (e.g., how many persons in a group, what individual should be grouped together, etc.). Group formation is important because the synthesis of the group is expected to be crucial for triggering productive peer interaction [ 6]. In this section, we include AICLSs that aim to optimize aspects of the group (i.e., its size or synthesis) based on data on an individual user and more specifically on various aspects of user's profile like preferences, learning domain knowledge level, learning style, etc. (13 studies out of 46 overall, see Table 1, line “Group formation”).
One major approach in group formation is to form heterogeneous groups based on students' learning styles. The key idea is that group heterogeneity may foster improved peer interactions [ 18], [ 19], [ 20].
For example, in [ 5], Vermunt's model for learning styles is used for grouping students as they are following the mechanics of a classical role-based play represented by avatars. Preliminary results of the experiments and conclusions of the NUCLEO system indicate that, first, the impact on the student's motivation is being very positive (even if the programming topic is not probably the best suited application domain). It is also observed that the marks obtained by the students that belong to the NUCLEO group in partial evaluations are slightly better (around 10 percent) than the ones obtained by members in the traditional group (the group formed without considering learning styles and without NUCLEO's guidance). In a similar direction, [ 18], [ 19], and [ 20] use learning styles (various models) and present improved learning outcomes in general for the treatment groups (heterogeneous groups formed based on learning styles models used). More specifically, in [ 19], the Felder-Silverman learning style model and its index of learning styles (ILSs) questionnaire are used, in order to classify students, depending on their preferences on four dimensions (active/reflective, sensing/intuitive, visual/verbal, and sequential/global). From the results obtained, certain conclusions are extracted: 1) learning styles seem to affect the performance of the students when working together, 2) the tendency seems to be that mixed pairs in the active-reflective and the sensing-intuitive dimensions work better [ 19].
There are research works that are based on stereotype theory in order to group individuals. In [ 21] and [ 22], the user characteristics are related not only to the knowledge state of the users, but also to their personality and performance types. In [ 22], Bayesian Networks are used as an AI technique for grouping. In [ 21], the stereotype theory is the basis for building the individual learner models in order to form groups. In the above works, experienced trainers decide on the combinations of various types of stereotypes, considered to be the most appropriate for modeling learners. Results regarding the use of stereotype theory for group formation are positive.
Two other studies present promising results [ 23], [ 24] combining group formation adaptation with learning domain adaptation. In the former, similarity measures between user models (based mainly on prior domain knowledge) and material lead to peer matching. Although this study presents simulated results, these appear successful in forming groups and simultaneously selecting appropriate learning objects for each group based on similarity coefficients between user and learning objects. In the latter, the learning domain is of greater importance as peers are matched according to the interactive solving process they follow, regardless of their learning styles. In [ 24], the MATHEMA system along with adaptive group formation, peer help also provides adaptive navigation support to the learners through the following techniques: direct guidance, link annotation, link hiding, and link sorting. This study, after presenting a review of AES/ITS and their implemented techniques, illustrates by results a significant improvement of student performance after MATHEMA's guidance during collaborative dialog and in forming groups and hints.
A series of works use AI techniques for peer matching/group formation [ 25], [ 26], [ 27]. In [ 25], association rules and clustering are employed, in [ 26], fuzzy and genetic algorithms, and in [ 27], fuzzy C-means. They all use models of an individual and specific properties of the student (e.g., in [ 26], individual traits are used as a property for building user profiles) and provide indications for the efficiency of the proposed approach in forming homogeneous and heterogeneous groups in a real context. Moreover, they are witnessed to be well-accepted by students in real educational contexts.
A concluding work, [ 28] is reported separately as it is standard based (i.e., uses IMS-LD [ 29] to express group synthesis). It uses CLFPs that are well known in the CSCL community (Jigsaw and TAPPS) [ 30] and offer some preliminary evaluation results from a controlled user study. Results show that such type of group formation is useful mainly in two cases: 1) when performing complex collaborative learning activity in which there are many constraints to control and 2) when preparing activities with a big number of students. This work takes into account various parameters aiming to adapt the predefined group after a CSCL process has started and adapt the group to various contextual constraints (e.g., some students have left and the group needs reorganization). We reckon that such an approach if applied in real fields needs to use AI techniques like those already mentioned.
At this point, we would like to clarify three terms that are often encountered in the CSCL area. “Collaboration scripts” are specific didactic scenarios that aim to engage students in fruitful learning interactions by providing explicit collective workflow description, guidance, and support during the collaborative activity [ 6]. IMS-LD is a modeling language that uses the metaphor of a theatrical play for expressing a computerized form of a teaching-learning process (that is, of a “learning design”) [ 31]. In the field of CSCL, there is an ongoing debate and effort regarding the extensibility of IMS-LD to express flexible collaborative learning design [ 31], [ 32], [ 33], [ 34]. Finally, “collaborative learning flow patterns” (CLFPs) are best practice learning designs, that is, learning designs that may lead to a successful CSCL process when applied under certain circumstances [ 35].
Overall, however, it is considered that the research on group formation is rather limited (see [ 36], [ 37], [ 38]), an issue that renders questionable the efforts for designing groups based only on learner's profiles, technologies, and tasks [ 39]. More recent research efforts emphasize the need for founding the group formation process on a sound pedagogical basis, which would offer the ground for an effective collaborative learning design. The work by Isotani et al. [ 39] provides an example toward this direction employing also Semantic Web techniques (more on this approach is presented in Section 5).
In conclusion, AICLS systems in group formation area aim at supporting the collaborative learning process by analyzing students' traits (preferences, learning domain knowledge level, learning styles, and stereotypes) and also teacher design decisions during the CSCL activity and suggest heterogeneous grouping [ 39]. More specifically:
Domain knowledge support-type refers to actions taken by the system in order to help learners improve their domain learning. The system aims to help students in acquiring an acceptable level of knowledge of the instructional domain without being necessarily concerned with the way the above objective has been obtained (i.e., more or less collaboratively).
This kind of support is related to aspects of users and groups (and learning tasks also) that need to be modeled in order to support group domain learning better and can be inferred or observed in system/user interaction. For example, the group performance models can contain learning-related aspects, such as student domain knowledge, type of exercises the student prefers, or mistakes the student has made.
There are 10 papers (out of 46) presenting current efforts on the development of AICLS systems that offer group domain knowledge support (see Table 1, line “Domain-specific support”). The main effort when designing such systems is on providing hints, prompts, or questions to students, based on their “distance” from the solution to a problem or from an acceptable result.
Suebnukarn and Haddawy [ 17] illustrate that Bayesian network clinical reasoning models can be combined with generic tutoring strategies to successfully emulate human tutor hints in group medical Problem-Based Learning (PBL). Evaluating the accuracy of the student models in determining the group-reasoning path in three different scenarios provided encouraging results. Also, empirical evaluation showed a high degree of agreement between the hints generated by COMET and those of experienced human tutors.
In [ 40], a more general approach to domain learning scaffolding through case-based learning is presented (i.e., not focused on a specific domain-like medicine). The system provides adaptive online scaffolds to students with higher domain knowledge in order to help them pose more meaningful questions to their lower knowledge peers. Results indicated that for some students, the online guidance served as “a starting point” to generate questions when they had difficulty asking questions. However, system guidance did not improve the quality of questions and thus the learning outcomes. The interview data also indicated that peer-generated questions served a critical role in facilitating learner's reflection and knowledge reconstruction. The authors suggest that considering adaptive forms of scaffolding and intermediate factors such as prior knowledge, metacognition, task complexity, and scaffolding type is an open research issue.
In [ 41], Walker et al. present a traditional ITS (algebra tutor) that was adapted to work in CSCL setting. Students worked in dyads acting as tutor-tutee. The system acted as a domain expert advising the peer tutor on how to provide feedback to the tutee. Two types of domain support were provided: adaptive support, which used the intelligent tutor domain models to provide feedback to the peer tutor and fixed support, which simply consisted of answers to the problems. The two peer tutoring conditions (adaptive versus fixed support) were compared to the individual use of the cognitive tutor (without peer-tutoring activities). No significant differences were found on student learning between the individual and collaborative conditions (however, students in the individual condition solved more problems during instruction). Authors suggest that a possible way to improve the effects of the adaptive tutoring is to take into account the details of peer interaction employing interaction analysis (IA) methods (see next section on PI-type support).
FLE3 system presented in [ 6] provides intelligent support to teachers' intervention in collaborative knowledge building. Technologically, an assistant agent monitors the collaborative activity, visualizes it and adaptively provides advice to the teacher on how and when to intervene. The author concludes that: 1) students believed that all the advice came from the human teacher, and 2) students changed their behaviors following the advice from the assistant agent toward improved learning outcomes.
In [ 42], the HabitPro models various group features (group abilities and preferences and group historical mistakes, user motivation and participation) and adaptively proposes different pedagogic methodologies and different exercises to learners. It uses a peer-communication turn protocol to increase the students' performance and facilitate collaboration. The user interface of HabitPro presents a traffic light that informs the student whether she/he has a turn (green light) or not (red light). Results indicate that HabiPro works efficiently with groups of two members or a maximum of three. With more partners communication was more difficult and collaboration suffered.
COLLECT-UML [ 43] is an Intelligent CSCL system for the instruction of UML. It uses Constraint-Based Modeling (CBM) approach (i.e., it represents the UML domain knowledge as a set of constraints and a rule-based system) and offers adaptive/intelligent support by providing learners with hints during individual and group problem-solving process and, also, feedback on peer interaction based on individual student contribution. Results show that Constraint-Based Modeling is an effective technique for both UML modeling and supporting students in developing collaboration skills (the participants acquired both declarative knowledge about good collaboration and did collaborate more effectively). On the other hand, in [ 44], Dragon et al. use a system with both expert knowledge and argumentation knowledge databases in order to support group learning by integrating intelligent coaching within argumentation-based collaborative activity. Authors highlight that: 1) the system seems to increase the amount of work students complete when working with the system and 2) there is a research potential for building a system to promote collaboration between students by using the expert knowledge base to match interventions with students' arguments during peer interaction process.
Concluding, four major issues are emphasized through analyzing studies that fall in this subdimension:
Most of these systems capture in a relevant representation/model the knowledge of the domain usually possessed by a domain expert. Finally, there are also studies that aim to support both domain learning and peer collaboration (i.e., support both the “collaborate to learn” and the “learn to collaborate” perspectives of a CSCL process) [ 6].
Several of the reviewed studies report on systems that aim to provide peer interaction support during the CSCL activity (“PI-type” or “learn to collaborate” support). There are 32papers (out of 46) that present efforts aiming to support peer interaction (see Table 1, line “Peer Interaction support”).
Peer interaction support refers to the actions taken by the system in order to help partners improve their in-group interaction and possibly develop also domain-general knowledge and skills (for example, argumentation, peer tutoring, and peer reviewing). In general, PI support is considered important since the system interventions are expected to help individuals in acquiring important collaboration skills (e.g., [ 45], [ 46], [ 47]).
Typically, an AICLS system offering PI-type support focuses on the collaboration process regardless of the domain and provides customized feedback to learners by modeling peer interactions during the activity. To accomplish this, the system tracks the progress of the collaboration and performs a type of interaction analysis. “Interaction Analysis” is a relatively new research direction aiming to extract useful indicators by processing data recorded during student-machine and/or student-student interactions. These indicators (presented to users in graphical or literal mode) are expected to help students and/or teachers in order to self-assess their activity [ 48], [ 49], [ 50], [ 51].
In [ 51], authors analyze the main problems and alternatives encountered in the process of effectively applying IA in CSCL, based on lessons learned from the authors' active involvement in this field during the last decade. Authors propose a system architecture together with the use of a standard common data format for interaction as main components for a global design process that will support practitioners, researchers, and technology providers in a sustainable and transparent way. Similarly, studies [ 50], [ 52], [ 53], promote the core idea of:
Certain systems employ data mining techniques to infer knowledge about the collaboration process. These systems can be characterized by the inferring method used to derive the value of certain features, such as the quality of interaction that has occurred. An example is [ 45] where the results indicate that the clustering approach infers useful information regarding the learners' interaction. Moreover, in [ 46], results reveal that by presenting useful and structured information to students and tutors on aspects of student collaboration, may have beneficial impact both on students' learning and on the management of their interaction.
Many works use AI techniques implemented in the form of a software agent in order to process data relevant to the peer interactions occurring during a collaborative activity. As such, [ 47] is an example where agent technologies are used to analyze statistical information about user activity and provide peer interaction hints. Results of the study indicate that it is possible to take advantage of statistical information in collaborative learning environments in order to scaffold aspects of peer interaction and knowledge building.
Harrer et al. [ 54] presented a system using intelligent techniques (bootstrapping novice data (BND) from log files capturing student problem-solving actions) to automatically build an example-tracing agent tutor that supports learners by adaptively presenting hints on how to improve interaction. Although the system does not fully provide help or tutoring yet, the initial evaluation data led authors to identify five dimensions of peer interaction and problem-solving behavior. These dimensions emphasize the need for abstraction of student actions in order to improve recognizing, analyzing, and providing feedback during interaction. The authors also interviewed a domain expert who provided evidence for the advantage of bootstrapping over nonautomated creation of a collaboration tutor.
In [ 55], [ 56], [ 57], it is stated that prior work on supporting partners' interaction has relied largely on comparing student discourse to reference models of collaborative discourse. Comparison of student work to expert solutions is prevalent in individual coaching paradigms. Although this approach is valuable, in the above studies, the authors chose to distinguish the potential contribution of tracking student participation and comparing students' individual to group solutions. These studies focus on developing problem-solving skills in software engineering (e.g., entity relationship (ER) diagramming). They use agent technologies that locate conflicts between individuals in order to intervene and hopefully promote fruitful collaboration (conflicts meaning low or no participation at all or a typical participation, that is repeatedly accepting, rejecting, or ignoring proposals without proper justification). In a similar philosophy, [ 58] succeeds in identifying detrimental peer interaction. The system models users and builds their profile from information gathered by their interactions with the system combined with the students' learning style, in order to intelligently detect possible conflicts and adaptively elaborate a summary. The interface agent interacts with the teacher, notifying him or her about possible conflicts with an elaborate summary. Although the system does not yet fully and systematically use this finding for helping students directly, support is achieved indirectly through hints to the teacher. Results suggest that intelligent detection and intervention not only helps teachers while supervising groups, but also improves the final success of students in their assignments.
In [ 59], authors have proposed a theoretical model for peer tutoring—based on agent technologies—that can be used as a basis for providing feedback to students (i.e., the intelligent tutoring system can pinpoint the student error and act if deviations from this model accumulate). Afterwards, they then validated the model using data that separated positive tutoring behaviors from negative ones, with results showing that the proposed model is a valid representation of good peer tutoring.
In [ 60], an advanced AI technique (agent-based swarm intelligence system—SIS) is employed to adaptively manage the group learning path and resources. The results indicate that: 1) various characteristics of system behavior (emerging from the implementation of the SIS technique) are positively associated with user system appropriation (that is, user satisfaction when using the system to accomplish learning goals) and 2) the learners' system appropriation is also positively linked with learning benefits. Findings also show that learners in this environment outperform their counterparts who use a generic web-based learning environment.
In several studies [ 61], [ 62], [ 63], [ 64], [ 65], text-lexical analysis is employed, focused on the problem of analyzing and representing the student discourse. These endeavors also hold the potential for enabling substantially improved online instruction both by providing teachers and facilitators with reports about the groups they are moderating and by triggering context-sensitive peer interaction support on an as-needed basis. More specifically, in [ 65], an approach is described, which employs intelligent data analysis techniques, to support teachers as they moderate multiple simultaneous e-discussions. The authors have generated six machine-learned classifiers for detecting potentially important discussion characteristics, such as a “reasoned claim” or an “argument-counterargument” sequence. The classifiers were integrated within a system and allowed teachers to analyze simultaneous online discussions. The results were promising, in summary: 1) the classifiers might help a teacher find patterns of beneficial and detrimental discourse situations, 2) the classifiers can be used in a quantitative way to summarize discussions at various abstraction levels thus making the system potentially automatically responsive and adaptive when specific interaction patterns occur.
Rosé et al. [ 62] report promising results as their study uses the “TagHelper” text-lexical analysis tool, demonstrating that an important aspect toward making text classification technology effective is designing and building linguistic pattern detectors. These detectors can be extracted reliably from texts and are reported to have high predictive power for the categories of discourse actions that interest the CSCL community. Authors do not present results on the learning impact of implementing their approach. Instead, they show that some processes (highly valued in CSCL research, for example, partners formulating counterarguments or collaboratively applying scientific concepts to solve problems), could be automatically identified in text messages with a level of reliability considered acceptable for agreement with human coders.
By contrast, García et al. [ 66] (and a series of other relevant works) present their efforts to adapt the process of peer review according to student profile (the presented model focuses on student's knowledge level as the key element to influence the process). Results are promising as they confirm the influence of student's profile on the learning outcomes of the peer review process and, consequently, the value of the adaptive peer review methodology.
Marcos-García et al. [ 67] use interaction analysis techniques (social network analysis—SNA—through a tool called Role-Adapt IA) to model aspects of interactivity (like density and frequency of individual interactions) and locate interaction patterns in mainly negative situations. System intelligently warns teachers about role changes. Results indicate that the warnings received by teachers and their later interventions managed to cope with these negative situations and eventually, to improve the collaborative activities. It is worth noticing that without the Role-Adapt IA support most of the situations demanding interventions would have slipped the teacher's attention.
In [ 68], Bravo et al. suggest considering groups as complex adaptive systems that should be dynamically studied in their embedding context. The authors propose a thorough framework, which is reportedly quite promising. The analysis models used (i.e., the user analysis model, group analysis model, and solution analysis model) comprise information about the users' work and interaction as well as an estimate of their knowledge level evaluated by the solution provided. Thus, the system can intervene by guiding students during the execution of the tasks toward more effective learning interactions. Results deriving from the use of the system are satisfactory as judged by the teachers and also by the students, because the models (user, group, and solution) provided them with a fairly accurate view of the students' problem-solving process.
Sakurai et al. [ 69] and Kinshuk et al. [ 70] are representative works that propose an “Enriched Cyberspace” approach for web-based CSCL to overcome problems encountered by distant collaborators. More specifically, they use biological sensors to capture the socioemotional states of the individuals participating in a collaborative environment. Results indicate that the system enriches the virtual collaborative learning environment by making adaptive use of keyword/summary presentation and successfully adapts the context of the environment. In [ 70], learning and collaboration are especially promoted, as the system identifies potential misunderstandings of individual users by: 1) capturing socioemotional states, 2)taking into account different cognitive capacity, competency levels, and backgrounds of users, and 3) providing these users with personalized content of the current discussion through keyword summarization functionality.
A study depicting formative assessment focus is Sugimoto et al. [ 71], where externalization of what has happened inside a game is used for the assessment of the collaboration process. The proposed system integrates a board game and a computer simulation, and is used for studying urban planning and environmental problems. Each learner externalizes and represents his/her own ideas on a board game, which allows him/her to actively participate in a learning situation and to share the representations with other learners. Several experiments were carried out in a public elementary school. The results show that ePro system is enhancing interactions, discussions, and learner engagement effectively.
In a work from Hadjileontiadou et al. [ 72], techniques from AI field and in particular fuzzy algorithms are used to support collaboration of peers, assess this process according to ideal patterns of collaborations, and provide feedback to peers. In particular, this enhanced formative feedback aims at diminishing the possible dissonance between the individual collaborative skills by challenging self-adjustment procedures. This proposal has been evaluated in many contexts like law and music learning domain fields and has been proven to promote collaboration.
Martin and Carro [ 73] present a work from the field of m-learning attempting to touch the area of adaptive CSCL environments. In their system, activities are dynamically recommended to users depending on different criteria (user traits, context, etc.) and workspaces are dynamically generated to support the corresponding activity accomplishment. A noticeable outcome is that recommendations of activities to be accomplished helped students mostly to adequately manage their time for learning, especially, in situations where they had not much time available. Student motivation also increased, not only for learning but also for discussing with their partners about theoretical explanations, examples, or exercises proposed by the environment. Learners found recommendations based on their personal features, previous actions, and context useful. In this work, technologies that are used include AI techniques like rule-based recommendation and Markov-model-based recommendation.
A non-AI technique for capturing and representing collaborative activities and interactions is used in Soller's work [ 74] by modeling specific characteristics exhibited by effective collaborative learning teams (i.e., participation, social grounding, performance analysis and group processing, application of active learning conversation skills, and promotive interaction). The system can take actions to help students collaborate more effectively with their peers depending on deviations from the aforementioned characteristics of effective collaborative learning teams. The study results suggest that structured knowledge of student conversation in context may be sufficient for automating the assessment of group interaction, thus making it possible for the system to behave intelligently and adaptively.
A non-AI technique is also employed in the work by Gogoulou et al. [ 75]. Their Adaptive Communication Tool (ACT) system supports structured dialog using two types of Scaffolding Sentence Templates (SSTs): sentence openers or communicative acts. The system adaptively provides suggestions for the supported form of dialogue and SST type and also for the most meaningful and complete set of SST with respect to the learning outcomes. Preliminary research results indicate—among others−that student users were satisfied by the capability of personalizing their communication by selecting the desired communication means and extending the SST set.
Soller and Lesgold in [ 76] used Hidden Markov Models (HMMs) and Multidimensional Scaling to analyze and assess sequences of coded online student interaction in an attempt to identify when and how problems occur in a collaborative process. Research results suggest that methods that do not account for the sequential and stochastic nature of student interaction perform less well than the HMM approach.
In [ 77], the authors present a web-based system designed to enable teachers and researchers to create, enact, and test CSCL scripts in actual educational settings. They present an approach and potential for adaptive collaboration scripting, aimed to tailor the activity according to students' performance, personal characteristics, and preferences. In their system implementation, they distinguish between design-time and on-the-fly adaptations and present encouraging preliminary results concerning learning outcomes and system usability.
The results from studies reviewed suggest that:
In this section, we focus on the types of modeling techniques and types of software (mainly) technologies that AICLSs employ. We present modeling in conjunction with technology, as modeling and technology are not decoupled in the reviewed AICLSs. Also, technology in many cases is strongly connected to the way modeling is accomplished.
In general, adaptive systems for individual learning include (Stephanidis et al., [ 78]): 1) modeled entities on which the decisions on adaptation are based (e.g., user preferences), 2) adaptation constituents (the aspects of the learning environment that are subject to adaptations, e.g., navigation tools), and 3) adaptation rules (set of rules defining how to implement adaptations, that is, connecting the state of determinants to the state of constituents). Intelligent systems also utilize computational models to model both the solution as an outcome and the problem-solving process as states of a system targeting this outcome.
In AICLS systems, many different models are used in order to represent various entities of the collaborative learning setting (i.e., an individual, a group, a solution path, or a collaboration process), as no widely adopted standards exist. We suggest that AICLS systems can also be classified considering their adaptation determinants, constituents, and rules.
Reviewed AICLSs were systematically analyzed from the above perspective and classified depending on their focus of modeling, as follows:
It should be mentioned that the classification above simply reflects the emphasis that the various studies are based on specific modeling implemented in the system, without implying that other type of models are not implemented in the system.
The various modeling types across the target of intervention of the system are also presented in Table 2. Thus, AICLSs adaptation determinants (models) are linked to the adaptation constituents (target of intervention).
In many of the above systems, AI techniques are employed to perform modeling, such as Bayesian Networks and Fuzzy Networks [ 22], [ 23], [ 27], [ 26]. Regarding the system knowledge representation, all systems that model the problem-solving process aim to adaptively support the domain learning and use intelligent technologies (such as [ 17], [ 23]). Respectively, in systems that offer peer interaction support one identifies at least data mining technologies for interaction analysis plus some inference mechanism in order to provide supportive prompts to students or teachers (e.g., [ 66], [ 67]).
Worth mentioning is that although during our investigation we came across efforts for group modeling, none of them is standardized and very few use these models nor provide evaluation results of a working system [ 79], [ 39]. However, further analyzing the group modeling area is beyond the scope of this review.
From the technological perspective, we suggest that AICLS systems can be classified in a continuum (from AI to NON-AI-based systems) depending on whether the system employs computationally intensive data processing methods traditionally rooted in the AI field (see Table 1). Systems that employ simple models and algorithms are considered NON-AI technologies. Certain conclusions can be drawn from Table 1:
Walker et al. [ 14] have proposed that the AICLSs design space may vary in two distinct dimensions: 1) the Explicit/Implicit dimension (whether the action that students should take is explicitly described in the feedback or implicitly arises as a result of the support) and 2) the Direct/Indirect presentation (whether the instruction is presented directly to the partner it targets or presented indirectly to another partner or through a change in the learning environment). The authors also conclude that the most of systems can be considered as providing Explicit Instruction(s) and Direct Presentations (EI/DP). For example, clinical reasoning support in COMET ([ 17]) is directly presented to learners in the form of explicitly stated instructions/hints. By contrast, Walker et al. [ 59] present an example toward the alternative design direction, where peer tutoring—based on agent technologies—is used as a basis for implicit feedback to learners (that is, instead of providing feedback directly to learners who need help, to present hints to their peers in order to help them).
The majority of the systems we studied provide the typical explicit guidance and direct feedback to the learner. However, we believe that such a user-oriented view is valuable for the field as it helps toward tracing the possibilities opened by systems that provide, for instance, implicit feedback and indirect presentation to the user. Put simply, there is an open research space for developing adaptive and intelligent CSCL systems that provide a hint to the partner in order to support the target user indirectly, instead of providing a direct help to the user. This implies more complex forms of AICLS environments and enactments.
So far, we have presented results from reviewing group-1 papers that report student-based evaluation data of an AICLS system. In this section, we review group-2 and group-3 articles that focus on integrated frameworks presenting comprehensive approaches for adaptively supporting the CSCL activity, AICLS system architectures, and integration of current advanced technologies in the development of an AICLS system.
These articles do not convey to the reader student-based evaluation data of some system. The key criterion for including them in the review is that they reflect current innovative research efforts that shape the advances of the field in the near future.
Certain articles provide detailed presentations of AICLS system architectures (“architectures”) while others propose integrated approaches that combine various methods and tools to adaptively support CSCL activities (“frameworks”).
A typical framework-type article is [ 33] where aLFanet system represents an implementation of the named Collaborative Learning Framework (CLF). CLF aims at adaptively supporting CSCL process by controlling how the issues that cannot be predicted at design time should be managed on runtime. In this work also, we track an effort of combining standards (i.e., IMS-LD) with other technologies (e.g., agents). According to authors, applying the CLF promotes the collaboration among the members of a group interacting in a course.
<e-adventure> [ 34] is a working system tested in a modified version of the official service-oriented implementation of the CopperCore engine. The system has been developed at the Open University of the Netherlands, to extend the IMS-LD specification in order to represent also game-based learning activities, however, no user-based evaluation results have been publicly reported. In <e-adventure>, the learner profile (based on Vermunt's model for learning styles [ 5]) is used to adapt game settings, and—reversely—the activity in the game provides the basis for further adjusting the user profile.
“Remote-Control” [ 80], is an architecture for the integration of tutoring scaffolds into existing collaborative applications. The architecture allows integrating IMS-LD-based representations of the collaborative activity into existing and tested collaborative learning environments like Cool Modes [ 81]. Also, the architecture enables controlling the learning environments either by a human or a technological pedagogic agent.
In [ 82], the implemented framework system provides simple recommendations (e.g., tell the students to visit a forum, or alert the teacher that a user seems to fail the course). These recommendations can be enriched based on information that the system collects about the student's background (such as level of expertise in the use of the services, preferences, and interests).
Though most frameworks remain independent of the learning domain, some works (e.g., [ 83], [ 84]) present frameworks specifically oriented to a domain. However, the proposed implementation of these frameworks caters also for a “plug and play” capability to extend the application of the framework to other learning domains.
A relatively new framework is introducing adaptation patterns as an abstraction of adaptation strategies in CSCL settings. Adaptation patterns are extracted mainly by analyzing teachers' flexible interventions to address students' needs when engaged in collaborative learning activities [ 85]. An initial architectural implementation using IMS-LD has been presented in [ 32].
Finally, certain efforts have been focused on integrating student assessment activities in the adaptive design of CSCL scripts. The objective of the proposed system is to enable the designer/teacher record student assessment data and adapt the next steps of the collaborative activity based on the data [ 86].
Integration of semantic technologies in the design of AICLS systems constitutes a significant current trend in the field and there are already many works suggesting the use of ontologies to formally represent key aspects of AICLSs (e.g., [ 39], [ 87], [ 88]). However, none of these ontologies has a level of maturity sufficient of being widely accepted. Anaya and Boticario [ 46] argue that we lack formal semantics to create and use ontologies that represent various types of knowledge relevant to personalized adaptive learning.
A representative work in this area by Isotani et al. [ 39] presents a broad ontology integrating several key entities of the collaborative learning activity, which facilitates group formation and collaborative learning design. Evaluation results suggest that this ontological framework facilitates the effective design of group activities and can positively affect the performance of individuals during group learning. Semantics have also been used in combination with web services as in [ 87]. The presented ontology is the basis of a service discovery facility that is developed in order to allow educators to flexibly search service-based CSCL tools that fit to their learning activity design.
Several other works attempt to represent various aspects of an AICLS with ontologies, for example, ontology for the teaching model expressed in IMS-LD [ 88], for assessing the learning interactions (e.g., gSTudy system in [ 89]), ontology-driven inquiry of learning scenarios [ 90], ontology for domain models built by user tags (e.g., MOAT [ 91]), and user models (e.g., FOAF in [ 92]). The issue, however, with semantics is twofold: 1) the constant need for ontology evolution (maintenance and updating) and 2) the knowledge sharing, reuse, and exchange among several different AICLSs covering the same or similar domain, without the help of a human indexer and performed by a system in an open corpus of documents in web, as suggested by Brusilovsky and Nejdl [ 93].
Jovanović et al. [ 94], in reviewing the use of semantics in intelligent learning environments, suggest that the integration of the Semantic Web technologies into the social web paradigm promises the solution of the above problems. The social web can be seen as a web of collective knowledge systems (social networks), which are able to provide useful information that is based on human contributions (folksonomies) and which improve as the number of people participating increases [ 95], [ 96], [ 97]. Currently, one of the major obstacles for collaborative creation and sharing of knowledge in the social web is the fact that in these online social networks, knowledge can be exchanged within the networks but not across them, at least not without a significant effort (i.e., manually copy-and-paste folksonomy elements). The idea of merging the best of the worlds of semantics and social web has converged in the concept of the Social Semantic Web, in which socially created and shared knowledge on the web leads to the creation of explicit and semantically rich knowledge representations [ 94]. Social Semantic Web satisfies both requirements of an AICLS, that is, adaptive behavior through engagement of learners in a folksonomy/Web 2.0 oriented systems and intelligent operation driven by reasoning of ontological/semantic automata.
In the direction of social Semantic Web, Soller [ 98] discusses the challenges that emerging web-based technologies face when trying to enable distributed users to discover and construct new knowledge collaboratively. One key example is the advanced collaborative and social information filtering technology that not only helps users discover knowledge, peers, and relevant communities, but also plays a powerful role in facilitating and mediating their interaction. This technology supports adaptive system behavior as it could be used, for instance, for group formation purposes. It also depicts intelligence as it may guide (i.e., recommend) peers to collaborate and form groups based on data collected constantly by the users occupying the system. Following the same perspective, Kruk et al. [ 99] present the Social Semantic Collaborative Filtering (SSCF) method which allows users to easily share their knowledge with others within and across online social networks. For example, one could easily import friends' bookmarks and utilize their expertise and experience in specific domains of knowledge.
Ubiquitous learning places emphasis on both the learning context and mobile/ubiquitous context [ 100]. In relation to AICLS systems, the reviewed articles demonstrate current efforts to combine the advantages of adaptive systems with the benefits of context-aware systems and flexible mobile technologies.
In [ 101], learning and user context is discussed as affecting and at the same time being affected by CSCL activities. The authors present an innovative approach for modeling a “two-way” adaptation between context and learning activities on the basis of mutual influence between them in learning processes. An interesting proposal for the use of a knowledge awareness map is made in [ 102]. The map visualizes the relationship between the shared knowledge and the current and past interactions of learners. The map plays a very important role in finding peer helpers and inducing collaboration. Important is also the suggestion that ubiquitous learning, especially, when viewed through the CSCL perspective, demands ontology-based frameworks for capturing and representing learning context data (e.g., [ 100]).
Our approach in reviewing the AICLS systems has been twofold: first, we analyzed the key design aspects of AICLS systems commenting also on the reported impact of these systems on student learning. Second, we highlighted current innovative research efforts in the field. Below, we summarize and comment on our major findings from three perspectives: the classification scheme, the impact on student learning, and the emerging research opportunities.
The classification scheme of AICLS systems has been proposed as a tool for a structured analysis of their major design characteristics. In Fig. 1, we present a comprehensive diagram integrating all aspects of the scheme. The diagram illustrates the CSCL process from an adaptive and intelligent system perspective as unfolded in two major layers and several components.
Figure Fig. 1. A layered representation of the AICLSs classification scheme.
AICLS systems employ various AI and non-AI methods and technologies (column far right) to construct and update necessary modeling components regarding various aspects of the collaborative activity (like user, group, interaction, solution, goal(s), etc.). Supportive intervention (adaptive or intelligent system operation) is presented to learners after application of specific rules and filtered by pedagogical strategy and design space. The intervention of an AICLS can target either of two layers:
Our data suggest that learning benefits do not emerge unconditionally when using AICLS systems to support collaborative learning. Learning impact is subject to learning design and capability of AICLS to adapt and intervene in an unobtrusive way. Significant research efforts focus on providing PI-type support which seems to motivate students and improve collaboration and learning. Also, there is a lack of coherence in assessing AICLSs learning impact, since no common benchmarks have been agreed upon, making it almost impossible to compare the efficiency of using different methods in AICLSs for supporting the same target of intervention.
Regarding group formation, analysis of the relevant studies suggests that heterogeneous grouping can increase the possibility to yield beneficial learning outcomes. Finally, PI-type support seems to motivate learners regarding their collaboration and quality of learning, however, the majority of the studies (21 out of 32) are not concerned with formally evaluating student learning but focus primarily on assessing the effectiveness/efficiency of the implemented peer interaction modeling and scaffolding techniques.
Promising research directions as demonstrated in the reviewed articles include:
Group formation. Various techniques can be employed to support group formation and comparatively analyzing their efficiency is an open research question [ 39]. Moreover, peer matching based on both the interactive solving process that peers follow and their prior domain knowledge seems to result in beneficial collaboration (e.g., MATHEMA system in [ 24]). This approach can be combined with semantic techniques (like [ 39]) leading to an innovative research direction.
Interaction analysis for PI-type support. PI-type support still poses significant questions that should be answered by research in the interaction analysis area. Although research has already been conducted in this field [ 52], [ 50], [ 53], [ 51] there are no widely accepted standard(s), set of indicators, or interaction patterns and there is opportunity for proposals, improvements, and evaluations.
IMS-LD extensibility. Extending (or otherwise empowering) the IMS-LD specification to allow expression of powerful adaptive designs in CSCL systems is a significant research road map [ 32], [ 103], [ 104]. Relevant studies attempt to develop an appropriate framework enabling the linking between IMS-LD-based applications and tools for Interaction Analysis (e.g., [ 80]).
Assessment in AICLS systems. Student assessment as a means to adapt the CSCL activity and the design of systems that enable adaptations according to collected assessment data is a research challenge [ 86]. Moreover, in reviewed AICLSs and especially in D-type support systems the assessment of student knowledge is almost always focused on the individual; thus, other methods of assessment—including more complex models of the group and aspects of learning domain and collaboration process is considered a research issue [ 7].
Design space. Altering the adaptation perspective from explicit/direct to implicit/indirect designs also opens a new area of experimentation exploring the possible benefits when implicit/indirect feedback is provided to learners either through peers or through agents of the system [ 9], [ 83].
Frameworks/architectures incorporating Web 2.0 tools. Researchers can develop frameworks for end users that address context-specific learning objectives. Frameworks should facilitate reusability and exchange of end-user produced Web 2.0 components/services with other educators, students, and software developers (one way to accomplish this technologically is by using grid architectures, e.g., [ 104]). Evaluation studies in real-world settings for testing the frameworks' validity constitute also a research issue.
Social web tools. Tooling support is still at the level of research prototypes and further studies are needed for testing the scalability of the proposed software solutions [ 94]. Social web tools are mostly concerned with informal learning and integration in more disciplined CSCL learning design. In an AICLS system, the integration procedure of the social web tools requires further investigation and is a compelling research issue.
Semantic social web (SSW). Social web environments enriched with semantics appear to be a significant opportunity for introducing adaptive and intelligent behavior in AICLS systems [ 94]. In such a system, semantic annotation of resources and/or activities can be based on collective user contributions. The system ontological representations can be constantly verified and updated by use. For example, a system that employs collaborative filtering (CF) technique to present hints or recommendations to learners, exhibits: 1) adaptive behavior as the recommendations may change depending on learner-based filtering and b) intelligent operation through reasoning based on formal ontologies derived from evolving folksonomies [ 105].
Summarizing the above, we argue that key items in the current research agenda of the field are: 1) Semantic social web technologies and their application in adaptive and intelligent CSCL settings, 2) extension of CSCL standards to permit decoupled communication with external components/services, and 3) benchmarking and agreement upon common peer interaction indicators to efficiently assess and model the “learning to collaborate” and “collaborate to learn” constituents of CSCL.
The authors wish to thank the IEEE Transactions on Learning Technologies' Associate Editor, the special issue guest editors, and the anonymous reviewers for all their recommendations, suggestions, and support during the review process of this paper.