1. Synthetic Generation. Each learning principle is articulated through one or more plan-based templates to allow automatic generation of game play elements that embody that principle, rather than expensive and time-consuming human-authored design elements.
2. Dynamic Adaptation. A secondary advantage of synthetic generation is that it allows for generation to be performed at runtime, where the game can dynamically adapt to the behaviors exhibited by the student.
3. Quantifiable Extent. With the ability to generate and adapt, as needed, comes the capability to measure or specify the frequency and extent to which learning principles are realized. In other words, it provides researchers with a mechanism to freely vary the prevalence of one principle versus another and measure the effects.
5.2.1 Discovery Principle “Overt telling is kept to a well-thought-out minimum, allowing ample opportunities for the learner to experiment and make discoveries.”
Quantifiable Properties. We use the term remediation to describe an action Annie inserts into the game environment to attempt to correct what it perceives to be a misapprehension on the part of the student. We can count the number of remediations applied for each student, the best-case, worst-case, and average number of remediations required for each particular knowledge component, and the comparative frequency of stronger or weaker hints that correspond to different type of remediations. Across a broad range of students, these measurements can be used to characterize the difficulty of different parts of the game world and help pinpoint areas where more student guidance opportunities may be required.
Computational Pattern. Remediations are organized in such a way as to allow software to choose between successively more explicit modes of instruction. This builds on extensive ITS research into the optimal selection strategy between the frequently used guidance options of “Prompt,” “Hint,” “Teach,” or “Do.”
5.2.2 Multiple Routes Principle “There are multiple ways to make progress or move ahead. This allows learners to make choices, rely on their own strengths and styles of learning and problem solving, while also exploring alternative styles.”
Quantifiable Properties. Annie can quantify the number of distinct successful plans, the number of qualitatively different plans in the plan space, the number of actions that must be included in any successful plan, or even the ratio of the number of these critical actions to the mean total number of actions in successful plans.
Computational Pattern. Annie allows for extensive mining of the space of potential plans to reveal bottlenecks, potential for off-task activity, etc., in a way that could be much cheaper and more extensive than traditional game design play testing strategies.
5.2.3 Explicit Information On-Demand and Just-in-Time Principle “The learner is given explicit information both on-demand and just-in-time, when the learner needs it, or just at the point where the information can best be understood and used in practice.”
Quantifiable Properties. The timeliness of explicit information can be measured by the duration between when the information is provided and when it is needed. This can be compared and contrasted with the number of opportunities for on-demand information in the environment. For some students or groups of students, Annie may want to vary how far in advance help can be provided, based on projected memory persistence of those students. As post-hoc measurements, analysis of these properties over many students can be used to calibrate the guidance within Annie.
Computational Pattern. As described in later sections on system implementation, the Annie system continuously calculates the immediacy of information requirements in terms of proximity of plan operators in successful plans.
5.2.4 The Other Six Principles Space does not allow for a full description of the remaining six of Gee's principles that Annie currently implements. However, they are briefly listed and described below:
1. Incremental Principle. Orders challenges so that complex situations build on earlier, simpler examples.
2. “Regime of Competence” Principle. Provides only as much support as the student needs to avoid frustration.
3. Semiotic Principle. Uses a broad range of sign systems to communicate pedagogical content.
4. Achievement Principle. Provides intrinsic rewards, customized to individual performance to signal mastery.
5. Practice Principle. Provides paths to success that allow for repetition and even failure.
6. Transfer Principle. Varies the levels of specificity and generality in learning content.
6.3.1 Remediation Templates Assume that Annie has selected the following knowledge gap for remediation:
This means that Annie wants to increase the student's strength of belief that the delete-file operator has a precondition that the file cannot be inUse. Annie chooses a remediation template from the library to apply to the plan. The following example shows one possible instantiation of a demonstrate remediation template:
This fairly complex example was chosen to highlight that the remediation templates are not simple atomic activations, but partially specified plan structures that Annie must dynamically weave into the particular state of the particular session to achieve defined pedagogical goals. The template is “partially specified” in that it contains place holders that allow Annie to find the right combination of operators and ground terms to bring about the intended changes.
6.3.2 System Initialization At initialization, Annie uses the external template libraries, the problem description, and the rest of the LPD to compile its runtime knowledge base. The first task is to initialize the knowledge base with the problem description amended with pedagogically focused elaborations of each operator. Annie uses these elaborations to represent a student's state of belief for each precondition and effect of each operator in the planning problem description. Included in this construction phase is the compilation of all the remedial and diagnostic templates to represent beliefs, specific to the individual operators of the domain.
The final step of Annie's initialization is creating a tutorial plan, consisting of a plausible, partially ordered sequence of student and system-initiated action that is guaranteed to bring about a specific goal state for the world. Annie then uses heuristics, derived from its knowledge base, to choose the initial plan from the set of successful plans. These heuristics will consider the student's specified proficiency, plan adaptability, as well as traditional measures of plan structure. The planner generates a set of successful plans as well as a wider space of possible plans, both complete and incomplete. Annie uses this plan to initiate the tutorial session within the game world. Because the planner is only concerned with the states of the world, the tutorial plan does not guarantee that the goal beliefs for the student will be achieved. Rather, Annie monitors student behavior and optimizes the frequency and extent of its tutorial interventions to increase the likelihood that the goal beliefs are acquired.
6.5.1 Remediation Consideration: Calculate the Mean Goal Proximity Ratio (MGPR) The MGPR provides a rough estimate of how close the student is to completing any of the remaining potentially successful plans. It is calculated as the mean of the Goal Proximity Ratio (GPR) of all such plans. For a given plan, the GPR is given by
Annie uses the MGPR as an estimate for the remaining number of useful tasks consistent with successful plans. This helps determine the urgency with which Annie guides a student toward tasks on known paths to success.
6.5.2 Remediation Consideration: Identify and Rank the LPD Gap Set The LPD Gap Set is the subset of the LPD beliefs that are not currently satisfied. Annie ranks each gap according to the distance between its current belief strength and its target belief strength, given in the LPD.
6.5.3 Remediation Consideration: Identify and Rank the Proximal Belief Gap Set The Proximal Belief Gap Set (PBGS) is the subset of the beliefs about the proximal actions in the set of possibly successful plans. These are ranked according to belief distance, just as with the LPD gap set. In addition, each belief is ranked according to the proximity of the action that contains the belief.
6.5.4 Remediation Consideration: Select a Gap that Exceeds Threshold Empirically derived thresholds are applied to the beliefs in each of the gap sets to decide which beliefs, if any, should be remediated in this execution cycle. If none of these exceeds the thresholds, Annie considers the LPD gap set. These thresholds will be calibrated so that Annie's proclivity for remediation is low. In most cases, no candidate remediation will exceed the urgency threshold, and Annie will proceed directly to Stage 3. Only when a gap is chosen for remediation will Annie proceed to Stage 2 to revise the plan.
There are two key reasons for Annie's remediation reticence. First, the student needs to be in charge. Annie's interruptions will become increasingly transparent and annoying as they break the student's flow and sense of being in charge of the learning experience. Second, Annie must respect the uncertainty, inherent in its projection of the student's plan. Annie runs the risk of remediating gaps in actions that are not even required for the student's plan to succeed. This could result in Annie inefficiently oscillating between several candidate plans forcing the student to adjust to an incoherent sequence of Annie's actions.
6.6.1 Plan Revision: Remediation Selection The primary focus of the plan revision stage is for Annie to adopt tutorial plans to the needs of the student by selecting from the remedial operators, whose knowledge outcomes address the knowledge gap selected in the previous stage. The simplest variety of remedial operators correspond to common ITS interventions including prompt, hint, demonstrate, teach, or do. In addition, Annie can be provided with hand-authored composite remedial operators, particularly attuned to common misconceptions for a particular domain. If more than one remedial operator is found, Annie considers the knowledge outcomes and prerequisite knowledge that annotate each alternative, comparing these to the current state of knowledge represented by the student model and the selected knowledge gap. In addition, Annie considers the student's proficiency and progress history to determine how aggressively the knowledge gap should be remediated. Our initial work uses a simplified selection algorithm, similar to the “Fixed Strategy” studied by Murray and vanLehn [ 33].
6.6.2 Plan Revision: Subplan Formation Subplan formation is simply the instantiation of the selected remedial operator (likely an abstract, or hierarchical operator) with details from the current plan context.
6.6.3 Plan Revision: Subplan Integration Once the new remediation subplan has been created, it must be integrated into the current plan. Our work extends the DPOCL planning algorithm [ 34] at the heart of Zócalo with a new function capable of wedging a new subplan into an existing plan at a particular point (i.e., so that the subplan is ordered to precede a particular action in the original plan).
6.6.4 Plan Revision: Alternative Plan Consideration In the final step, Annie considers abandoning the currently active and newly remediated tutorial plan in favor of another potentially successful plan in the plan space, if it offers a better fit for the current plan history. Remember that Annie is largely a spectator, and its active plan is merely a guess at a likely course of action for the student. If the student's actions show a pattern of increasing divergence from the current plan, coupled with increasing similarity to an alternative plan, it is rational for Annie to adopt the alternative plan rather than to continue to steer the student back onto a plan that the student has chosen not to follow.
6.6.5 Plan Revision: Example To show how Annie would traverse the four steps of the plan revision stage for a typical case, we return to the example of the operator, named deleteFile, depicted in Fig. 2 in Section 4.1. We assume that the current plan calls for the student to perform a and then , and that Annie has identified the knowledge gap
This means that Annie is concerned that the student does not know that delete-file operator has a precondition that the file cannot be inUse. In Step 1, Annie consults the set of remedial operators defined for “Unknown Precondition.” Based on the current state of student learning, Annie might choose the demonstrate remediation.
In Step 2, Annie instantiates the remedial operator, so that it forms the following subplan:
In Step 3, this subplan is merged into the current plan, such that the subplan will complete prior to the action already in the plan that required the student to . In Step 4, Annie may or may not choose to abandon the current plan in favor of an alternative.
This example also illustrates some of the challenges in writing sufficiently general operator templates that produce predictable pedagogical impacts. Clearly, some cleverness is required on the part of the template writer. As Annie is applied to different domains, we expect to discover some best practices for the confluence of knowledge representation and template construction.
6.9.1 Diagnostic Templates: Scenario Descriptions Each of the diagnostic templates in Annie's library corresponds to a particular scenario that describes the relationship between the most recently executed actions, the plan space, and the rest of Annie's knowledge base. We describe each of the scenarios below. For one of the scenarios, we provide a detailed illustration.
Failed action. When an action, that the student attempts, fails, there is a good probability that an underlying misconception can be identified, because Annie can identify at least one precondition that was not established at the time the student's action was performed. This could be caused by a precondition that has never become true, since the beginning of the session, or, as shown in Fig. 7, where a student's attempt to execute action fails because precondition , which was established by action Action , but subsequently negated as an effect of action . Table 1 provides a brief logical description of five possible misconceptions Annie evaluates in this scenario.
Successful, butfatally flawedaction. The action may be predicted to succeed, but Annie may detect that its execution will eliminate all possible successful plans from the plan space (e.g., the student just destroyed an irreplaceable resource required for successful completion of the plan).
Successful, butpath-limitingaction. A successful student action eliminates some but not all possible successful plans from the plan space (e.g., student burned a bridge).
Successful, buttask-irrelevantaction. An action that does not limit future success may reflect a misconception in the student model, if it does not advance the student down a path of success. We assume that the student is performing tasks with an intention of reaching the goal. If an action does not result in progress toward that goal, perhaps the student holds mistaken beliefs.
Successful, plan-relevant, butsuboptimalaction. Even though an action succeeds and is on a path toward the goal, it may introduce problems that result in a less-optimal plan (e.g., one that requires more actions than would otherwise be required).
Successful, plan-relevant, optimal, butthreat-ignorantaction. An action that will be required as part of any successful plan, but does not resolve an obvious immediate threat, may indicate a gap or error in the student model. Perhaps, the player is unaware of the sniper, because he is reloading his gun instead of scanning the roof tops for muzzle flashes.
Successful, plan-relevant, optimal, threat-aware action. In this situation, the student has chosen an optimal action.
Inaction. This is the case where the student has not performed any action.
6.9.2 Diagnostic Templates: Sources of Belief Each student belief in a diagnostic template is classified according to the five components of the student model it affects: the Operator Schema, the Initial State, the Goal State, the Plan History, and the student's understanding of future possibilities described by the Plan Future. For the diagnostic template depicted in Table 1, the first misconception concerns the Operator Schema, and the remaining four misconceptions correspond to errors in the student's understanding of the possible Plan Future.
6.9.3 Student Model: Updating Beliefs Each time a student-initiated action succeeds or fails, Annie considers all of these scenarios in determining which beliefs should be updated in the student model. Where the scenarios describe positive beliefs, Annie will strengthen the associated belief in the student model. Where the scenarios describe negative beliefs, Annie will weaken the associated belief strength in the student model.
1. Kill the harmful child process.
2. Delete startup file that infects the parent process.
3. Kill parent process (child process' hidden respawner).
The authors are with the Department of Computer Science, North Carolina State University, Engineering Building II, 890 Oval Drive, Raleigh, NC 27695-8206. E-mail: firstname.lastname@example.org, email@example.com.
Manuscript received 1 Dec. 2009; revised 23 Mar. 2010; accepted 24 Sept. 2010; published online 13 Oct. 2010.
For information on obtaining reprints of this article, please send e-mail to: firstname.lastname@example.org, and reference IEEECS Log Number TLTSI-2009-12-0175.
Digital Object Identifier no. 10.1109/TLT.2010.32.
1. The plan-based interactive narrative representation used by Zócalo is described more fully in [ 24].
James M. Thomas received the BA degree in mathematics in 1983 and the MEng degree in computer science in 1986 from the University of Virginia, Charlottesville. He is currently working toward the PhD degree in computer science as a US National Science Foundation graduate research fellow at North Carolina State University, Raleigh. He has worked at IBM, BNR, Nortel Networks, and 3-C ISD. His research interests include intelligent tutoring systems, plan-based knowledge representation, social skills training, and games-based learning. He is a student member of the IEEE.
R. Michael Young received the BS degree in computer science from the California State University at Sacramento in 1984, the MS degree in computer science from Stanford University in 1988, and the PhD degree in intelligent systems from the University of Pittsburgh in 1998. He has worked at Hewlett-Packard, FMC Corporation, Rockwell, and as a postdoctoral fellow at Carnegie Mellon University. He is currently an associate professor of computer science and a codirector of the Digital Games Research Center, North Carolina State University, Raleigh. His research interests include the computational modeling of narrative, planning, games-based learning, and automated cinematography. He is a senior member of the IEEE.