Despite the large body of research, which suggests a high correlation between home tutoring and a child's academic success [ 1
], [ 2
], parental involvement in learning activities at home may not occur spontaneously due to self-believed lack of ability, knowledge, and skills [ 3
]. Research suggests that parents with high levels of self-efficacy tend to make positive decisions about active engagement in their child's education, while parents with weak self-efficacy are often associated with less parental involvement [ 4
], [ 5
]. Therefore, endowing intelligent tutoring systems with the ability to adapt the level of support provided for the parent based on their self-efficacy may be of great benefit. Such a system might provide high levels of support for parents with low self-efficacy, while providing lower levels of support for parents with high self-efficacy.
The Parent and Child Tutor (P.A.C.T.) is an intelligent tutoring system that endeavors to address the challenge of building an adaptive system, which simultaneously supports both parent and child in the home-tutoring environment. First, P.A.C.T.'s design uses a novel dual-user architecture that comprises dual-user models and domain models. Second, P.A.C.T. provides an instantiation of Talent Education philosophy, which defines tutoring best practice. Third, using two levels of adaptivity, P.A.C.T. provides varying levels of support based on parental self-efficacy while simultaneously providing affective support for the child.
This paper focuses primarily on the effect of P.A.C.T. on parental self-efficacy. In particular, it reports on two complementary empirical studies. In the first study, a dynamic self-efficacy model, learned from runtime self-report data, is used to provide adaptive support for the parent. In the second empirical study, the dynamic self-efficacy model was expanded to allow parents to request further support outside that deemed necessary based on their self-efficacy model. Both studies comprised a control group which received full support regardless of their self-efficacy throughout the entire experiment. Results indicate clear increases in parental self-efficacy as a result of the provision of adaptive support throughout the home-tutoring process. In particular, results highlight the dichotomy between parental self-efficacy and desired levels of support. More specifically, parents reported high levels of self-efficacy but still desired high levels of support.
This paper is structured as follows: Section 2 discusses the role of self-efficacy in learning and how intelligent tutoring technology has been applied thus far. Additionally, it describes one learning context that might benefit from such an approach, which is the area of home tutoring. Section 3 describes P.A.C.T. and how its architecture allows for the provision of adaptive support based on parental self-efficacy. Sections 4 and 5 describe the two empirical studies conducted to investigate the effect of P.A.C.T. on parental self-efficacy. Section 6 provides a discussion or results and concludes outlining the next steps in the research journey.
Bandura believed that the beliefs people have about themselves are the key elements in their ability to achieve desired outcomes [ 6
], [ 7
]. Consequently, he believed that how people behave might be a result of their beliefs, and therefore, behaviors may be better predicted by these beliefs than by the result of their previous performances. Research has indicated that self-beliefs may have an impact on cognitive engagement, which suggests that enhancing self-beliefs may have a positive effect on learning [ 8
]. An individual's self-efficacy is included in such self-beliefs.
2.1 Sources of Self-Efficacy
Self-efficacy beliefs are developed from four sources: mastery experiences, vicarious experiences, verbal persuasion, and physiological state [ 6
]. Mastery experiences are arguably the most influential source of these beliefs. Successful experiences boost self-efficacy, while failures erode it. However, easy success can lead to an expectation of quick results therefore leading to a lack of resilience when faced with challenges [ 6
]. The second source of self-efficacy beliefs are vicarious experiences. Observing similar individuals succeed by sustained effort raises observers' beliefs that they too possess the capabilities to master comparable activities required to succeed. Pajares [ 9
] suggests that this is a weaker source of self-efficacy than mastery experiences; however, if individuals have little previous experience with the given task, they may become more sensitive to it. The impact of success or failure of the observed on the perceived self-efficacy of the observer is strongly influenced by their perception of the extent of their similarity. If the observers believe their capabilities are superior to the observed then failure of the model does not have a negative effect [ 10
]. Verbal persuasion is based on the premise that verbal judgments that others provide have an influence on self-efficacy beliefs [ 11
]. Bandura [ 6
] suggests that it is often easier to weaken self-efficacy beliefs through negative appraisals than strengthen them through positive appraisals. Finally, physiological states such as anxiety, stress, fatigue, and affective states provide insights on self-efficacy beliefs, where positive moods enhance perceived self-efficacy and despondent moods diminish it [ 7
]. It is important to note that with all the aforementioned sources of self-efficacy, it is the interpretation of the experience that forms the judgments of self-efficacy, and not that the sources themselves are directly translated into judgments of competence.
2.2 Measuring Self-Efficacy
Self-efficacy is not a global trait but a differentiated set of self-beliefs linked to distinct sets of tasks. The "one measure fits all" approach to self-efficacy provides little valuable insight and poor predictive value [ 12
]. It is important to measure self-efficacy in terms of perceived capability, that is, in terms of can do
rather than will do
. The development of effective self-efficacy scales relies on a deep analysis of the domain of functioning. Identifying which aspects of self-efficacy should be measured arises from a thorough understanding of the domain. If scales are created around factors, which, in fact, have little or no impact on the domain of functioning, the results will have little predictive power.
Standard methodology for measuring self-efficacy involves presenting individuals with different tasks and requesting a measure of their belief in their ability to execute the tasks. Bandura [ 12
] suggests the use of a 100-point scale, ranging in 10 units from 0 ("Cannot do") through 50 ("Moderately certain can do") to 100 ("Highly certain can do"). The scale can also be collapsed to a 0-10 scale. Scales with too few steps should be avoided as individuals usually avoid extremes; using a scale with few steps could easily result in it shrinking to one or two points. Self-efficacy judgment should be recorded in private in an endeavor to reduce evaluative concerns.
2.3 Self-Efficacy and Intelligent Tutoring Systems
Research has reported that students' self-efficacy beliefs are correlated with their academic performance [ 13
]. However, limited research exists, which investigates the role of self-efficacy within intelligent tutoring systems. To date, the majority of research has explored techniques for eliciting self-efficacy with little focus on how learning environments might use such information to provide personalized support for the learner. Two recent efforts have explored the possibility of eliciting and predicting self-efficacy values at runtime.
Bica et al. [ 14
] investigate the possibility of inferring self-efficacy through user interaction with the system. A model of self-efficacy is employed, which comprises effort, persistence, and performance. Persistence is calculated based on the percentage of selected tasks completed. Performance is calculated based on the mean of correct answers in the exercises. A set of inference rules, comprising fuzzy logic, is used to determine the student's self-efficacy, where self-efficacy is categorized in terms of low, medium, or high.
McQuiggan and Lester [ 15
] describe an inductive approach for automatically constructing models of self-efficacy that can be used at runtime to inform pedagogical decisions. This research concludes that induced decision tree models that learn from demographic data and data gathered with a validated self-efficacy instrument administered prior to problem solving and learning episodes can make reasonably accurate predictions about students' self-efficacy. In addition, if runtime physiological data is available, it can significantly enhance self-efficacy modeling allowing self-efficacy to be predicted more accurately.
Other research has focused on adapting the learning environment based on students' self-efficacy. Beal et al. [ 16
], [ 17
] present a pedagogical model that considers student motivation, mood, and cognitive processes in making instructional decisions. This pedagogical model consists of a self-report instrument within the intelligent tutoring system, which comprises a range of questions including questions pertaining to self-efficacy. Based on the submitted self-efficacy value, the pedagogical model can select the appropriate level of problems for that student (e.g., the use of a random problem selection mechanism for high-achieving students, while the tutor may present easier problems for students with lower self-efficacy).
Kim [ 18
] reports on the effect of empathetic response and gender of pedagogical agents as learning companions on a number of variables including self-efficacy. In particular, the work highlights the positive effect on the students' self-efficacy of working with responsive pedagogical agent. Further work by Kim [ 19
] highlights the benefit on students' self-efficacy of working with peer pedagogical agents as opposed to teacher pedagogical agents.
2.4 Self-Efficacy in the Home-Tutoring Context
Despite research indicating a high correlation between parental involvement in learning activities in the home and a child's academic success [ 1
], [ 2
], this may not occur spontaneously due to self-believed lack of ability, knowledge, and skills [ 3
]. Research suggests that parents with high levels of self-efficacy tend to make positive decisions about active engagement in the child's education, while parents with weak self-efficacy are often associated with less parental involvement [ 3
], [ 5
]. However, despite this, among the myriad of educational theories, which have been applied in the classroom context (e.g., Behaviorism [ 20
]), and indeed, the home context (e.g., unschooling [ 21
]), there has been little emphasis placed on increasing parental self-efficacy (e.g., through the development of parents' home-tutoring skills).
One notable exception is the Talent Education philosophy, which is based on the premise that talent is a product of environment rather than heredity [ 22
]. The actual process of teaching the young child to play the instrument involves a trio of players: the child, the teacher, and the Suzuki parent. Suzuki parents learn through mastery experience by attending the child's weekly music lessons and playing the role of home tutor. However, support for the home-tutoring process can often be unstructured and ad hoc. The use of an intelligent tutoring system that supports parents in developing home-tutoring skills might be of great benefit.
3. Parent and Child Tutor
The prospect of creating an intelligent tutoring system, which provides adaptive support, based on self-efficacy holds much appeal. To this end, the P.A.C.T. was developed.
Typically, intelligent tutoring systems support individual users and comprise user, domain, pedagogical, and presentation models [ 23
]. However, the domain of tutoring provides an added complexity in so far as there are two users, parent and child. Albeit, that future versions of the system might support multiple parents and multiple children, this version supports the typical homework set-up, one parent and one child. Fig. 1
illustrates P.A.C.T.'s overall architecture, which comprises dual -domain models, dual-user models, a pedagogical model, and presentation model.
Fig. 1. P.A.C.T. architecture.
3.1 Domain Model
As a result of P.A.C.T.'s dual-user architecture, there are two domain models within P.A.C.T., one, which comprises the skills to be learned by the parent, and the other, which comprises the skills to be learned by the child. The parent is supported in developing best home-tutoring practice, informed by Talent Education philosophy. Current implementations of P.A.C.T. support the child in developing Suzuki violin or mathematics skills.
The parent model comprises a set of tutoring tactics based on Talent Education philosophy. A considerable amount of research was undertaken in articulating this set of tutoring tactics [ 24
], [ 25
These tutoring tactics comprise expert demonstration, mastery learning, motivational game, positive reinforcement, repetition, review, and tutoring variation.
Current implementations of P.A.C.T. support the child in developing Suzuki violin skills or mathematics skills. The content for the Suzuki violin child domain model is based on the beginning Suzuki repertoire [ 24
]. The mathematics content within P.A.C.T. was developed using the Irish Primary School Mathematics Curriculum issued by the Department of Education. In particular, the content was derived from the Junior Infants (aged 4-5 years) syllabus comprising a number of strands and strand units, e.g., Time (Mathematics Primary School Curriculum, 1999).
provides an example of how the parent is supported in developing the motivational game tactic. Fig. 3
illustrates how, for example, the child may be supported in developing Suzuki violin skills using motivational games.
Fig. 2. Motivational game (reinforce stage).
Fig. 3. Motivational game (activity stage).
3.2 User Model
In an intelligent tutoring system, not only is it necessary to provide the skills to be learned (parent domain model and child domain model), it is also necessary to provide a mechanism for the provision of personalized support. In this regard, P.A.C.T. differs from other intelligent tutoring systems insofar as it provides dual-user models. A different user model is constructed for both parent and child. Both user models are dynamically updated during interaction to reflect the learner's current state. The parent user model has the following characteristics:
• It maintains the parent's dynamic self-efficacy model comprising a self-efficacy value (1-7) for each of the tactics: 1 indicates a low level of self-efficacy, while 7 indicates a high level of self-efficacy.
• It represents the navigation history, a record of the navigation path the parent has taken through the educational material and a record of the tutoring tactics encountered.
• It maintains a record of the parent's name.
• It is dynamically updated during interaction to reflect the learner's current state.
The child's user model in P.A.C.T. has the following characteristics:
• All affective states experienced by the child are logged, this allows for the provision of a history of the child's affective experiences.
• It also maintains a record of the child's current affective state and previous affective state.
• It maintains a record of the child's name.
• With the Suzuki violin domain, it maintains a record of the child's knowledge level in order to provide review suggestions tailored to the child's needs.
3.3 Presentation Model
The presentation model is based on an abbreviated version of Gagné's model [ 26
]. It comprises three key instructional events and supports both parent and child:
• Engage: the purpose of this is to attract the users' attention.
• Activate: the purpose of this stage is to allow the user to practice some newly acquired skill.
• Reinforce: the purpose of this stage is to reinforce the key message through reflection.
The engage state is where the parent is asked to enter the child's current affective state (reader is directed to Fig. 4
). It tries to stimulate interest through the use of colorful emotions. The child's user model is updated with each submission of the child's affective state. This allows P.A.C.T. to provide a personalized learning path. The activity stage provides an activity for parent and child based on input from the child's user model. This is illustrated in Fig. 3
where P.A.C.T suggests having a spider race! These activities are domain-dependent and based on the syllabi for Suzuki violin and mathematics. The reinforce stage provides a mechanism for parent reflection. More specifically, this stage provides an opportunity to reinforce the purpose of the tutoring tactic just encountered. This is illustrated in Fig. 2
, where the parent is given an opportunity to reflect on the benefit of the motivational game tactic. Fig. 2
also illustrates the mechanism for capturing parental self-efficacy around particular tactics. The parent is asked to select a value from 1 to 7, which best represents how confident they feel in using the particular tutoring tactic. This feedback is used to update the parent's dynamic self-efficacy model.
Fig. 4. Engage stage.
3.4 Pedagogical Model
P.A.C.T.'s design allows for flexibility in the use of different pedagogical strategies. Adaptivity is implemented using two adaptation technologies: adaptive presentation and adaptive sequencing [ 27
]. These technologies are incorporated to allow for a single pedagogical model to support both parent and child.
A number of adaptive presentation techniques are incorporated including text and multimedia fragment insertion and removal, explanation variants, and altering fragments. Text and multimedia fragments are inserted and removed as a result of changes in the parent's user model, in particular, their self-efficacy value. Higher self-efficacy values equate to removal of content, which, in turn, leads to the parent receiving less support. Lower levels of self-efficacy lead to insertion of content, thus the parent receives more support. This is illustrated in Fig. 5
, where P.A.C.T. suggests the parent plays a motivational game but does not suggest any particular game. The reader is reminded of Fig. 3
, where P.A.C.T. did suggest a particular game, the Spider race. In addition, multiple variations of the same content afford the possibility of providing the parent with various explanation of each tutoring tactic, in terms of explanation, example, and reflection.
Fig. 5. Text and multimedia fragment removal.
Text and multimedia fragments are altered in an endeavor to provide personalized support for the child based on their current learning goal. Content is altered based on input from the child's user model.
Finally, adaptive sequencing is used to simultaneously support both the parent and the child. Adaptive sequencing techniques allow for the presentation of appropriate tutoring tactics to the parent at each stage, thus guiding them through the tutoring process.
The implementation of such adaptive strategies is through a set of pedagogical rules, comprising tutoring rules, content rules, and efficacy rules. The tutoring rules support the parent in developing necessary tutoring skills by suggesting appropriate tutoring tactics at each stage of the process. The content rules are subdivided into rules that support the parent and those which support the child. Content rules, which support the parent, determine which explanation, example, and reflection will be presented for a given tutoring tactic, in addition to providing phase-specific support. The content rules, which support the child, alter the content based on the child's knowledge level. The efficacy rules determine the level of support provided by PA.C.T. for the parent and are based on the parent's self-efficacy. For a more detailed discussion on the rules, the reader is directed to [ 28
], [ 29
], [ 30
], [ 31
In summary, P.A.C.T.'s architecture differs from traditional intelligent tutoring systems in so far as it contains dual-domain models (parent and child) and dual-user models (parent and child). P.A.C.T. also comprises a set of tutoring rules, which determine the appropriate content to present for both parent and child. For a more detailed description of the architecture, the reader is directed to [ 32
]. The following sections provide a description of the empirical studies carried out with P.A.C.T.
4. Study A—Suzuki Vioilin
Thirteen parent and child dyads (12 female parents and 1 male parent, children comprised seven girls and six boys) participated in study A. The children's ages ranged from four to eight with an average age of five. All participants volunteered to take part in the study, and no reward incentives were provided. Participants were currently taking Suzuki violin classes and were students of a number of teachers. All children were beginning Suzuki students. Participating parents had varying degrees of previous experience. The study itself was conducted in parents' homes by using P.A.C.T. over a three-month period through the medium of the Internet.
4.2 Method and Materials
All participants were inducted with the purpose of providing a short demonstration of P.A.C.T. and explaining the terminology used within P.A.C.T. The reason for this was to define a common language for homework strategies, which a parent may or may not have prior knowledge of. For some induction was face-to-face, while for others, it was delivered through a movie distributed via C.D. Both modes comprised identical content. On the completion of induction, the participants were asked to complete a prequestionnaire, which examined parental self-efficacy (static self-efficacy model) across a number of home-tutoring skills. The instrument comprised seven questions, where parents were asked to rate their confidence in using each of the seven tutoring tactics where appropriate. Parents were asked to rate their confidence using a Likert scale comprising seven values, 1 being not confident and 7 being very confident.
For the purpose of the study, participants were randomly assigned to one of the two groups: adaptive or nonadaptive. The level of support provided for participants in the adaptive group was based on their dynamic self-efficacy model, learned from runtime self-report data. Participants in the nonadaptive group received full support regardless of their self-efficacy. Seven (all female parents, children comprised three girls and four boys) participants were assigned to the full-support version, while six (one male five female parents, children comprised four girls and two boys) participants were assigned to the adaptive support version. Parents were interviewed during the period when using P.A.C.T. The interview process allowed for the collection of qualitative data.
A postquestionnaire, which again measured parental self-efficacy across the seven home-tutoring skills (static self-efficacy model), was completed at the end of the experiment.
The resultant data set comprised approximately 19 hours of interaction-logged data and approximately 6 hours of recorded interview data. The 19 hours of interaction-logged data consisted of approximately 2,000 interactions. Based on usage statistics, a more detailed analysis was performed on the data of 11 of the 13 participants. A threshold representing the minimum number of uses of P.A.C.T. necessary to identify its effect was defined and dyads falling below this threshold were discarded from the data set. This resulted in the analysis of 18 hours of interaction-logged data and 6 hours of recorded interviews.
The results were analyzed in order to identify the effect on parents' self-efficacy of providing support throughout the home-tutoring process. As self-efficacy is best measured when task-specific, it was expected that there would be an increase in self-efficacy across those tutoring tactics encountered by the parent throughout the study. It is important to note that due to the adaptive nature of P.A.C.T. and the dynamic nature of the home-tutoring process, a parent may not encounter all seven tutoring tactics and will encounter some more frequently than others. Parental self-efficacy was analyzed at two levels:
4.3.1 Static Self-Efficacy In terms of analysis of static self-efficacy, an increase in mean values across all seven tutoring tactics can be observed from pre- to posttest scores. This is illustrated in Table 1. A paired sample t-test was conducted, which showed a statistically significant increase at the level across mastery learning ( ), motivational game ( ), and repetition ( ). Additionally, positive reinforcement ( ) is approaching statistical significance. Only statistically significant p-values are listed in Table 1.
• Static self-efficacy—which comprised analysis of pre- and posttests, which were conducted prior to commencing the study and after their last interaction with P.A.C.T.
• Dynamic self-efficacy—which comprises a log of parents' self-efficacy scores for each tutoring tactic throughout the tutoring process, provides more precise insights into parents' self-efficacy. All self-efficacy scores were logged directly after the parent completed each activity. The instrument used for collecting the self-efficacy path values is illustrated in Fig. 19.
Table 1. Study 1—Static Self-Efficacy
Expert demonstration, review, and tutoring variation showed no statistically significant increase between pre- and posttests scores. The lack of a significant increase in parents' self-efficacy across expert demonstration and tutoring variation is not altogether surprising considering the limited number of times these tutoring tactics were encountered. Fig. 6 demonstrates that of the 506 tutoring tactic suggestions P.A.C.T. made, 0.2 percent of these suggestions involved suggesting expert demonstration while 2 percent of the suggestions involved suggesting tutoring variation. This indicates that parents gained little experience in using these tactics, which may explain the lack of a statistically significant increase in their self-efficacy. The review tactic was suggested 21 percent of the time throughout the course of the study. This suggests that parents had the opportunity to gain significant experience in using the review tactic. Albeit, there was an increase in parents' self-efficacy between pre- and posttest scores for review, this increase was not statistically significant. Perhaps, the reason for this may be that on using P.A.C.T., parents gained a deeper understanding of the intricacies of the review tactic, and therefore, remained somewhat cautious in using it.
Fig. 6. Tutoring tactic occurrence.
In summary, P.A.C.T. had a positive effect on parents' self-efficacy values as measured from pre- and posttest scores. For some tutoring tactics, namely, mastery learning, repetition, and motivational game, this increase was statistically significant.4.3.2 Dynamic Self-Efficacy Since each parent submits a self-efficacy value after using each tutoring tactic, it is possible to determine a self-efficacy path (or dynamic self-efficacy model) for each parent across each tactic. This may be of benefit in understanding the effect on self-efficacy of supporting the home-tutoring process through the use of an intelligent tutoring system. As previously outlined, participants were randomly assigned to one of two groups. The first group received full support regardless of the self-efficacy value entered. The second group received adaptive support, where the level of support received was determined by their self-efficacy value for that tactic. Due to the adaptive nature of P.A.C.T. and the dynamic nature of the learning environment, each of the seven tutoring tactics were encountered with varying frequency (reader is directed to Fig. 6). As a result of this, it was decided to perform a pattern analysis of those tutoring tactics, which were most frequently encountered. These comprised mastery learning (23 percent), repetition (22 percent), and review (21 percent). Due to the nature of the data logged by the system, it is possible to plot individual self-efficacy paths. However, some interesting observations arise when self-efficacy paths are averaged across all participants. For the purpose of this study, the self-efficacy paths of participants receiving full support will first be analyzed, and subsequently, the self-efficacy paths of those who received adaptive support. This will provide insights into the effect of providing full versus adaptive support on parents' self-efficacy.
Figs. 7, 8, and 9 illustrate the self-efficacy paths for mastery learning, repetition, and review tutoring tactics for participants who received full-support when using P.A.C.T. The x-axis corresponds to the number of interactions with that tutoring tactic. The y-axis corresponds to parental self-efficacy and is measured using a value from one to seven. Each self-efficacy value illustrated in Fig. 7 is calculated based on the average self-efficacy value of all participants who received full support for that interaction, where all values are rounded to the nearest whole number. More specifically, the first value illustrated on the graph corresponds to the average submitted self-efficacy value of parents' first interaction with that tutoring tactic. In order to overcome the bias, a threshold was identified whereby averages were only included that comprised the self-efficacy values of a minimum of three participants. This was because the averages calculated, using self-efficacy values of less than three participants, may not be representative and may bias the results.
Fig. 7. Mastery learning dynamic self-efficacy—full support.
Fig. 8. Repetition dynamic self-efficacy—full support.
Fig. 9. Review dynamic self-efficacy—full support.
Fig. 7 shows a steady increase from 5 to 7 over the nine interactions. Fig. 8 shows no overall increase; however, some points on the graph show a marginal increase from a self-efficacy of 5 to 6. Fig. 9 shows an overall increase in self-efficacy from 5 to 6 with a peak in self-efficacy at interaction 7.
Albeit, the results suggest that overall P.A.C.T. had a positive effect on parental self-efficacy; it was expected that this effect would be more dramatic. However, qualitative data collected during interviews may provide further insight into these results. Following is a sample of the type of data pertaining to self-efficacy, which was collected during interviews with participants. Participant 1 states, "She was saying to me put in a 7. She thinks you have to put in the best one the whole time. She wants to be a part of the whole packet," while participant 2 states, "I wouldn't personally see any benefit and I don't do it with any thought there sometimes he wants to take over the mouse and he wants to click on the number he likes and my attitude is along as it is keeping him happy and it is keeping him involved and it is keeping him positive about the whole thing that's the biggest benefit for me."
Parents were asked how they felt about being asked to enter a self-efficacy value after using each of the tutoring tactics. The sample data presented in the previous paragraph indicates that the submitted self-efficacy value maybe the result of a suggestion by their child as opposed to a reflection of how confident they were feeling at that time. Participant 1 describes how her daughter wanted her to enter the highest self-efficacy value possible (7) as her daughter believes that it is important to enter the "best" number. This suggests an implication that the self-efficacy scale is graded, where seven is the best. Sarah reports that she does not see any personal benefit in submitting her self-efficacy value, and therefore, has developed somewhat of an ad hoc approach to it. Participant 2 states that the most important thing for her is keeping her son happy if that entails him choosing the value she is happy for that to happen, as due to parents receiving full support the value submitted has little relevance to them. This is understandable, as these participants received full support, the level of support remaining the same regardless of the submitted value.
In summary, few patterns have emerged from the self-efficacy paths of those parents who received full support from P.A.C.T. There was a slight increase in parents' self-efficacy when using review and a more significant increase in parents' self-efficacy when using mastery learning. Interestingly, all values (apart from the peak in mastery learning and review) across all paths lay in 5-6 efficacy level. However, conclusions may only be tentative as qualitative data suggests a somewhat ad hoc approach in submitting self-efficacy values due to a perception of its lack of relevance.
In terms of the effect of providing adaptive support on parental self-efficacy, Figs. 10, 11, and 12 show no clear patterns emerging. Fig. 10 illustrates the self-efficacy path for the mastery learning; however, the path is quite erratic with no clear increases. This is also the case for the repetition tutoring tactic detailed in Fig. 11. The self-efficacy path for the review tutoring tactic of those who received adaptive support is plotted in Fig. 12. This shows an overall increase in self-efficacy from 4 to 6. Again, the graph is erratic: the lowest value 3 was recorded on the 10th interaction, while the highest value 6 was recorded on the 13th interaction. All values lie between 3 and 6.
Fig. 10. Mastery learning dynamic self-efficacy—adaptive support.
Fig. 11. Repetition dynamic self-efficacy—adaptive support.
Fig. 12. Review dynamic self-efficacy—adaptive support.
Qualitative data collected during interviews provide some insight into the aforementioned results. A sample of such data is now presented. Participant 3 states, " In the beginning I wasn't sure I was going one away from the top mark at one stage, when everything was a 7 you wouldn't get much to do, so on a couple of occasions I'd ease back and instead of doing all the 7s thinking I'm great we will come back and get our jobs to do." Participant 4 states, " That took me a while to manage because at the beginning I was saying I was very confident and I wasn't getting as much feedback as I did when I said I was less confident or that I hadn't a clue it seemed to work best when you said you hadn't an idea and look for suggestions rather than saying I'm very confident at doing this I'm very confident at doing the other where as if you suggested that you weren't as confident it gave you a few extra ideas." Participant 5 states, " Getting on better because going lower in scores gives you more help."
Interestingly, all sample data presented in the previous paragraph reports some level of experimentation with P.A.C.T. in an endeavor to receive the desired level of support. Qualitative data suggests that participants submitted self-efficacy values lower than desired in order to receive the desired level of support. To this end, the submitted value may be a representation of the level of support they desired as opposed to their perceived level of self-efficacy. For example, participant 3 reports of easing back in order to receive more "jobs" (more activity suggestions) from P.A.C.T, while participant 4's strategy is pretending that she "hadn't a clue." Participant 5 frames it as tricking the system into working for their benefit by submitting 1s. This is clearly illustrated in Fig. 13, where participant 5's self-efficacy values across mastery learning, repetition, and review quickly plummet. Certainly, it appears that there is a dichotomy between their perceived level of self-efficacy and the desired level of support; in so far as parents are entering high levels of self-efficacy but still desire high levels of support.
Fig. 13. Participant 5's dynamic self-efficacy model.
A comparison of self-efficacy paths of those participants who received full support and those participants who received adaptive support highlights the lack of emergent patterns. However, qualitative data indicates that the reason for this may vary between groups. Participants receiving full support indicated that there might have been a somewhat ad hoc approach in submitting self-efficacy values. On the other hand, participants receiving adaptive support indicated a need to submit self-efficacy values lower than desired in order to receive the desired level of support.
In summary, posttest scores indicate an increase in parental self-efficacy values across all seven tutoring tactics. However, on closer inspection, based on parents' self-efficacy paths, it may be that these values are not as true a reflection as initially expected. Parents receiving full support report that, on occasion, they allowed their child to select the value to submit. Parents receiving adaptive support identify the need to submit a value lower than their perceived self-efficacy value in order to receive the desired level of support. These results provide important insights in so far as they indicate dichotomy between parents' perceived self-efficacy and desired level of support. Second, it is clear that there is a need for a more subtle instrument for eliciting self-efficacy if it is to be used within adaptive educational systems.
Thirty-six parent and child dyads (32 female parents, four male parents, 18 male children, and 18 female children) participated in study B. All participants volunteered to partake in the study and no reward incentives were given. All children were at the Junior Infants level (aged 4-5-year olds). The study itself was conducted across three schools all designated disadvantaged in terms of the number of students in the school from families with socioeconomic characteristics that have been found to be associated with low levels of educational achievement (e.g., unemployment, medical card holders, etc.) [ 33
]. In the first and second school, the study was conducted during schooltime in the schools' computer laboratories. The reason for this is that these families did not have access to a computer at home. Participants from the first school had the opportunity to use P.A.C.T. for eight sessions, while participants from the second school had the opportunity to use P.A.C.T. for four sessions. Participants from the third school used P.A.C.T. at home, and therefore, could use it as often as desired over an eight-week period.
5.2 Method and Materials
As with study A, all participants received induction with the purpose of providing a short demonstration of P.A.C.T. and explaining the terminology used within P.A.C.T. The reason for this was to define a common language for homework strategies, which a parent may or may not have prior knowledge of. For all participants, the induction was face-to-face.
The first study revealed a possible dichotomy between parental self-efficacy and the desired level of support in so far as parents with high levels of self-efficacy still desired high levels of support. More specifically, parents with high self-efficacy still desired high levels of support. This is surprising as research suggests that highly efficacious students seek challenging learning experiences [ 34
], which suggests that low levels of support would be adequate. Clearly, efforts to decrease parental self-efficacy, in order to receive better levels of support, should be avoided. This suggests the need for subtle modification in the way P.A.C.T. adapts based on parental self-efficacy.
Therefore, the second study involved a slightly modified version of P.A.C.T., where P.A.C.T. continued to provide support based on parental self-efficacy. However, users were simultaneously provided with the option of receiving the next level of support. For example, Fig. 14
illustrates the level of support provided by P.A.C.T. for a user with a self-efficacy value of 7 for the review tutoring tactic. The user is only provided with the suggested tutoring tactic.
Fig. 14. Self-efficacy value of 7.
However, they are also provided with the option to select "more help." If selected, P.A.C.T. provides the next level of support (this level of support equates to a self-efficacy value of 5-6), where they are again given the opportunity to request for more help. The users' submitted self-efficacy value and the self-efficacy value pertaining to their desired level of support are both logged by the system.
Additionally, P.A.C.T. was modified in order to omit the repetition tutoring tactic from its tutoring model. The reason for this was that its inclusion would create duplication in the tutoring process. When a mathematics teacher prescribes homework for her students, she implicitly includes the repetition tactic. It is usual for a mathematics teacher to prescribe a number of mathematics problems on the same concept in order for the learning to be internalized through repetition. This may not be the case with Suzuki violin homework where the teacher prescribes a set of notes but may not suggest the number of required repetitions.
Participants were randomly assigned one or other versions of P.A.C.T., full-support or adaptive-support (where adaptive-support, in this case, refers to the modified version of P.A.C.T. just described). Parents were interviewed during the period when using P.A.C.T. The interview process allowed for the collection of qualitative data. A postquestionnaire measuring parental self-efficacy across the home-tutoring skills (static self-efficacy model) was completed at the end of the experiment.
Study B resulted in excess of 16 hours interaction-logged data and approximately 3 hours of interview data. The 16 hours of interaction-logged data comprised in excess of 2,800 interactions. Based on usage statistics, a more detailed analysis was performed on the data of 20 of the 36 participants. In terms of the 20 participants, 9 (eight female and one male parent, five female and four male children) were assigned to the group which received full support, while 11 (nine female and two male parents, four female and seven male children) were assigned to the group which received adaptive support. A threshold representing the minimum number of uses of P.A.C.T. necessary to identify its effect was defined and dyads falling below this threshold were discarded from the data set. This resulted in the analysis of 12 hours of interaction-logged data and approximately 3 hours of interviews. Similarly, in the first study, results were analyzed in an endeavor to identify the effect, if any, of supporting the home-tutoring process with an intelligent tutoring system on parental self-efficacy. In the second study, the home-tutoring context was mathematics, and similar to the first study, self-efficacy was analyzed in terms of static self-efficacy and dynamic self-efficacy.5.3.1 Static Self-Efficacy Table 2 shows an increase in mean across all of the six tutoring tactics from pre- to posttest scores. This suggests an increase in parent self-efficacy on using the intelligent tutoring system. In addition, a paired sample t-test was conducted which identified a statistically significant increase at the level in the positive reinforcement ( ) and review ( ) tactics. Additionally, results based on the motivational game tactic were approaching statistical significance ( ). In Table 2, only statistically significant p-values are presented.
Table 2. Study 2—Static Self-Efficacy5.3.2 Dynamic Self-Efficacy with Additional Support As P.A.C.T. adapts the tutoring process based on the affective needs of the child, participants used each of the tutoring tactics a varying number of times. It can be observed from Fig. 15 that, of the 441 tactic suggestions that P.A.C.T. made, the review tutoring tactic was suggested most frequently (43 percent) followed by mastery learning (25 percent) and positive reinforcement (23 percent). Motivational Game (7 percent), tutoring variation (1 percent), and expert demonstration (1 percent) were suggested less often. Therefore, when analyzing parents' self-efficacy paths, we will concentrate on those tactics suggested more frequently, namely review, mastery learning, and positive reinforcement. First, we will look at the self-efficacy paths (or dynamic self-efficacy model) of those receiving full support. Subsequently, we will look at the self-efficacy path of those receiving adaptive support, and finally, we will provide some comparison.
Fig. 15. Study B tutoring tactics occurrences.
Figs. 16, 17, and 18 illustrate the self-efficacy path for the review, mastery learning, and positive reinforcement tutoring tactics for participants who were provided with adaptive support when using P.A.C.T. Similar to study A, the reader is reminded that the x-axis corresponds to the number of interactions with the tutoring tactic; the number of interactions may vary for each tactic. The y-axis corresponds to self-efficacy and is measured using a value from 1 to 7. As with study A, in order to overcome bias, a threshold was identified whereby only values were included where they comprised the average of a minimum of three submitted self-efficacy values. The self-efficacy path is erratic across interactions. Fig. 16 shows a slight decrease in the review self-efficacy path from a self-efficacy value of 6 to 5. Fig. 17 illustrates the self-efficacy path for the mastery learning, and after the initial drop, a steady increase in self-efficacy values can be observed, save the last two values. Fig. 17 illustrates the self-efficacy path for the positive reinforcement tutoring tactic, where the graph appears a little more erratic than the previous two graphs.
Fig. 16. Review dynamic self-efficacy—full support.
Fig. 17. Mastery learning dynamic self-efficacy—full support.
Fig. 18. Positive reinforcement dynamic self-efficacy—full support.
In summary, few patterns have emerged from the self-efficacy paths of those parents who received full support from P.A.C.T. However, interestingly, all values across all paths lay in one of two efficacy levels, level 3-4 or level 5-6. The unstructured order of the efficacy paths may not be altogether surprising when one remembers that self-efficacy is task-specific. Albeit, every endeavor was made to encourage participants to submit a self-efficacy value corresponding to their confidence in using the tutoring tactic, this may not have occurred. Instead, it may be that participants submitted values based on their reaction to specific activities suggested by P.A.C.T. as opposed to their confidence in using the particular tactic.
Figs. 19, 20, and 21 illustrate the self-efficacy path for the review, mastery learning, and positive reinforcement tutoring tactics of those participants receiving adaptive support, where as with the other graphs, values are based on the self-efficacy value submitted by the parent and not on the requests for additional support. All three graphs show a steady increase in self-efficacy. Fig. 19 illustrates a significant increase in self-efficacy from 4 to 6 across the nine interactions.
Fig. 19. Review dynamic self-efficacy—adaptive support.
Fig. 20. Mastery learning dynamic self-efficacy—adaptive support.
Fig. 21. Positive reinforcement dynamic self-efficacy—adaptive support.
Fig. 20 illustrates the self-efficacy path for the mastery learning tutoring tactic where, again, there is an overall steady increase in self-efficacy. There is one drop in self-efficacy, where at interaction 3, self-efficacy decreases from 5 to 4. At interaction 4, self-efficacy is on the increase once more with its value increasing from 4 to 5 for interaction 5. Finally, the self-efficacy path plotted in Fig. 21 represents the self-efficacy values for the positive reinforcement tutoring tactic. Similar to the mastery learning tactic with positive reinforcement, an overall increase in self-efficacy is illustrated. There is one drop in self-efficacy at the fifth interaction where self-efficacy decreases from 5 to 4. However, at interaction 6, self-efficacy increases once again with its value increasing from 4 to 5. Self-efficacy remains at a value of 5 for the remainder of the path.
Results indicate that overall there was an increase in self-efficacy across the review, positive reinforcement, and mastery learning tutoring tactics for those participants who received adaptive support. As described previously, P.A.C.T. adapts the level of support provided based on the self-efficacy value submitted by the parent for that tutoring tactic. However, in study B, P.A.C.T. also provided parents with an opportunity to request the next level of support. Interestingly, 63 percent of participants receiving adaptive support requested additional support from P.A.C.T. at some point throughout the study. This indicates that, on average, participants did not receive the desired level of support, which suggests a dichotomy between parental self-efficacy and desired level of support.
A number of patterns emerged from the data surrounding requests for additional support. Interestingly, further support was requested for four of the six tutoring tactics, namely, mastery learning, motivational game, positive reinforcement, and review. Based on all requests for additional support, the request for further support with review comprised 82 percent followed by mastery learning (9 percent), positive reinforcement (8 percent), and motivational game (1 percent) as illustrated in Fig. 22. This may not be altogether surprising as these were also the order for most frequently suggested tutoring tactics.
Fig. 22. Breakdown of requests for additional support by tutoring tactic.
However, perhaps more interestingly, upon further inspection, results indicate that 56 percent of the time that the review tutoring tactic was suggested, participants asked for further support; 24 percent of the time that mastery learning was suggested, participants asked for further support; 23 percent of the time that positive reinforcement was suggested, participants asked for further support; and finally, 8 percent of the time that motivational game was suggested, participants asked for further support. This is illustrated in Fig. 23. Furthermore, based on all requests for additional support, 27 percent of requests involved parents requesting an explanation of the tutoring tactic (self-efficacy level 5-6), while 73 percent of requests involved parents requesting an example of the tutoring tactic (self-efficacy level 3-4). As participants can only request further support at the 3-4 self-efficacy level if they have previously received support at the 5-6 self-efficacy level, this suggests that participants may require an example in order to perform the task. Additionally, there may be a tendency to submit a self-efficacy value of 5-6 even if the level of support desired equates to a self-efficacy value of 3-4.
Fig. 23 Percentage time further support required by tutoring tactic.
In terms of strategies used for requesting additional support, 31 percent of requests involved requesting further support at the 5-6 self-efficacy level and immediately requesting further support at the 3-4 self-efficacy level. This equates to asking P.A.C.T. for an explanation of the tutoring tactic and immediately asking for an example. Six percent of the requests involved only asking for further support at the 5-6 level, which corresponds to only asking for an explanation and not requiring an example.
Finally, 63 percent of requests involved asking for further support at the 3-4 self-efficacy level, which equates to P.A.C.T. suggesting an appropriate tutoring tactic and providing an explanation and the participant asking for an example. This suggests a dichotomy between parents' perceived level of self-efficacy and desired level of support, as 31 percent of requests involved participants submitting a self-efficacy value of 7 but desiring a level of support corresponding to a self-efficacy value of 3-4, which is a substantial difference.
In summary, results indicate an overall increase in self-efficacy between pre- and posttest. Self-efficacy paths of participants receiving adaptive support show an increase in self-efficacy. However, the same cannot be said for participants who did not receive adaptive support. On participants receiving adaptive support, results indicate a dichotomy between parents' perceived level of self-efficacy and desired level of support. More specifically, patterns have emerged which indicate that parents are entering efficacy values, which are too high for their desired level of support. Thirty-one percent of requests for further support were as a result of parents submitting a self-efficacy value of 7 but desiring a level of support corresponding to a self-efficacy value of 3-4. This may indicate a lack of understanding about the link between self-efficacy and ability, or indeed, an eagerness to appear confident in their ability leading to unwillingness to ask for assistance. Therefore, there is a need for future research into the design of a more subtle instrument for collecting self-efficacy values.
In both studies, an increase in static self-efficacy across all tutoring tactics was observed. For both studies, a deep analysis was performed on parents' dynamic self-efficacy model (self-efficacy paths) of the three most frequently suggested tutoring tactics. For study A, these comprised mastery learning, repetition, and review. For study B, they comprised review, mastery learning, and positive reinforcement. In terms of the self-efficacy paths of participants who received full support, no clear patterns emerged in study A or study B. The reason for this may be that as the self-efficacy value submitted by the parent did not influence the level of support provided by P.A.C.T., the values were submitted in a slightly ad hoc manner.
Similarly, no clear patterns emerged from the self-efficacy paths of those receiving adaptive support in study A. Qualitative data indicates that this may be because of a dichotomy between parents' perceived level of self-efficacy and desired level of support. More specifically, parents' desire to enter high levels of self-efficacy resulted in P.A.C.T. providing low levels of support, which they found unsatisfactory. As stated previously, this was surprising as research suggests that highly efficacious students seek challenging learning experiences [ 35
], which suggest that low levels of support would be adequate. However, interaction logs suggest the contrary with parents reanalyzing their self-efficacy in order to receive greater support leading to turmoil in the self-efficacy paths. Bandura [ 6
] provides one possible explanation in his suggestion that the most functional efficacy judgments tend to exceed what one can actually accomplish. This suggests that parents may enter self-efficacy values that exceed their ability. Although the benefit of high perceptions of capability in the face of low knowledge of tutoring tactics remains unknown, it is clear that efforts to decrease parental self-efficacy, in order for parents to receive appropriate levels of support, should be avoided.
Therefore, in study B, a slight modification was made to the adaptive strategy as far as once parents received the level of support corresponding to their self-efficacy value and they had an opportunity to request further support. It appears from the data that this slight modification had a stabilizing effect on the self-efficacy paths. A clear pattern emerged from the data consisting of overall increases in all three self-efficacy paths. The provision of additional support in study B provides a mechanism to further investigate this possible dichotomy between parental self-efficacy and desired level of support. Indeed, data collected in study B corroborate the findings from study A. More specifically, data indicate that 31 percent of requests for further support in study B resulted in parents progressing from a level of support corresponding to a self-efficacy value of 7 to a level of support corresponding to a self-efficacy value of 3-4. This represents a substantial difference. Although Bandura [ 6
] suggests that this overestimation serves to increase effort and persistence, but it provides an additional complexity for intelligent tutoring systems, which attempt to provide personalized support based on self-efficacy. There is a need for future research to investigate the use of more subtle instruments in the collection of self-efficacy values and to investigate the correlation between increased self-efficacy and learning.
• O. Lahart is with the National College of Ireland, Mayor Street, IFSC, Dublin, Ireland. E-mail: firstname.lastname@example.org.
• D. Kelly is with the Centre for Research in IT in Education, Trinity College Dublin, National College of Ireland, Mayor Street, IFSC, Dublin 1, Ireland.
• B. Tangney is with the Centre for Research in IT in Education, Department of Computer Science, Trinity College Dublin, Dublin 2, Ireland.
Manuscript received 22 Dec. 2008; revised 13 Mar. 2009; accepted 3 Apr. 2009; published online 21 Apr. 2009.
For information on obtaining reprints of this article, please send e-mail to: email@example.com, and reference IEEECS Log Number TLTSI-2008-12-0118.
Digital Object Identifier no. 10.1109/TLT.2009.20.
received the PhD degree from Trinity College Dublin. She is a member of the Research in Education and Learning Technology (REALT) Group at the National College of Ireland and Centre for Research in IT and Education (CRITE) at Trinity College Dublin, where she is an active researcher in adaptive education systems, tutoring strategies, and collaborative learning. She is a member of the IEEE.
is with the Centre for Research in IT in Education, Trinity College Dublin. He is an active researcher in adaptive education systems, individual learning characteristics, and multiple intelligence. He is also working on the EDUCE adaptive education system that utilizes learning characteristics to provide an individualized learning environment. Previously, he was the Director of the Research in Education and Learning Technology (REALT) Group at the National College of Ireland. He was a local chair for the Adaptive Hypermedia Conference 2006 and a coauthor on the eLearning Research and Development Roadmap for Ireland, 2004. He is a member of the IEEE.
is a senior lecturer in the Department of Computer Science, founder and codirector of the Centre for Research in IT in Education (CRITE), and warden of Trinity Hall at the Trinity College Dublin, Ireland. CRITE's research covers areas ranging from the development of learning tools and environments to social outreach activities. His current research interests include the innovative use of ubiquitous technology to enhance the experience of learners. Particularly, his research has developed educational tools in the areas of adaptive hypermedia, music education, and learning tools for mobile devices. He has held visiting positions at the Universities of Sydney and Kyoto. He is a recipient of Trinity's Provost's Teaching Award for excellence and innovation in teaching and learning. He was a pedagogical advisor on MIT's Toy Symphony Project, is a member of the editorial review board for the AACE Journal of Computers in Mathematics and Science Teaching
, and was a program chair for the 2007 Computer Assisted Learning Conference. He is a member of the IEEE.