Free Article: CG&A - Developing Children's Regulation of Learning in Problem-Solving With a Serious Game

Mathematics is a key topic in today's society and a core subject assessed in the PISA tests. Mathematics is linked to problem solving, which is a critical strategy development area in young children's school education. One of the most important steps related to learning in problem solving in schools is understanding the problem itself. That is, students need to understand clearly what they are being asked to do. Solving a problem effectively requires the use of specific self-regulation strategies, such as reading the problem carefully, identifying the context and what is being asked, paraphrasing the problem, taking notes on necessary data, and drawing diagrams and/or schema to organize the data. In fact, students who tend to be more self-regulated usually have more success with problem solving because the concept involves various phases and different intellectual processes, such as accessing prestored information and applying problem-solving strategies. The development of problem-solving strategies is related to acquiring self-regulation strategies that allow the learner to understand what to do, how to do it, and why. Considering this sociocognitive perspective and the importance of self-regulation processes, it is pertinent to develop educational resources that facilitate teaching and learning in problem solving in math. This work is part of an activity developed to help the Lisbon City Council perform an intervention with the students from Lisbon's public schools. Our approach is based on the development of a novel educational resource in the form of a digital serious game named Festarola. The goals of the game as a ludic activity include fostering the self- and shared regulation of learning and improving problem-solving accuracy in math in third- and fourth-grade students. In line with this goal, we present the following research question for this investigation. Can a serious game foster the regulation of learning and problem-solving accuracy in math of third- and fourth-grade students? This work is an extension of the paper presented at VS-Games 2019 with further details on the game design and user studies.

REGULATION OF LEARNING IN COLLABORATIVE PROBLEM-SOLVING

Self-regulated learning refers to the degree with which students motivationally, metacognitively, and behaviorally manage their own learning. It involves various phases, such as the forethought phase, where individuals reflect on the task they must perform and engage in strategic planning; the performance phase, which involves executing and monitoring the task; and last, the self-reflection phase, which refers to the self-evaluation and self-consequating of one's own performance. These phases are in line with the problem-solving phases proposed by Polya. Specifically, these phases consist of understanding the problem, planning how to solve the problem, carrying out a plan, and reviewing the process. Following these phases as a self-regulated process may lead to better problem-solving accuracy in math. In sharing the regulation of learning, students may develop self-regulation strategies, prosocial skills (e.g., conflict resolution strategies) and an assertive communication style by having opportunities to verbalize their thoughts and strategies to solve problems. During the execution of tasks, such as problem solving in math, the members of a group can regulate their own learning, support their colleagues in the regulation of learning to reach different objectives (i.e., coregulation of learning), and/or regulate their learning process to reach a common objective (i.e., shared regulation of learning). Additionally, the success of a student during this process seems to depend on the success of the whole group, thereby making it essential for different elements to trust the team and recognize a common direction.

GAMES FOR THE REGULATION OF LEARNING IN PROBLEM SOLVING

In parallel with the self-, co-, and shared regulation of learning, the literature has highlighted the role of games in developing learning skills. Games, problem solving, and learning are part of everyday life. Games, in particular, are seen as a tool to help one think, learn, and learn to think. Digital games require diverse skills that are considered essential to learning, such as attention, research, planning, communication, creativity, and self-confidence. Additionally, games promote problem-solving skills, as they are challenging, accessible, and cognitively demanding. Digital game-based learning (DGBL) has been a focus of education professionals in various fields, including mathematics, for the improvement of learning effectiveness and efficiency in motivating and ludic learning environments. The literature has highlighted that games can make specific subjects that are often perceived as difficult seem more accessible. Specifically in terms of problem solving, some of the literature has indicated that digital games are user-centered regarding the development of problem-solving strategies while also promoting overcoming challenges, cooperation, and engagement. Shih et al. demonstrated how a game designed to help elementary school students solve problems fostered collaboration during problem solving, and thus, better learning effectiveness. Yang investigated the effectiveness of DGBL on ninth-grade students’ problem solving, learning motivation, and academic achievement. The results revealed that DGBL was effective in promoting students’ problem-solving skills and motivation. Additionally, a recent study of sixth- and eighth-grade students using a programming game found that problem-solving behaviors were significantly associated with students’ self-explanation ability.

OBJECTIVE

Considering the previously discussed impact of the self- and shared regulation of learning, as well as games for learning to solve problems, we developed a digital serious game named Festarola designed for young children aged 8–10. With this game, we aimed to fulfill the following objectives:

1. to foster problem-solving accuracy in students through a process based on the regulation of learning (i.e., self- and shared regulation), including various phases that are interrelated with the heuristics of Polya, namely, (a) understanding the problem, (b) elaborating a plan to solve the problem, (c) executing the plan, and (d) reflecting on the obtained results;
2. to provide students with diverse ludic and appealing learning scenarios to foster motivation and knowledge in problem solving by bridging mathematics and a real-world scenario;
3. to foster students’ self- and shared regulation of learning through interactive scenarios by promoting face-to-face group discussions, as well as individual experiences.

FESTAROLA, THE GAME

Festarola's gameplay integrates different phases of problem solving and revolves around organizing a party for a group of children (see Figure 1). The player is part of the organizing team, which consists of between two and four players. The overall goal is to please the guests at the party. To organize the party, the team needs to decide on a proper theme for the party, buy food, drinks, and decorations, and hire entertainers, while also dealing with budget and time restrictions. The gameplay is divided into five activities (see Figure 2). Most of the activities can be mapped to a problem-solving phase either individually or collaboratively to reach the main goal of pleasing the guests at the party and meet the budget and time restrictions. This solution allows the development of problem-solving strategies and fosters self- and shared regulated learning in students.

Figure 1. Festarola's town shops cardboard mockup.

Figure 2. Festarola's five activities in order of execution. All are performed in groups and focus on shared regulation of learning, except for Executing the plan which is performed individually and focuses on self-regulated learning.

Choosing the theme for the party and defining a plan for the team are the first two activities, which are performed by a group to reach a common objective (i.e., shared regulation). These activities correspond to the first two phases of the regulation of learning in problem solving (i.e., understanding the problem and elaborating a plan to solve it). Defining the plan consists of creating a list of things to buy, rent, or hire and assigning individual responsibilities to players. Different shopping lists are defined (one per team member), and part of the overall budget is assigned to each player. In the end, the players need to agree on which list each one is responsible for. The third activity is performed individually (i.e., self-regulation) and corresponds to the third phase (i.e., performance phase) of the regulation of learning in problem solving, which implies executing the plan. Each player performs the concrete shopping actions in the town shops according to the items on their shopping list and the assigned budget. However, this activity does not impose strong restrictions, as players can buy different things than those on the list and exceed the budget. Such actions in town take time; hence, there is a limited number of actions that players can perform during this activity. The fourth and fifth activities, revising the performance and setting up the party, respectively, are conducted in the group (i.e., shared regulation). The fourth activity maps the reflection of the obtained results (i.e., self-reflection in problem solving) and occurs when the team meets to share the items they bought. This activity provides them with the opportunity to check the initial plan and revise their shopping options by returning some items if they choose to. These decisions are made together in the group to reach the common goal of organizing a successful party through planning, performance, and self-reflection (i.e., shared regulation). Once the team reaches an agreement, the game advances to the activity of setting up the party. This activity allows the players to distribute the items in the room where the party will take place. Once they finish, the party starts, and the final score is presented. When playing in a group, the team shares the same computer. Players gather around the computer during these activities and are invited to move to a different computer once the third activity starts so that each player is performing the third activity individually on a separate computer. Festarola was developed in Unity and is deployed on desktop computers with an active internet connection. It supports the Windows, macOS, and Linux operative systems. For teachers to deploy the game in class, they need to have the standalone game installed on the class computers. If they do not want to use the default scenario, they may configure different scenarios to use (see the “Defining New Scenarios” section), distribute students into different groups as teams, and manage the time spent on each game activity, which may be separated into different class sessions. After the game is finished, the teachers have access to the data for analysis.

Detailed Gameplay

Game Start

In the beginning, the players are asked to choose a character and give it a name. This character serves as the player's avatar during the game. Once all the players are happy with their avatars, they must choose a name for their team. The main idea of this action is to reinforce the establishment of a shared identity of the players as a team. The team name is used to support the game's save and resume mechanisms as well. After initializing the game, which starts after the definition of the team name, the players have the option to resume a previously saved game. Saving is automatically performed after each activity.

Gameplay Activities

1. Choosing the party theme (shared regulation of learning). In this activity, the players take time to explore the interests of the party's guests and choose a theme for the party (see the image at the top left of Figure 3). A guest is either one of the players or a nonplayer character (NPC), with the total number of guests always amounting to eight (e.g., if there are three players, then there are five NPCs). Each guest is automatically assigned up to four likes and dislikes from a set of four different themes: medieval, space, farm, and sea. For example, a character may like the medieval theme and dislike the space and sea themes. The challenge of the team is to select the theme that will please the most guests. The players have to count the number of guests who like/dislike each theme and pick the theme that maximizes the like/dislike ratio. This activity presents a numerical challenge and enables perspective taking, as the players should consider others’ preferences. This choice is presented in the remaining activities to support the team's future decisions; however, it does not enforce any decision. For example, if the medieval theme is the theme selected, the players can still buy a pirate cake (from the sea theme) if they choose to. This may be useful to help please guests who do not like the party's theme, but it can also be a trap for students who consider their preferences instead of those of the guests (i.e., not being able to take the perspective of others). The choice of the theme will determine the main decorations of the party room chosen in activity 5 (e.g., image on the left of Figure 4). Furthermore, at the end of this first phase, the students are asked to interpret the information that is being given to them and what is being asked of them (i.e., by replying to the question “What information do you have and what do you have to do?”). This information is gathered through an in-game dialog, where the students can write a response (see the “Guiding and Reflection” section for more details).

Figure 3. Festarola screenshots. On the top left is the theme selection, whereas on the top right is the definition of the team plan. The bottom left depicts a visit to a shop while executing the plan. The figure on the bottom right shows an example of the revision of the execution.

Figure 4. On the left is an example of the party setup activity. On the right is a sample of a party scoring.

2. Defining the team plan (shared regulation of learning). In this activity, the players create several shopping lists and assign each one to a different player (see image on the top right of Figure 3). Each shopping list contains a set of items and is built around 10 different categories (e.g., drinks, cakes, entertainers, activity items, etc.). The challenge in this activity is to define a set of lists with a good distribution of items. A good distribution assigns a similar number of categories to each list and places items of similar categories in the same list. Additionally, it should include enough items for the eight guests. The main idea is to distribute the items in a way such that all the members use a similar amount of time to perform their shopping task while avoiding traveling to the same shops, hence, minimizing the total travel time of the team members. An additional concern in this task is the distribution of the budget. The team receives a limited amount of money that they can spend (e.g., € 100). That amount needs to be distributed throughout the lists. This brings an additional challenge to the players regarding the recognition of which list contains the most expensive items. There is no direct scoring in this activity, but the planning will influence the performance of the players in activity 3. A good balance in the plan leads, for example, to the players spending less time traveling and shopping and ultimately getting a higher score in the end. Similar to the previous activity, at the end of this phase, the students are asked, through an in-game dialog, why they chose to proceed the ways in which they did. 3. Executing the plan (self-regulated learning). In this activity, the players navigate throughout the town and enter shops to buy the items needed according to the plan (see image on the bottom left of Figure 3). The players can consult their assigned shopping list anytime during the activity. Movement actions (i.e., entering a shop) and buying actions take time. Time in this context only changes when players perform an action. Players receive a time budget (equal for all) and need to buy everything before their time runs out. The players may return items if they wish to with no monetary penalty, but this action also takes time. The challenge in this activity is to buy all the items on the list while taking into account the party theme and the budget restriction. Note that the list specifies generic items, such as a cake. However, the stores sell items related to different themes as well, such as farm cakes or pirate cakes. The players need to make their buying decisions carefully to match both the party theme and the interests of the guests. However, themed items are more expensive than generic ones and players should stay within the budget. The actions that players may take in this activity are not limited by any means. The players can buy items that are not on the list and are not forced to buy items based on the chosen theme. Additionally, they may spend any amount of money over the budget. This freedom is included in the gameplay to allow the players to make mistakes and to make them actively responsible for following the plan. The players may leave the town as soon as they feel that their part of the plan is concluded. However, they are forced to leave if their time runs out. At the end of this phase, the students are asked why they bought what they did. 4. Revising performance (shared regulation of learning). In this activity, the players see the items that all the team members bought and decide if they want to keep them all or if they want to return some items (see image on the bottom right of Figure 3). The team members can check the initial plan to support their decision. The team may only complete this activity if the total cost of the items kept is equal to or lower than the budget limit. The team incurs no monetary costs for returning any items but will be penalized for the number of items they return (check the budget scoring in the “Scoring” section). The challenge in this activity is to reach an agreement about the items to keep and to ensure that the cost is lower than the budget. A good decision keeps the items that will please the greatest number of guests without overspending. Note that if the team defined a good plan and its members carried it well, this activity will not present any challenge, as there is no need to return any item. In the end, the students are asked to explain their decisions in this phase through an in-game dialog. 5. Setting up the party (shared regulation of learning). In this activity, the players may distribute the items they bought throughout the room where the party is to take place (see Figure 4). The room starts off mostly empty with basic decorations according to the chosen theme. The players may select items and place them in the room. There is no restriction on the placement of items and there is no need to place any items at all. This activity is not a challenge per se, as it functions more like a gratification and cool-down activity. It presents a playful and fun experience where players express themselves and imagine the party setting. Nevertheless, the team may discuss the actions they wish to take. Once they feel that the setup is ready, they start the party and receive feedback from the guests in the form of a final score.

Scoring

The score in the game has three different components, namely, the success of the party, the budget spent, and the time taken while shopping for items. The success of the party depends on the number of pleased guests. A guest's satisfaction is measured by comparing his or her interests to the number of different themes portrayed by items at the party. Guests like an item if they like its theme, but dislike an item if they dislike its theme. Guests are pleased with the party if there are more items in the party that they like than items that they dislike. Big items count more than small items (e.g., an inflatable castle counts more than a princess cupcake for the medieval theme). Also, there is a demand for themed items according to the number of guests who like that theme. This means that if most of the guests like a theme such as space, for example, then the party needs several space items, as one is not enough to please them all. Despite the choice of the party theme, the computation of the number of happy guests takes into account all the likes and dislikes. The decision is made for each guest, according to the following: The ItemsLike variable represents the weighed number of items of the themes that the guest likes, whereas the ItemsDislike variable represents the same for the themes that the guest dislikes. The constant K is introduced to control the challenge. The higher the value of  is, the more difficult it is to move the guests from a neutral state. For example, imagine a scenario with two guests. One guest likes the space theme and dislikes the farm theme, whereas the other guest likes the space theme and dislikes the medieval theme. If the party includes two items of the space theme and one item of the medieval theme, for , the first guest is happy (as he gets one item he likes and no dislikes, hence ), and the second guest is neutral (as he gets one item he likes and one item he dislikes, hence ). In the case of , both guests will be neutral. More items of the space theme would be needed to please the first guest. Each happy guest accounts for 1 point in the score, and each unhappy guest accounts for 1 point. The budget score is computed based on the number of items returned in the revision of the teams’ performance (activity 4). There is a base number of points (e.g., 10) that is decremented by one unit for each item returned. The rationale behind this scoring mechanism is the fact that the main reason for returning items is overspending beyond the limits of the budget. The time score has a similar mechanism. There is a base score (e.g., 10) that is decremented for each player who runs out of time. There is additional score feedback in the form of stars attributed to the performance. The party can earn zero to three stars. It earns one star for each of the following conditions: The team had no time penalties, the team returned no items, and there were more than five participants pleased with the party.

Guiding and Reflection

The game includes two additional characters used to guide the players in the activities and prompt them to reflect on their decisions. At the beginning of each activity, one character presents a description of the activity (see the left image of Figure 5). During or at the end of an activity, another character asks the players to justify their actions and decisions (see the right image of Figure 5). The players are asked to justify their choice of theme, the way in which they divided the tasks in the plan, the reasons for their shopping actions in town, and why they returned items (or not) during the performance revision activity. These justifications are written in a text box and are in most cases a team responsibility. Therefore, discussion among the group of children is promoted.

Figure 5. On the left is a character presenting the overall goal of the game. On the right is a character asking for the reason for cancelling the changes performed in the revising performance activity.

Defining New Scenarios

This game was developed to be a tool used by teachers and researchers. For this reason, the game records all actions performed in the activities and all texts written by the students to allow future analyses to be performed. Additionally, different scenarios may be presented in the game by configuring a set of parameters. It is possible to change the set of interests of the party's guests and, thus, change the difficulty of the overall challenge. For example, if most of the guests share common interests, it will be easier to reach a good solution regarding the theme of the party. However, in case of conflicting interests, reaching a good solution will be harder. It is also possible to define the budget limit and time limit for actions taken in town. By doing this, teachers can define different levels of pressure and flexibility in the task, for example, by allowing more exploration and correction actions if the time limit is higher. Finally, the prices of the items in the shops can be changed as well. Such changes may impact the nature and difficulty of the math calculations needed in the game. For example, if the prices are all in small rounded (e.g., one-figure) numbers, it will be easier for the players to estimate the overall costs of buying actions rather than if the numbers in the prices are higher (e.g., two figures) or include decimals.

USER STUDY

Following the invitation from Lisbon's City Council, we conducted a user study in which the participants were tasked with completing the phases of the game in different sessions. In the context of the intervention, Lisbon's City Council required that all participating students be included in the gameplaying activities, meaning that it was not possible to include a control group. Due to this intervention, we aimed to understand whether a serious game can be presented as a ludic activity with phases based on the regulation of learning to promote both self- and shared regulated learning and consequently foster problem-solving accuracy in the math learning of third- and fourth-grade students.

Sample

This user study involved 363 primary school students aged 8–11 years who were from 16 classes (i.e., 8 classes from the third grade and 8 from the fourth grade) from 3 public schools in the district of Lisbon (see Table 1). A pilot test was also conducted with a fourth grade class with 21 students. Due to difficulties when performing the intervention in class, such as the lack of a stable internet connection, we will only take into consideration data from 269 of the 363 students.

Resources

Students’ Reflections on the Self- and Shared Regulation Phases During Gameplay

The students provided reflections in each of the four phases during gameplay. The students were asked to justify their actions through written text in the self- and shared regulation phases during gameplay to solve the problem at hand (i.e., organize a birthday party). We verified the interrater reliability of the coding for each phase, which was done by two independent raters who computed an intraclass correlation (ICC). This analysis revealed a good ICC value  for the first phase, an ICC  for the second phase, an ICC  for the third phase, and an ICC  for the fourth phase. Specifically, 99%, 100%, 95%, and 99% of the variance in the mean of two raters was true score variance in the four sessions, respectively. The users’ in-game responses revealed a reasonable reliability of . These reflections in the four phases were used as the independent latent variable of the regulation of learning in problem solving in a structural equation modeling analysis to examine whether they are predictive of problem-solving accuracy in math (i.e., performance in the game and a mathematical problem on paper as the dependent variables). The replies were coded by phase as follows.

• Choosing the party theme: 1 = no response; 2 = irrelevant information for the resolution of the problem; 3 = information provided regarding what students were asked to do; 4 = information provided and students mentioned what they were supposed to do.
• Defining the team plan: 1 = no response; 2 = irrelevant information for the resolution of the problem; 3 = explanation as to why students planned the way they did (e.g., “we divided the budget evenly so we could all have time to go shopping”).
• Executing the plan: 1 = no response; 2 = irrelevant information; 3 = information provided according to the game's overall objective of organizing a party (e.g., “we bought things for the party”); 4 = information provided according to the team's specific plan to organize the party (i.e., “I respected the budget I planned with my colleagues.”).
• Revising performance: 1 = no response; 2 = irrelevant information; 3 = general self-evaluation with no criteria (e.g., “we did well”); 4 = specific self-evaluation with criteria (e.g., “we did well because we bought only what we had planned to.”).

Performance in the Game

We collected the objective performance of the students in the game. We focused this performance on the number of items the students returned (in the gameplay activity 4), as this outcome indicated whether the students were able to execute the task according to their plan (e.g., by buying items not included in the original plan) or if they spent too much money, thereby not respecting their budget. This outcome also indicates if the students did not define a good plan. The in-game performance served as one of the dependent variables and was re-coded from 1 (returned more items) to 5 (returned fewer items).

Mathematical Problem on Paper

With the aim of better understanding the students’ performance with regards to problem solving, a team of primary education teachers developed a mathematical problem, which the students solved on paper (i.e., a math problem from session 8, see the “Procedure” section). The math problem served as an objective measure of performance, revealed a reasonable reliability of  and was used as another dependent variable reflecting the students’ performance. Since this mathematical problem evaluated the same construct as the performance in the game, it also served as concurrent validity. After the resolution of the mathematical problem, the students were asked to explain what they did to solve the problem. Their responses were coded and grouped into two categories: Justifications that mention calculations made and/or strategies used to get the results and irrelevant comments that show an inability to explain their answer.

Scale of Motivational Beliefs for Problem Solving (SMBPS)

The SMBPS evaluates the students’ perceptions regarding self-efficacy in problem solving, their objective goals, and their utility perception regarding these problem-solving tasks.

Checklist of Expectancies and Evaluation of Problem Solving (CEEPS)

The CEEPS is composed of eight questions, namely, four initial questions (before problem solving) and four final questions (after problem solving). Its objective is to collect information regarding the expectations and self-assessment perceptions of the students regarding their performance before and after problem solving.

Questionnaire for the Evaluation of the Game by the Students

A questionnaire with open-ended questions was created to gather the students’ self-reported perceptions of the game (i.e., “What I learned from the game was...”; “What I most liked about the game was...”; “What I least liked about the game was...”). A thematic analysis was performed with the data. When applying the questionnaire, we also performed a semistructured group interview with the students.

Questionnaire for the Evaluation of the Game by the Teachers

A questionnaire was created to gather the teachers’ perceptions of the game, namely, its impact and potential as a pedagogical tool. A thematic analysis was performed with the data. Along with the questionnaire, a semistructured interview with the teachers was also performed.

Questionnaire of Collaborative Work Among Peers (QCWAP)

The QCWAP was used to measure the students’ perceived shared regulation of learning. This instrument was developed and used in previous studies and is based on the shared regulation of problem solving in math and on Polya's model of problem solving. It is a heteroevaluation measure, where students evaluate how their peers/teammates (i.e., contextual variable) helped the team reach a common goal (i.e., in this work, to organize a successful birthday party in Festarola and, therefore, solve math problems). It has eight items that are responded on a Likert-type scale ranging from 1 (not at all) to 5 (always) and pertain to the problem-solving phases. Thus, question 1 refers to the first phase (i.e., understanding the problem); questions 2, 3, and 4 pertain to the planning phase and the different subtasks that exist at this stage of the game; questions 5 and 6 ask about the performance and monitoring phases, respectively, and together they constitute the performance phase; and finally, questions 7 and 8 refer to the last reflection/evaluation phase.

Procedure

This study included 2 workshops with teachers (one at the beginning and one at the end of the study) and 8 sessions with each class of students (128 sessions total). Each session lasted around 60 min, and each class would have one session per week. Extra four sessions were performed for the pilot test. The sessions occurred as follows. Teachers’ workshop 1: First workshop with teachers. This workshop was performed to present the theoretical context of the project and create awareness of the themes and contents of the game. Furthermore, a pilot test was performed to test the game and evaluation procedures (e.g., detect bugs and errors in the game). Sessions 1: Introduction to the intervention in problem solving. The game was presented to students, who could view it and play around with it to learn how to use it (see Figure 6).

Figure 6. Photo of a team of students playing Festarola.

Sessions 2: Resolution of a mathematical problem and students’ reflection on how they solved it. Students also filled out the SMBPS and CEEPS. There was a group reflection on strategies used for problem solving. A presentation and training on self-regulation strategies for problem-solving were also included. Sessions 3 and 4: Understanding the problem and planning (shared regulation of learning). In this session, students played the first and second activities of Festarola, choosing the party theme and defining the team plan, respectively. The students were also asked to reflect on how they thought and what strategies they used. Session 5: Executing and monitoring the plan (self-regulation of learning). In this session, students played the third activity of Festarola, which included executing the plan by shopping in the stores in town. Session 6: Revision of the performance (shared regulation of learning). In this session, students played the fourth and fifth activities of Festarola, which included revising their performance and setting up the party, respectively. The students discussed their results and reflected on the successes and/or failures of their tasks, thereby self-evaluating their performance. Session 7: Improving self-regulation strategies and problem solving. In this session, students were tasked with playing the full game individually with a greater difficulty level, i.e., the party guests would have conflicting tastes in themes. Session 8: Problem solving. In this session, students solved a mathematical problem on paper and reflected on how they solved it. Students filled out the SMBPS and CEEPS. The students also evaluated the game through the previously mentioned questionnaire and a group interview. Teachers’ workshop 2: Second workshop with teachers. In this session, the teachers were asked to evaluate the game and its impact by filling out the questionnaire described earlier.

RESULTS AND DISCUSSION

Fostering the Regulation of Learning and Developing Problem-Solving Accuracy in Math

This investigation proposed to present Festarola, a game that constitutes a ludic activity with the objective of fostering the self- and shared regulation of learning and improving problem-solving accuracy in math for third- and fourth-grade students.

The Impact of the Regulation of Learning on Problem-Solving Accuracy in Math

The phases of the regulation of learning in problem solving in the game were used to measure the impact on students’ performance. Structural equation modeling (SEM) was computed with the AMOS 24.0 software package (IBM, SPSS, Amos 24). The chosen causal model presented a good fit to the data with the independent variable of the regulation of learning in problem solving (i.e., the phases presented in the game) and the dependent variable of performance in problem solving in math (i.e., a problem on paper and game performance) [, , , , , , , ]. Bootstrapping confidence intervals were used, and the p values were calculated. The model proposes that students’ accuracy in problem solving in math is predicted by their regulation of learning in the game (problem solved on paper, ; performance in the game, ) (see Figure 7). All trajectories were positive and statistically significant. The students who were more self-regulated in solving problems in math attained better problem-solving accuracy.

Figure 7. Representative conceptual model of the results indicating that those who regulated their learning more throughout the game tended to attain better problem-solving accuracy in math.

These results are in accordance with the teachers’ perceptions of their students’ performance. Specifically, teachers highlighted positive changes in the students, mainly regarding mathematical reasoning. In particular, a teacher noted changes in the “explanation of results, data sorting, strategy planning, the steps to follow in problem-solving and development of mental calculation.” Most teachers also reported an increase in the awareness of the students regarding the problem-solving phases, as shown in the verbalization of the different tasks regarding understanding, planning, executing, and revising (i.e., the phases of the regulation of learning). Furthermore, teachers also reported an improvement in students’ calculations (such as mental calculations), memorization capabilities, and the ability to follow and apply clues throughout problem-solving tasks, which were presented during classroom activities.

Providing Ludic and Appealing Scenarios to Foster Knowledge in Problem Solving

In the group interview, students mentioned that Festarola showed them the different phases of problem solving, enabled them to practice throughout the sessions, and helped them transfer this knowledge to different problems (“I learned that when I am going to solve a problem it is necessary to understand, plan, solve and review it”; “I learned to review the problems”; “I learned to plan before doing”). This outcome was visible in the traditional math problem, as the regulation of learning in the game predicted students’ performance. Students also mentioned that the tasks performed in the game allowed them to reflect on the importance of some topics lectured on in class, such as the mathematical operations and the explanation of a solution (“I learned to divide the money”; “I learned how to write a complete answer”). Moreover, other learning topics were also mentioned, such as money management and teamwork (“I learned to work in a group”; “I learned to manage my money”). The game also provided the students with an awareness of important factors to achieve a good performance, such as effort, responsibility, and organization (“I learned to use the time set and to do everything correctly”). In the interview with the teachers, the teachers shared that their opinion of the game was quite positive and stated that the game offers a learning dynamic that favors the development of mathematical reasoning. The use of this type of technology is perceived by teachers to be advantageous since it makes the tasks more appealing and motivating to students. The value of the game was particularly highlighted for money management, conflict resolution, and decision making. The ability to play games and perform teamwork in a collaborative learning context was considered beneficial for the students’ growth on both a personal and social level. In the teachers’ opinions, the game “educates through respect and mutual aid,” since students “need to learn to give in, hear others’ opinions, respect ideas, and listen to others’ strategies,” which fosters positive “relationships, help and sharing among colleagues.” The game was overall described as “very stimulating,” “fun, important,” “a motif for learning,” “positive,” “motivational,” and “beneficial.”

Fostering the Regulation of Learning: Opportunities for Self- and Shared Regulation

From a sociocognitive perspective on the regulation of learning, we considered measuring behavioral, personal, and contextual variables to enable a better understanding of how students participated in the different phases (i.e., forethought, performance, and self-reflection). Specifically, we investigated the students’ performance expectations/anticipations related to learning outcomes, including the level of difficulty they felt, their self-evaluation of how they solved a specific problem in math and their self-efficacy beliefs with regards to successfully solving problems in math. To understand how the students engaged in self- and shared regulation of learning during problem-solving, the students answered an open-ended questionnaire and the QCWAP to provide feedback on the game and describe their experience of solving problems with their classmates. We analyzed the students’ expectations (i.e., in the forethought phase of the regulation of learning) and the self-evaluations of their performance (i.e., part of the self-reflection phase of the regulation of learning) regarding problem solving with the CEEPS both immediately before and after the resolution of the mathematical problems in sessions 2 and 8. The results from a t-test showed a decrease in overestimation and consequently, an increase in awareness of the students regarding their performance in the resolution of the problems between the first problem in session 2 () and the second problem in session 8 (); . These results reinforce the findings of previous studies, where it has been shown that children tend to overestimate their task performance in class. These results are in accordance with what we found regarding self-efficacy beliefs, which also revealed less overestimation at the end of the user study. In other words, to analyze the students’ self-efficacy beliefs regarding problem solving, i.e., how well they thought they performed, we analyzed the responses from the SMBPS before the resolution of the mathematical problems in sessions 3 and 8. The results from a Wilcoxon analysis of related samples revealed a decrease in the self-efficacy beliefs of the students between sessions 2 () and 8 (), (). These results are also in accordance with previous studies that have revealed that children tend to overestimate their performance in class. We also analyzed the students’ level of difficulty while solving the mathematical problems in sessions 2 and 8. Using data from the CEEPS, an analysis was conducted using Rasch models in item response theory. The distribution of the items and the sample revealed moderate difficulty in the students’ performance on the problem in session 8 (), whereas the problem in session 2 revealed an extension of greater difficulty. Specifically, the results showed that the students had fewer difficulties solving the problem in session 8 than in session 2, highlighting that students could regulate their difficulties in performing better at the end of the project. Furthermore, if we compare the problems that the students had regarding solving a math problem on paper in sessions 2 and 8, then the number of students who were able to explain their actions in detail after the resolution of the mathematical problems increased from 44% to 49%. This increase shows an improvement in the students’ ability to report and review their actions. This result is also suggestive that more students were able to register their reflections more specifically, indicating that they engaged more in the regulation of learning. In the questionnaire, most students mentioned feeling involved in the execution tasks (i.e., shopping in town, setting up the party) and appreciated the look of the game. Students stated that they felt capable of solving the tasks presented by the game autonomously and that they gave more importance to numeric calculations and the related written explanations to better verbalize their mental processes throughout the stages of the game. The opportunity to play the game in a team and the task of managing money were features considered equally positive and motivational for the students. Last, we performed frequency analyses with the data retrieved from the QCWAP to understand the students’ perceived shared regulation of learning. The results indicated (see Table 2) that most students felt that their colleagues shared the regulation of learning in all phases of the problem-solving process (i.e., during the Festarola gameplay).

CONCLUSION

Festarola is a digital serious game designed to promote problem-solving strategies by mapping its different activities to the phases of the regulation of learning in problem solving in math, and, at the same time, to foster both the self- and shared regulation of learning through group decisions and individual activities. The game was used in a user study performed with 363 primary school children aged 8–11. In this study, the children interacted with the game in class throughout multiple sessions spread out over several weeks, thereby allowing for the evaluation of each stage separately. The positive results indicated that the game successfully stimulates and develops not only problem solving but also collaborative work and other skills needed later in life. It also provides ludic learning scenarios to foster knowledge related to problem-solving strategies. Furthermore, the results also point to a decrease in the children's overestimation of their ability to perform problem-solving tasks at the end of the user study. The game is also a useful tool to be used by teachers in class, as it enriches and facilitates learning and teaching problem-solving strategies in collaborative environments. The current study served to test possible trends of performance within the game; thus, future studies with different study designs may find our results useful to confirm and explore further the findings presented herein.

FOOTNOTES