The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - October-December (2010 vol.3)
pp: 307-318
Published by the IEEE Computer Society
Junjie Shang , Peking University, Beijing
Fong-lok Lee , The Chinese University of Hong Kong, Hong Kong
Jimmy H.M. Lee , The Chinese University of Hong Kong, Hong Kong
ABSTRACT
Virtual Interactive Student-Oriented Learning Environment (VISOLE) is a constructivist pedagogical approach to game-based learning. It encompasses the creation of a near real-life online interactive world modeled upon a set of multidisciplinary domains, in which each student plays a role in this “virtual world” and shapes its development. With sophisticated multiplayer simulation context and teacher facilitation, VISOLE aims at providing opportunities for students to acquire subject-specific knowledge in a multidisciplinary manner as well as sharpen their generic skills for problem solving. With a combination of quantitative and qualitative methods of inquiry, an evaluative study on VISOLE was conducted in Hong Kong, involving the deployment of VISOLE in 16 schools (with 254 students and 28 teachers). Notwithstanding, the positive quantitative findings (the students' advancement in the knowledge and problem-solving generic skills concerned) of the study, the qualitative data revealed that some phenomena which emerged during the VISOLE process impeded the students' learning process. On the other hand, some of the teachers, who also observed the same impeding phenomena, initiated or suggested some new interventions to mitigate these phenomena. The findings threw light on the issue of how to enhance the current design of VISOLE.
Introduction
Digital games are an important part of leisure lives of young generation. Even if gaming is prohibited at school or at home, youngsters may make all attempts to conduct this beloved activity somewhere else, such as game arcades, cyber cafés, or even game sellers' free demo machines on the streets [ 1]. This “addiction” has been one of the premises of harnessing games in education for years [ 2], [ 3], [ 4].
In the recent decade, along with the advancement of multimedia and Internet technology, as well as the pervasive promotion of student-centered educational paradigms, the focus of game-based learning has shifted from “sugaring the pill” (making boring lessons more interesting) onto the issue of how to harness the ability of games to facilitate constructivist learning. The related work includes using games to sustain learners' engagement (e.g., [ 5], [ 6], [ 7], [ 8]), offer learners near real-life simulation-based experiences (e.g., [ 9], [ 10], [ 11], [ 12]), and exploit proactive learners' communities (e.g., [ 13], [ 14], [ 15], [ 16]).
Constructivist learning emphasizes strongly an active student role [ 17], [ 18]. Nevertheless, upon this learning paradigm, it is still believed that teachers are always the best at seeing when, what, and why students are confronted with puzzles arising in the process of learning, and supporting them to solve the puzzles in a student-centered fashion [ 19], [ 20]. The same pedagogical idea should also apply to game-based learning [ 21], [ 22]. Hence, the discussion of game-based learning should not only focus narrowly on exploiting games into a “self-contained” constructivist learning environment. It is vital to explore how to articulate gaming and learning with teacher facilitation which assists students in transforming their gaming experience into learning experience.
VISOLE [ 23] is a teacher-facilitated constructivist game-based learning approach. It encompasses the creation of a near real-life online interactive world modeled upon a set of multiple real-world domains, in which each student plays a role in this “virtual world” and shapes its development. VISOLE aims at providing students with opportunities not only to acquire subject-specific knowledge in a multidisciplinary fashion, but also sharpen their generic skills for problem solving, through near real-life gaming participation. Farmtasia [ 24] is the first instance developed based on the VISOLE approach.
The focus of this paper is to discuss an evaluative study on VISOLE (with Farmtasia) conducted in the winter of 2006. The aim of the study was to investigate the educational realization of VISOLE. Using a combination of quantitative and qualitative methods of inquiry, we got positive results in terms of students' advancement in the knowledge and problem-solving generic skills concerned. On the other hand, we found some phenomena which emerged during the VISOLE process impeded the students' learning process. Some of the teachers who also observed the same impeding phenomena, initiated or suggested some new interventions to mitigate these phenomena. The findings shed light on the issue of how to enhance the existing design of VISOLE.
The rest of the paper is organized as follows: In order to facilitate readers to understand the study, we provide brief descriptions of VISOLE and Farmtasia in Sections 2 and 3, respectively. Sections 4 and 5 will delineate the design and results of the study, respectively. We will give our concluding remarks in Section 6. Last but not least, further study will be discussed in Section 7.
2. A Closer Look at VISOLE
VISOLE [ 23] is composed of three operable pedagogical phases, namely Multidisciplinary Scaffolding (Phase 1), Game-based Situated Learning (Phase 2), and Reflection and Debriefing (Phase 3), as shown diagrammatically in Fig. 1. As illustrated in the figure, Phase 2 and Phase 3 take place in an interlacing fashion, but Phase 2 starts a bit earlier than Phase 3. The following sections will discuss the specific tasks to be carried out by students and teachers in each phase. We will visualize further the pedagogical idea of VISOLE through the introduction of Farmtasia in Section 4.


Fig. 1. Three pedagogical phases of VISOLE.




2.1 Phase 1: Multidisciplinary Scaffolding
A VISOLE teacher acts as a cognitive coach to activate VISOLE students' initial learning motive. The teacher assists the students in gaining some preliminary high-level abstract knowledge (as their prior knowledge to the next learning phase) based upon a selected multidisciplinary framework through some face-to-face scaffolding lessons. In this phase, the students are equipped with “just enough” knowledge, and given only some initial “knowledge pointers.” They have to go on acquiring the necessitated knowledge and skills on their own in the next learning phase, not only from the designated learning resources but also a wider repertoire of nondesignated resources, such as the Internet.
2.2 Phase 2: Game-based Situated Learning
This phase deploys an online multiplayer interactive game portraying a virtual world. The scenarios therein become the dominant motivator driving the students to go on to pursue the interrelated understandings of the multidisciplinary abstractions encountered in Phase 1. The game encompasses the creation of a virtual interactive world in which each student plays a role to shape the development of this world for a period of time. The missions, tasks, and problems therein are generative and open-ended, and there is no prescribed solution. Since every single action can affect the whole virtual world, the students have to take account of the overall effects associated with their strategies and decisions on others contextually and socioculturally. Being situated in this virtual world, the students have to acquire the subject-specific knowledge in a multidisciplinary fashion. Apart from that, they also need problem-solving generic skills to analyze problems occurring therein, evaluate the current gaming context and different possible solutions to solve the problems, and create optimal strategies or remedies to pursue the gaming.
2.3 Phase 3: Reflection and Debriefing
This phase interlaces with the activities in Phase 2. After each round of gaming, the students are required to write their own reflective journal to reflect on their learning experience in the virtual world in a formative fashion. Moreover, at the end of this phase, they are required to write their own report in a summative fashion to conclude their overall learning experience. On the teacher side, he/she monitors closely the progress of the students' development of the virtual world at the back-end. He/she looks for and tries to act on “debriefable” moments to “lift” the students out of particular situations in the game. Respectively, during the course and at the end of this phase, the teacher extracts problematic and critical scenarios arising in the virtual world, and then conducts case studies with his/her students through some face-to-face debriefing lessons.
2.4 Discussion
The design of different phases of VISOLE was based on the theoretical foundation of gaming and constructivism. Prior knowledge is a prerequisite to learning [ 25]. The “scaffolding” in Phase 1 equips students with the necessitated prior knowledge for facilitating learning to take place in the process of gaming. The “intrinsically motivated” [ 26] and “contextually and socioculturally situated” [ 27] gaming environment in Phase 2 supports students to pursue their learning in a constructivist fashion. Learning is experience plus reflection [ 28]. The reflective writing process in Phase 3 provides students with opportunities to transform their gaming experience into learning experience. The detailed discussion of the theoretical foundation of VISOLE can be found in [ 23].
3. Farmtasia
Farmtasia [ 24] is the first instance of VISOLE. It was designed to implement the VISOLE approach. The content of Farmtasia was developed upon a multidisciplinary topic, Agriculture, in the senior secondary Geography curriculum of the Hong Kong Certificate of Education Examination (HKCEE) [ 29]. This topic involves eight areas of subject knowledge, including

    1. natural environment,

    2. biology,

    3. government,

    4. economics,

    5. technology,

    6. production systems,

    7. natural hazards, and

    8. environmental problems.

The system design and other technical aspects of Farmtasia can be found in [ 24].
Farmtasia is composed of four components (the game, teacher console, knowledge manual, and blogging platform) which will be elaborated in the sections below. Along with the following discussion, the terms “player(s)” and “student(s)” are interchangeable.
3.1 Game
This is an online multiplayer interactive game which enables Phase 2 of VISOLE. It features interacting farming systems which cover the domains of cultivation, horticulture, and pasturage. The “virtual world” of the game is modeled upon real data simulation based on sophisticated scientific models. Botanical and biological models are adopted to simulate how crops and livestock evolve in a near real-life fashion. A geographical model is adopted to create a four-season climate, which alternates wind speed, temperature, and rainfall therein. In addition, an economic model is adopted to deal with the market value and exchange of labor, machinery, seeds, fodder, farm products, revenues, etc.
In this virtual world, each player acts as a farm manager to run a farm which comprises a cropland, orchard, and rangeland. Each of them competes for two quantified outcomes—financial gain and reputation, with three other farm managers who are also running their own farm simultaneously somewhere nearby. Fig. 2 shows the interface of the game. Part A is a cropland for cultivating vegetables. Part B is an orchard for rearing horticultural fruits. Part C is a rangeland for keeping livestock. Part D shows the wise genie (an NPC [ nonplayer character]) in the game. He will appear in the virtual world for giving advice or hints to players in some critical instants. The “showing-up” frequency of the wise genie will decrease in a gradual manner, and in the latter rounds of gaming, he will disappear completely.


Fig. 2. Interface of the game: (A) Cropland, (B) Orchard, (C) Rangeland, and (D) Wise Genie.




This game operates in a round-based manner (consisting of eight rounds of gaming, 1 hour per round), and in an accelerated mode (every round equates to six months in the virtual world). In each round of gaming, players have to formulate and implement various investment and operational strategies to yield both quality and abundant farm products for making a profit (the financial gain) in the market. Besides, they should always keep an eye on the contextual factors (e.g., temperature, rainfall, wind speed, etc.) of the virtual world so as to perform just-in-time actions (such as cultivating and reaping crops at appropriate time).
In spite of the competition for the financial gain, the richest may not be the final winner. Players' final reputation in the virtual world is another crucial judging criterion. This reputation is governed by good public policies and determined by players' practice in sustainable development and environmental protection. Apart from that, as in real life, hard work does not guarantee rewards, and sagacity may not come along with fortune. Catastrophes from the nature, and disasters caused by other farm managers can ruin one's achievement in a single day. However, by setting a range of precautionary measures, “skilled” players can often minimize their loss in unfavorable situations.
3.2 Teacher Console
One of the key functions of the teacher console is to assist teachers in preparing and conducting debriefing lessons (Phase 3 of VISOLE). When students are running their farm in the virtual world, the game server records their every single gaming action. Teachers can access all students' gaming histories through the teacher console. The console interface presents the histories in Gantt chart format (see Fig. 3). Every rectangular block in a Gantt chart represents the proceedings of a student in a particular time slot with respect to the wind speed, temperature, and rainfall in the virtual world. By clicking the block, teachers can replay the proceedings in the form of video playback. This function is termed as record-and-replay function of the teacher console [ 24].


Fig. 3. Student's gaming history in Gantt chart format.




With the record-and-replay function, teachers can look for and extract interesting, problematic, or critical scenarios taking place in the virtual world to conduct case studies with their students. Since all these scenarios come from students' actual gaming experiences, it is expected that students will be more motivated to explore, empathize, and understand the constructive and destructive occurrences therein, and the corresponding enhancement and corrective actions. Teachers can also ask students to perform what-if analysis or have second thoughts [ 30] based on those extracted scenarios. Therefore, they can have deeper reflection on the differences between their current outcomes and other possible outcomes with respect to other possible acts.
Apart from the record-and-replay function, before each round of gaming, teachers can also use the teacher console to inject artificial catastrophes into the virtual world. Situating players in these artificial catastrophes provides them with opportunities to sharpen their ability to deal with contingency and emergency. Fig. 4 shows how an artificial twister happens in a player' farm which has been injected by a teacher.


Fig. 4. Twister happens in the virtual world.




3.3 Knowledge Manual
In parallel with the development of the game and the teacher console, a knowledge manual was developed to support Phase 1 and Phase 2 of VISOLE. It covers comprehensively all underlying knowledge employed to model the game. This manual serves two purposes. First, it is a reference guide for teachers to prepare and frame their scaffolding lessons during Phase 1 of VISOLE for equipping students with the required high-level abstract knowledge. Second, it is a learning resource bank for students to look up when they meet some insolvable problems or difficulties arising in the virtual world during Phase 2 of VISOLE.
An electronic version of the manual is put online so that both students and teachers can access it conveniently. The interface of the manual (see Fig. 5) consists of two navigation menus. The horizontal menu is used for navigating the content between the eight knowledge domains (from left to right: natural environment, biology, government, economics, technology, production systems, natural hazards, and environmental problems.) The vertical menu is used for navigating the subtopics within a knowledge domain. For example, the content shown in Fig. 5 belongs to the domain of natural environment, and the subtopics therein are earth's atmosphere, water, and soil.


Fig. 5. Knowledge manual.




3.4 Blogging Platform
Phase 3 of VISOLE requires students to reflect on their gaming experiences in both formative and summative manners. Farmtasia engages players in reflection exercises through blogging.
After each round of gaming, players are required to “blog” their own reflective journal formatively. Fig. 6 shows the interface of the blogging platform. This platform provides players with a journal template (i.e., “hard scaffolds” [ 31]) for helping them write their reflection more meaningfully. The template is composed of a set of open-ended guiding questions developed based on Heinich et al. [ 21] “4D” debriefing approach. The questions include

    1. How did you feel while playing this round of gaming? (Decompressing - feelings.)

    2. What happened in this round? (Describing - facts.)

    3. How do you compare the happenings in this round to the real world? (Drawing comparison - enhancing transfer.)

    4. What will you do differently in the coming round? (Deriving lessons - application.)

Through the blogging platform, everyone can view and reply to others' blogs. Moreover, by reading students' blogs, teachers can grasp more clues about each student's gaming/learning progress. These clues can assist teachers in selecting and extracting more critical debriefing content (students' gaming proceedings) to conduct case studies with their students.
Upon the completion of the game, each player needs to publish a summative report on their own blog. This report is not an ordinary report that requires only a summary of what players have experienced in the game, instead, each of them needs to give a proposal to help a fictitious character, “Mr. Lam” who is running a farm in Panyu, the Guangdong province of China. However, Mr. Lam's farm is about to close down due to his poor management. This proposal should provide him with suggestions on how to operate a farm successfully.
4. Design of the Evaluative Study
The primary objective of the evaluative study was, through Farmtasia, to investigate whether VISOLE could “yield” the new constructivist game-based learning opportunities for students as proposed in its original design. Farmtasia, as elaborated in Section 3, was designed for teaching the topic of Agriculture in the senior secondary Geography curriculum HKCEE. However, this study was carried out in the form of an extra-curricular activity, rather than a school curricular learning and teaching activity. This decision was due to the fact that, at that time (even today), VISOLE was a rather new pedagogical idea to the education community in Hong Kong (even to other places in the world). It was difficult to get schools and teachers to “risk” implementing VISOLE in the HKCEE subject. 1


Fig. 6. Blogging platform.




4.1 Participants
In late August 2006, all secondary schools in Hong Kong were invited to participate in evaluating the educational realization of VISOLE by allowing their teachers and students to join a competition, namely, “VISOLE Competition: Farmtasia.” To join the competition, each school was required to have

    1. two teachers (preferably one senior secondary geography teacher, and one computer teacher) to implement VISOLE and coordinate the competition-related activities and

    2. 16 secondary-4 students to participate in the competition.

Eventually, 16 secondary schools joined the competition which took place from mid-October to mid-December 2006. It involved 28 teachers 2 and 254 3 students from the 16 participating schools (School 1 to School 16).
4.2 Teacher Enablement Training
Three weeks before the competition started, all teacher participants were required to attend an enablement training to get familiarized with VISOLE (the ideas of scaffolding, game-based situated learning, students' reflective writing, and debriefing), and the technical operation of the game and teacher console of Farmtasia. Besides, every teacher was given a set of scaffolding materials which included scaffolding videos 4 and visual aids with regard to the eight subject knowledge concerned. These materials aimed at assisting the teachers in conducting the required scaffolding tasks in Phase 1 of VISOLE. Apart from that, some examples of debriefing lessons 5 in video format were also provided as references for the teachers to conduct the required debriefing tasks in Phase 3 of VISOLE.
In a week after the training, the teachers were required to “act as students” to participate in a four-round trial competition in the Farmtasia game so as to familiarize themselves with its operation further.
4.3 Competition
The competition was composed of two stages. The first stage was conducted on an intraschool basis, while the second stage was on an interschool basis. In the first stage, each of the 16 schools was required to implement the full VISOLE approach independently. At the end of this stage, four winners from each school then entered the second stage of the competition. In the second stage, all 64 (4 × 16) first-stage winners were grouped randomly in an interschool fashion to form 16 intracompetitive groups, and then competed in the game again. After two bouts, four final winners came out at the end of this stage.
4.4 Research Procedures
All research manipulation of the study was carried out in each participating school individually in the first stage of the competition (the part of the VISOLE implementation). The whole research process was divided into three substages, namely Pre-VISOLE, In-VISOLE, and Post-VISOLE, as shown in Fig. 7 diagrammatically.


Fig. 7. Research procedures.




Pre-VISOLE (the upper part of Fig. 7). A student knowledge pretest had been administered one week before the VISOLE process began. The students were allowed to use 35 minutes to complete the test. There were 42 questions in the test, including

    1. 25 multiple-choice questions for assessing the students' prior knowledge corresponding to the eight subject areas covered in Farmtasia;

    2. 15 true or false questions for assessing their prior knowledge in the application of the eight subject areas in a multidisciplinary manner; and

    3. two open-ended short questions for assessing their prior knowledge in the application of the eight subject areas in a multidisciplinary manner.

In-VISOLE (the middle part of Fig. 7). In each school, the teachers formed a facilitation team to facilitate the full VISOLE process (as described in Section 2). This substage took around four weeks to complete, involving four scaffolding lessons (in Phase 1), one gaming preparation lesson (before the start of Phase 2), eight rounds of gaming (in Phase 2), and four debriefing lessons (in Phase 3). Each lesson took 30 minutes. 6Table 1 shows a summary of the setting. In each scaffolding lesson, the students were equipped with two (out of eight) areas of the preliminary high-level abstract knowledge. The gaming preparation lesson was to familiarize the students with the operation of the Farmtasia game. 7 The four debriefing lessons were held after rounds two, four, six, and eight, respectively. Each round of gaming took 1 hour to finish, and the students were required to play 1 round every two days at home. The researchers (the first and the second authors of this paper) made observations on the scaffolding and debriefing lessons. However, since every school implemented the VISOLE approach in parallel with others, the researchers could only afford to observe one scaffolding class and/or one debriefing lesson in each school.

Table 1. Setting of the VISOLE Process


Post-VISOLE (the lower part of Fig. 7). Within one week after the VISOLE process:

    1. A student knowledge posttest, which was at the same level of difficulty 8 as the knowledge pretest, was administered. The students were allowed to use 35 minutes to complete the test.

    2. A student self-evaluated generic-skill enhancement questionnaire, which was designed by Bennett et al. [ 32], was administered. This questionnaire contained 35 question items which fell into four dimensions ( self, information, others, and task) of generic skills for problem solving. The students were given 20 minutes to self-evaluate the extent of the enhancement of their generic skills for problems solving after the VISOLE process. A five-point Likert scale (5: Very Great; 4: Great; 3: Moderate; 2: Little; 1: No) was adopted in the questionnaire. Appendix A shows the question items with respect to the four dimensions.

    3. A number of student and teacher interviews 9 were conducted for gaining more understanding of their learning and facilitation process in VISOLE respectively. Each interview took around 30 minutes to complete.

5. Findings
This section delineates first the quantitative findings of the evaluative study, in terms of the students' advancement in the multidisciplinary subject knowledge and enhancement on their generic skills for problem solving after the VISOLE process. Afterward, the qualitative findings of the impeding phenomena (emerging during the students' learning process in VISOLE), and the new interventions (initiated or suggested by the teachers to mitigate these impeding phenomena) will be discussed.
5.1 Knowledge Advancement
The findings discussed in this section were derived from the results of the knowledge pre- and posttests.
The returning rates of the pre- and posttests were 94.1 percent (239 [returned]/254 [delivered]) and 90.6 percent (230 [returned]/254 [delivered]), respectively. The mean scores of the pre- and posttests were 56.5 (out of 100) and 67.5 (out of 100), respectively, with an increase of 19.5 percent. A paired T-test was adopted to compare between the mean scores of the pre- and posttests. Table 2 shows the test result (the p-value is displayed in the right-most column). The result revealed that the students' advancement in the knowledge concerned was significant statistically (p-value $< 0.001$ ).

Table 2. T-test against Pre- and Posttests


5.2 Generic-Skill Enhancement
The findings discussed in this section were derived from the results of the generic-skill enhancement survey [ 32].
The questionnaire returning rate of the survey was 88.6 percent (225 [returned]/254 [delivered]). The results revealed that the students perceived their problem-solving generic skills were enhanced moderately. Table 3 shows a summary of the results in terms of the four generic-skill dimensions, ranging from the mean score of 3.31 (the information dimension) to 3.43 (the task dimension).

Table 3. Students' Self-Evaluated Generic-Skill Enhancement


5.3 Impeding Phenomena
As mentioned in Section 4, qualitative means of inquiry (lesson observations, as well as student and teacher interviews) were adopted for gaining more understanding of the students' learning and teachers' facilitation process in VISOLE. The collected qualitative data were analyzed with Maxwell's qualitative data analysis approach [ 33] (which interweaves coding, categorizing, memorizing, and contextualizing) supplemented with Creswell's thematic development technique of theme layering, and theme interrelating [ 34].
According to the analysis, there were some phenomena, which emerged during the VISOLE process, impeding the students' learning process. A number of “plausible factors” 10 leading to these impeding phenomena were identified. They included

    1. students' prior gaming experiences,

    2. students' interest in gaming,

    3. students' conception of learning,

    4. technical efficacy of the game system,

    5. teachers' time sufficiency for facilitating the VISOLE process, and

    6. teachers' prior gaming experiences.

Tables 4a and 4b shows 1) the instances of these impeding phenomena (in the right column), and 2) the corresponding plausible leading factors (in the left column). The term “nongamer students” (mentioned in Factor A1) refers students who have no or very few gaming experiences, while “gamer students” (mentioned in Factor A2) refers to students who have rich gaming experiences. Factors D2, E1, F2 related to the impeding phenomena emerging in the teachers' facilitation process. However, these phenomena influenced the students' learning process implicitly.

Table 4a. Impeding Phenomena and the Corresponding Leading Factors


Table 4b. Impeding Phenomena and the Corresponding Leading Factors


5.4 New Teacher Interventions
On the teacher side, some of the teachers also observed the same impeding phenomena (mentioned in Tables 4a and 4b) during their VISOLE facilitation process. A number of them initiated or suggested some new interventions that were targeted on mitigating these impeding phenomena. These new interventions were not specified in the original design of VISOLE. They included

    1. inviting gamer students as game tutors,

    2. using an alternative strategy to select case study scenarios for conducting debriefing lessons,

    3. triangulating students' learning progress with other evidence,

    4. timely encouragement and counseling,

    5. fostering students' off-the-game collaborative sharing, and

    6. relating students' participation in VISOLE to their academic performance.

Tables 5a and 5b shows 1) the interventions (in the first column) and 2) their descriptions (in the third column), as well as 3) the corresponding impeding phenomena that the teachers aimed at targeting on (in the second column). In this table, the notations utilized to denote the impeding phenomena (in the second column) align with the ones in Tables 4a and 4b.

Table 5a. Teachers' Initiated/Suggested Interventions


Table 5b. Teachers' Initiated/Suggested Interventions


6. Conclusion and Discussions
There has been a worry that, in game-based learning, students may not learn anything more than clicking a number of buttons to receive desired gaming outcomes [ 37]. Moreover, students often have difficulties in making connections between a game and the referent real-world system that the game is intended to represent [ 38]. Although they are sometimes requested to reflect on their “game-based learning” experience, not everyone is able to do it well equally [ 39].
Mishra and Foster [ 40] argued that although the educational potential of “learning through gaming” has been discussed widely and with strong theoretical arguments, there is still a distance to put it in place, particularly regarding the pedagogical consideration. We attempt to address this issue by proposing VISOLE. It is a three-phase constructivist pedagogical approach to empower game-based learning. The importance of teachers' roles is emphasized therein.
Evidenced by the quantitative part of the evaluative study discussed in this paper, in general, VISOLE could provide the students with opportunities for acquiring the multidisciplinary subject knowledge and enhancing their generic skills for problem solving. However, we admitted that there were a number of weaknesses in the research setting, including the nonnaturalistic learning and teaching context, the students' distraction caused by the competition, the sampling problem (on both student and teacher sides), and the self-reported nature of the questionnaire about the generic-skill enhancement.
VISOLE has yet to be “perfect.” Revealed by the qualitative part of the study, there were some phenomena, which emerged during the VISOLE process, impeding the students' learning process. Nevertheless, a number of the teachers initiated or suggested some new interventions to mitigate these phenomena. The preliminary identification of these impeding phenomena and new interventions could throw light on how to enhance the current design of VISOLE. For example, the infusion of “game operation skill scaffolding” and “emotional debriefing” respectively into Phase 1 and Phase 3 is a sort of plausibly desirable enhancement.
Novice learners, like construction workers, need structures of temporary support to build things up [ 41]. Scaffolding [ 28] refers to a process by which a more knowledgable peer assists ones so that they can perform tasks that would otherwise be out of reach. In VISOLE, the scaffolding exercise focuses only on equipping students with preliminary high-level abstract knowledge, without emphasis on preparing them to begin their gaming technically. In the study, although a “practising round” was provided for the students to get themselves familiarized with the Farmtasia game, this kind of self-directed practice was not working for everyone (see Tables 4a and 4b, Factor A1). It is recommended that Phase 1 of VISOLE needs to be composed of two parts. The former part is still “multidisciplinary scaffolding.” The latter part would be “game operation skill scaffolding” which assists students (especially nongamer students) in starting up their gaming. Furthermore, the latter part is not necessary to be carried out by teachers, but gamer students (see Tables 5a and 5b, Teacher Intervention U).
Players hate to lose; they are even willing to “cheat” in gaming by using degenerate strategies [ 42]. In the study, the unfairness stemming from the “cheats” in the Farmtasia game irritated some students (see Tables 4a and 4b, Factor D1). In VISOLE, the debriefing exercise focuses only on students' cognitive aspect, without paying attention to their emotions emerging during the course of gaming. It is recommended that, in Phase 3 of VISOLE, besides monitoring the progress of students' development of the virtual world, teachers also need to be more aware of students' emotions when reviewing their reflective journals and conducting debriefing lessons. “Emotional debriefing” should be carried out whenever necessary (see Tables 5a and 5b, Teacher Intervention X).
Assessment in VISOLE is another issue that needs to be addressed (see Tables 4a and 4b, Factor C2). Assessment is a sort of important learning feedback for students [ 39]. Nonetheless, assessing students' learning outcomes in game-based learning is not easy. Relying on students' gaming results to assess their performance may trigger the issue of assessment unfairness (see Tables 4a and 4b, Factors D1, D2). Assessing students' learning performance through reviewing their gaming proceedings sounds more authentic; however, it is impractical (see Tables 4a and 4b, Factor E1). Using students' formative and summative reflective writing as a basis for assessment seems more practical; nevertheless, it may disadvantage the ones who are bad in language or hate writing. Therefore, assessment in VISOLE is an important topic that needs further research effort.
7. Further Study
Punch [ 43] argued that too much research has tried to go directly to measurement and quantitative mapping, without the fuller understanding of the process and phenomena emerging therein. Undoubtedly, “classical” controlled comparative studies have their value in highlighting the differences among game-based learning, conventional classroom teaching, and other more mature constructivist learning approaches. However, this kind of comparison will become more meaningful and significant only when a more established pedagogical framework for game-based learning appears in the domain.
Formulating pragmatic integration of games into education requires researchers and educators to reflect on what, when, how, and why students learn (or do not learn) in the process of gaming [ 37]. In this paper, we discuss some initial understanding of students' learning process in VISOLE. However, it has yet to be considerably “in-depth.” In the study, the scaffolding and debriefing lesson observations were not conducted thoroughly enough to carry out triangulation [ 44] and thick description [ 35] during the qualitative analysis. Apart from that, as elaborated in Section 4, this study was conducted in the competition context. In other words, what happens when VISOLE enters a “real” classroom is still unknown. Thus, it opened up the need of further research on gaining an in-depth understanding of the “inner-workings” of students' learning process in VISOLE in authentic classroom context. The findings will shed light on the further enhancement of VISOLE, and provide insights into the integration of constructivist game-based learning into school education.

Acknowledgments

The work described in this paper was substantially supported by a grant from the Research Grants Council of Hong Kong SAR (CUHK4200/02H).

    M.S.Y. Jong is with the Department of Curriculum and Instruction, The Chinese University of Hong Kong, Shatin, NT, Hong Kong.

    E-mail: mjong@cuhk.edu.hk.

    J. Shang is with the Department of Educational Technology, Graduate School, Peking University, Beijing, China.

    E-mail: jjshang@gse.pku.edu.cn.

    F.-l. Lee is with the Centre for the Advancement of IT in Education, The Chinese University of Hong Kong, Shatin, NT, Hong Kong.

    E-mail: fllee@cuhk.edu.hk.

    J.H.M. Lee is with the Department of Computer Science and Engineering, The Chinese University of Hong Kong, Shatin, NT, Hong Kong.

    E-mail: jlee@cse.cuhk.edu.hk.

Manuscript received 7 Dec. 2009; revised 10 May 2010; accepted 12 Oct. 2010; published online 2 Nov. 2010.

For information on obtaining reprints of this article, please send e-mail to: lt@computer.org, and reference IEEECS Log Number TLTSI-2009-12-0188.

Digital Object Identifier no. 10.1109/TLT.2010.34.

1. HKCEE is an important public examination in Hong Kong secondary education, equivalent to O-level examination in the United Kingdom.

2. Among 16 schools, 10 of them had two teachers, one school had three teachers, and five schools had only one teacher to participate in the study.

3. One of the schools had only 14 students participating in the competition.

4. The scaffolding videos were produced purposefully for this evaluative study. The teacher who presented the scaffolding materials in the videos is one of the instructional designers of Farmtasia (a former member in our research group).

5. The debriefing-lesson videos were recorded during a pilot run of VISOLE in a secondary school carried out in April 2006. The teacher who conducted the debriefing lessons in the videos is one of the instructional designers of Farmtasia (a former member in our research group).

6. It aimed at fitting the lessons into the existing lesson time slots in the schools.

7. It included 1) distributing the game manual to the students, 2) briefing them on downloading and installing the JAVA runtime, as well as operating the Farmtasia game, and 3) letting them try a practising round of the game.

8. The format and number of questions of the knowledge posttest were same as the knowledge pretest. The questions appearing in the posttest were rephrased (or reworded) and reordered from those in the pretest.

9. The students were interviewed in groups, while the teachers were interviewed individually. All interviews were audio-recorded, and later transcribed into text for qualitative analysis.

10. Notwithstanding the fact that the category construction of the factors was grounded upon the qualitative data with a “bottom-up” approach, the qualitative data per se had yet to be rich enough to give thick description [ 35] of the “inner workings” of the students' learning process in VISOLE.

References



Morris S.Y. Jong received the BEng (Hons) degree from The Hong Kong Polytechnic University, the MSc degree from The University of Hong Kong, and the PhD degree from The Chinese University of Hong Kong, respectively. He is currently the associate director at the Centre for the Advancement of Information Technology in Education, and an assistant professor in the Department of Curriculum and Instruction, The Chinese University of Hong Kong. His research focuses on digital-game-based learning, computer-supported collaborative learning, and teacher professional development. He has published more than 50 refereed journal papers, book chapters, and conference papers concerning digital-game-based learning, teachers' IT competency, and technology-enhanced learning environments. Currently, he is on the editorial boards of a number of academic journals in the domains of digital-game-based learning, virtual reality in education, and teacher education.



Junjie Shang received the BSc and MPhil degrees from Peking University in 1996 and 1999, respectively, and the PhD degree from The Chinese University of Hong Kong in 2007. He is currently the director in the Department of Educational Technology as well as the vice dean of the Graduate School of Education, Peking University. His research focuses on computer-game-based learning and virtual reality in education. He has published more than 35 papers in international and global Chinese journals and conferences.



Fong-lok Lee is the codirector at the Centre for the Advancement of Information Technology in Education under the Hong Kong Institute of Educational Research, The Chinese University of Hong Kong. He has 15 years of working experience in the specialized field, 22 years of teaching experience in local secondary schools, and more than 11 years in conducting teacher training courses for local secondary school teachers. He served as a secondary school teacher, extracurricular activities master, and mathematics panel chairman. Presently, he is a member of the executive committee of the Global Chinese Association on Computers in Education, the co-chief-editor of the Global Chinese Journal of Computers in Education, the past president of the Asia-Pacific Society for Computers in Education, and the chair of the HK & Macau Chapter, Global Chinese Society on Computers in Education. His research interests include educational technology cognitive processes, artificial intelligence, mathematics learning, and the application of information technologies in education.



Jimmy H.M. Lee received the BMath (Hons) and MMath degrees from the University of Waterloo in 1987 and 1988, respectively, and the PhD degree from the University of Victoria in 1992. Immediately upon graduation, he joined the Chinese University of Hong Kong. His research focuses on the theory and practice of constraint satisfaction and optimization with applications in scheduling, resource allocation, and combinatorial problems. In recent years, he has also been conducting research on novel-web-based learning platforms and the accompanying pedagogies, particularly in the development of educational games. He is currently the codirector (Research and Development) of the Centre for the Advancement of Information Technology in Education under the Hong Kong Institute of Educational Research, The Chinese University of Hong Kong. He has more than 100 refereed technical publications in international journals and conferences, and has obtained more than $11 million in competitive research funding. He is on the editorial boards of the Journal of Discrete Algorithms, the CONTRAINTS journal, and the Constraint Programming Newsletter. He is active in extramural educational activities. He has taught for the Programs for the Gifted and Talented, and was the chief examiner of Paper II of the HK A-Level Computer Studies subject from 1995 to 2001. Inspired by his many former good teachers from elementary school to universities, his passion for teaching garnered him the Vice-Chancellor's Exemplary Teaching Award in 2005 and the Faculty of Engineering Exemplary Teaching Award in 1999, 2000, 2001, and 2003.
21 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool