Empirical Evidence of Priming, Transfer, Reinforcement, and Learning in the Real and Virtual Trillium Trails

Maria C.R. Harrington, IEEE

Pages: pp. 175-186

Abstract—Over the past 20 years, there has been a debate on the effectiveness of virtual reality used for learning with young children, producing many ideas but little empirical proof. This empirical study compared learning activity in situ of a real environment (Real) and a desktop virtual reality (Virtual) environment, built with video game technology, for discovery-based learning. The experiences were in the form of two field trips featuring statistically identical wildflower reserves. While the results support that the Real is superior for learning activity, they also show that the Virtual is useful for priming and reinforcing in-curriculum material, or for situations when the real environment is inaccessible. Offering the Virtual first primes for learning activity in the Real; if used second, it reinforces the Real experience, as supporting evidence shows significant transfer effects. Thus, the Virtual may serve educational goals, if used appropriately, and can come close to the Real. As informal learning environments, such as field trips and video games, are accepted as motivational, an attitudinal survey was conducted postexperiences to capture motivational factors at play, to aid in comparison and contrast, and to provide context to the empirical results on learning activity in situ; however, more work is needed.

Index Terms—Child-computer-environment interaction, child-computer interface, discovery-based learning, educational simulation, evaluation/methodology, serious games, human factors in software design, human-computer interaction, human information processing, informal learning, intrinsic learning, salient events, simulation, modeling, visualization, software psychology, virtual reality, user-centered design, user interfaces.

Introduction

In May 2007, amid the springtime bloom of wild trillium at a long-term NSF deer exclosure study site, a group of elementary school students from a suburban Pittsburgh school district went on a field trip—with a twist. In this study of the Virtual Trillium Trail (VTT), students took both real and virtual field trips to the protected site, with the real trip functioning as the control.

A driving question behind the VTT was: Could a desktop virtual environment (Virtual) draw on elements of simulations, games, and educational software to enhance perceptual, cognitive, and emotional experiences?

The best way to approach this design problem was to frame it within the context of a proven, successful real-world model: the field trip. The real-environment field trip (Real) is such a baseline model that, at its best, embodies discovery-based learning experiences beyond the confines of the classroom.

In comparing a real field trip to a virtual one through empirical evaluation of in situ discovery-based learning activity and postexperience attitudes, the study held to the assumption that Real is preferable to Virtual; this common sense “given” is too often ignored in studies centered on the effectiveness of virtual environments, perhaps due to the theoretical idealism of designers. In contrast, the study underscored this primary assumption, thereby allowing for empirical, statistical measurements that provide instructional designers with tangible ways to implement highly realistic virtual reality (VR) applications and reality-based simulations (see Fig. 1), as an extension of the classroom, as educational simulations.

Figure    Fig. 1. An image of the Virtual Trillium Trail, an educational simulation, a simulated ecology for discovery-based, intrinsic learning.

The central software design focus was on the child and the child's needs. Personalization is used to advantage in intelligent tutoring systems [ 1], where extreme personalization is the goal, and where each student can have a personally unique and meaningful experience. As is advocated in user-centered design (UCD) [ 2] and human-computer interaction (HCI) [ 3], the child, as the main user, drives the software requirements and assessment (see Fig. 2).

Figure    Fig. 2. An illustration defining intersection of the research interests of a simulated ecological environment for education (SEEE).

While formal statistical hypotheses were stated to structure the work of the study, the study itself was driven by the desire to explore, generate insights, and frame future research in this area. The results inform design and development of virtual environments for independent and intrinsic learning. The empirical analyses reported here are unique and have not been reported elsewhere; however, they are part of a larger body of work, and the ethnographic and nonparametric results have been reported and published elsewhere [ 4]. This submission to the IEEE TLT is an empirical analysis using parametric statistics to show differences in learning activity between Real and Virtual discovery-based learning activity, transfer effects, and attitudes, where the paper published in [ 4] is an ethnographic report and a nonparametric analysis of the experiences. The two papers complement and support each other, while allowing detailed analysis from either a qualitative or a quantitative approach.

1.1 Framing the Design Problem in Broader Context

Climate change is the defining problem of the age. Learning about ecology requires tools that inform, inspire, and enlighten, as well as tools that increase perception, awareness, and constructive action. Highly realistic virtual reality applications and reality-based simulations are unique tools for learning, as students may transcend time, space, distance, and scale to develop an accurate and complete understanding of complex relationships. However, not much is known about the design and construction of such tools for stimulating perception, learning, and creating; hence, the need for research that goes beyond Presence [ 5] or the “feeling of being there,” to build tools that aid learning, problem solving, decision support, and creativity. Future research is required to understand how these tools might act as conduits to increase the probability of enhanced awareness, knowledge acquisition, and constructive creativity, with respect to our ecosystems and survival.

Real-time, interactive virtual reality allows one to experience different time periods or scenarios. Simulations are data-based, often abstract, and usually, not real-time interactive. Educational software allows for information search, exploration, annotation, and augmentation but is usually not executed in a form intended to enhance perceptual, intellectual, and emotional experiences. Could virtual reality merge with simulations, games, and educational software to provide a new combination of features? Capturing the feel of the game Myst [ 6] but retaining scientific information? Could such new tools deepen children's understanding of plants in their backyards as well as develop appreciation for ecologies in distant lands? The challenge was to design and build such virtual experiences to be truthful, effective, and motivating.

Modeling an informal, discovery-based science and ecology field trip provided the ideal opportunity to meet this challenge based on real-world best practices. The Trillium Trail field trip, offered by the Audubon Society of Western Pennsylvania and led by expert educators and naturalist guides [ 7], is currently part of school enrichment programs for science and ecology; first established in the 1960s [ 8], its teaching methods and curriculum are a superset of the Pennsylvania State educational standards. These outdoor classrooms offered an ideal situation to simulate and model in the software.

Additionally, the plant population distributions at the Real field trip location have been extensively studied by Susan Kalisz, PhD, who has been conducting biological field research at Trillium Trail since 1994 [ 9]. These deer exclosure data sets—free from overgrazing, man-made roads, power lines, or sewers—provided the required virgin population distributions on which to build and statistically extrapolate the virtual plant populations in the desktop VR visualization. Thus, the physical content of the VTT consists of unique set of statistical, scientific visualization of biological plot study data, Real field trip user activities, and Audubon Society educational content. Interestingly, this offers a reality to the students that is about to slip away, as we are on a cusp in time between past and the future ecologies, and only within the fenced areas of the deer exclosures is the forest protected.

Theoretical Background

Past reports on educational virtual reality were proof-of-concept, technical investigations, or educational case studies [ 10], [ 11], [ 12], [ 13], [ 14], but none offered a multidimensional synthesis of the child-computer-environment interaction or stated a comprehensive and complete design problem. Seminal research in virtual reality for education showed evidence of knowledge gains [ 15], [ 16]. Pioneering research in immersive virtual reality and ubiquitous environments explored the problem and documented emotional reactions such as enjoyment and sense of play [ 17], [ 18], [ 19].

Ecology simulations were attempted [ 20], [ 21], [ 22], [ 23] but left the educational impact undetermined. Related research in biology education using high-fidelity simulations showed convincing results [ 24], and thus, indicated that high-fidelity simulations were needed for such learning systems. Features from high-fidelity military [ 25] and medical [ 26] training applications, used for procedural knowledge transfer and shortened learning curves, were evaluated. Analyses of those applications' critical features were important in the design of the first prototype of the Virtual Trillium Trail and in the investigation of the ideal child-computer-environment interface. Horizontal functionality of user interfaces for virtual reality applications was developed and investigated with success, especially in the use of annotation and augmentation [ 27], which is required for effective information and knowledge inquiry and acquisition. Research on navigation in virtual reality shows that route and wayfinding transfer is effective [ 28], [ 29] as an aid in exploration. Data visualization systems, geographical information systems, and landscape architecture systems enhance decision support [ 30] and serve to increase the perception and understanding of complex interconnected relationships.

Orthogonal research suggests design criteria and desirable features, such as emotional and motivational affects that relate to play [ 31] and a sense of “flow” [ 32]. The sustained and independent play elicited by noneducational games [ 33], [ 34] may offer design clues for educational software. Thus, discovering and understanding the elements in existing virtual reality software that create such intrinsic motivation is of investigative concern.

Nonetheless, while VR technology for education and learning is maturing [ 36], [ 37] not much is known about how the virtual world interfaces with the user in learning: The interactions, interferences, and convolutions of “form and function” are indeed complex and dynamic. The Virtual Trillium Trail design challenge was to create a baseline study required for an empirical comparison and contrast of causal factors found in the interactions of child, environment, and user interface in the Real and the Virtual environments.

The overriding goal was to define the ideal virtual environment for independent and intrinsic learning. The problem space is represented by the intersection between learning, real and virtual environments and the user interface. The proposed conceptual model represents the intersection of the child's mental model, knowledge ( $\Delta$ Knowledge), the user interface ( $\Delta$ UI), and the environment ( $\Delta$ VE) as a dynamic interdependent network (see Fig. 2).

The Virtual Trillium Trail System

3.1 The System

The Virtual Trillium Trail application was developed between 2005 and 2008. The first 2007 prototype ran on a Dell XPS Gen 2 laptop and required a high-end NVDIA GeForce Go 6800 Ultra graphics card. The virtual field trip reported in this paper occurred in a PC-lab classroom at the University of Pittsburgh (see Fig. 3). Each PC had the application installed; the application ran as an Unreal map in a stand-alone instance of the Unreal Tournament application [ 35].

Figure    Fig. 3. A photograph of students using the Virtual Trillium Trail in the PC lab at the University of Pittsburgh, May 2007, with the naturalist guide pointing to a virtual fact card on the projected screen.

Using common desktop PCs was advantageous because students were familiar with the keyboard and mouse. The keyboard arrows allowed them to move forward, backward, and to either side. The mouse could be used to pan and rotate the view. The control keys were used to switch modes: Flying (Ctrl-F), walking (Ctrl-W), jumping (Ctrl-J), and running or swimming occurred automatically based on environmental context. The controls were known, guessed, or learned quickly. The one-ear headset was easily used for sound. Accessing the textual fact cards consisted of moving the cursor close to the cards, positioning in front, and then reading.

While problematic for real classroom deployment, video game technology represented a low-cost, high-fidelity platform for rapid development and use in research in 2005-2007 [ 38]. The PC lab had 12 Dell desktop computers, but all lacked the high-end graphics cards desired. While typical of most equipment found in most schools in 2007, this lack negatively impacted the frame-refresh rates, causing them to slow to between 3 and 10 frames per second. Future performance should improve frame rates with faster equipment and/or with compiled, more efficient runtime code.

The VTT was a prototype system that simulated approximately one square mile of terrain and biological plot study and transect data [ 9]. This was simply done by importing GIS data into ESRI [ 39] to generate accurate data-derived DEM terrains. As such, some features were not in the DEM database, so features such as ravines, waterfalls, caves, or rock cliffs were manually added. While the resulting model is not an exact reproduction of the terrain and ecology, it is a very close statistical population density clone at a moment of time, and one based on the biological plot study data and terrain data. The 10-meter samples of real plant data sets were used to create virtual plant data sets. Furthermore, clusters of plants that were not in the original biological plot studies but were observed in the real environment were manually added to improve the realism and accuracy.

Footpaths and the fact cards used in the real field trip were simulated and added to the virtual field trip. The footpaths gave the students a natural affordance for navigation. The Virtual fact cards simulated the Real fact cards, and were composed of a photograph, a schematic drawing, and text including scientific, biological, and ecological facts.

The virtual plants were 3D computer graphic models created in Maya [ 40], textured with photographs taken on-site at the Real location. Over 1,500 locally gathered photographs were taken in 2006 for this purpose. In this prototype, only 36 species of the 102 indigenous plants were modeled. As can be seen in Fig. 4, the level of realism is very high for a 2007 system.

Figure    Fig. 4. Photographs paired with 3D computer graphic models of the flowers used in the May 2007 Virtual Trillium Trail prototype.

3.2 Research Importance of Scientific Visualization

This research is different from previous projects in several important ways. A critical point in this research is that the Virtual Trillium Trail is based on a real place so as to minimize any programmer's or designer's unintentional introduction of misconceptions. “What you see is what it is” (WYS-I-WII) was the paradigm followed. Basing elements on reality was a critical design factor for success, as the goal was to create an authentic simulation of reality for intrinsic learning.

When is it acceptable for the software designer or the educator to deviate into fantasy?

In the past, most systems were “What you see is not what it is.” Regarding prior systems, there were three main concerns with the choice of technology for learning goals, as the medium can influence the meaning:

1. the image quality was often low;
2. the environment was dependent on a designer's interpretation;
3. the navigation, controlled by the designer, restricted freedom of movement.

For example, cartoon-like images used either for style or lower costs resulted in images that lacked detail. Other “virtual field trips” deployed in desktop VR environments, such as Quest Atlantis [ 41] or River City [ 42], used a type of multiuser virtual environment (MUVE) [ 12] based on a low-fidelity platform by Active Worlds [ 43]; they were constructed to reflect a fictitious environment, and therefore, are theoretically capable of introducing unintentional misconceptions. While such variants may be needed and desirable for some educational objectives, it is important to be clear about how a tool can influence learning, and to use it with intention, or guard against either intentional or ignorant misuse.

The Virtual Trillium Trail differs critically from prior virtual environments for education systems which allow for complete freedom of movement and object selection in that it is:

1. a data-based simulation of terrain and plant population species;
2. composed of graphics that are high-fidelity, photorealistic approximations of the real location, ecology, and plants;
3. based on a real informal educational curriculum;
4. based on real informal learning activity and interaction.

3.3 Implementation

The real field trip was to the real Trillium Trail, a wildflower reserve located outside of Pittsburgh, Pennsylvania, and the virtual field trip utilized the Virtual Trillium Trail, a high-fidelity virtual environment simulation of the real-world location installed in a PC-lab classroom in the School of Information Sciences at the University of Pittsburgh. Each experience lasted 1.5 hours, and the study with this sample was conducted over the first two weekends in May 2007.

The field trip activities were carefully controlled to be the same, except that the Real was in the real outdoor location and the Virtual occurred through the VTT software run in a computer lab at the University of Pittsburgh. In the Real, the students followed the naturalist guide on the trail and were presented with information, or the guide responded to the spontaneous inquiries of the students. In the Virtual, each student sat at a PC station, used an earphone on one ear, and independently selected when to listen to the naturalist guide and when to explore independently. At times, students would find something of interest and share with the guide and the class, while at other times, they would share information with the closest student in the room. Frequently, the students did not listen to verbal narration but instead were off exploring the software on their own. The students positioned the mouse to select a direction, tapped on the arrow keys to navigate, and used the space bar to select an object's fact card for more information on an onscreen object. At other times, the students navigated to an audio Sprite and listened to the concept, story, or lesson. The Sprites made it easy for the younger children to listen to and hear the entire concept, whereas the fact cards allowed students to read and flip through, read several times, or leave once some basic information was reviewed. When students found objects of interest, they would mark them on the paper maps distributed at the beginning of the session.

The obvious difference was that the Real was fully multimodal in that a student could smell, taste, and touch plants as well as perceive the temperature and feel the wind in the context of the environment. The students had to stay on the footpath, as it was a nature reserve that restricted human activity, and thus, the wayfinding and navigation were restricted to, essentially, a linear route. Additionally, there were surprises, conveniently provided by nature: a doe and her fawn, a mother turkey hen on her nest, a salamander in the stream, the cry of a red-tailed hawk, or a woodpecker pounding. The Virtual only had plants, so there was no opportunity for interaction with insects, amphibians, reptiles, or mammals.

However, the Virtual did allow for the students to fly, travel off-trail, and freely explore the entire space independently. The simulation allowed for the student and the teacher to interact in dynamic, real-time, synchronous, and asynchronous ways. It also allowed the teacher to present alternative views, such as a fly-through in the forest canopy, not possible in the Real. A more detailed account of the activity is published in Harrington [ 4].

Overview of the Study

4.1 Population and Participants

An overview of the study is presented here, but for a detailed ethnographic report, see Harrington [ 4]. There were 12 volunteer students from a local suburban public elementary school. They were all from a high socioeconomic class with identical high ratings in Enjoyment of Nature and Computer Knowledge on their preexperience user profile survey. This was an ideal user profile for software design activities, since a system that cannot perform adequately under ideal conditions with an ideal user profile would probably not perform well in challenging situations that presented confounding variables. Due to the complexities of working with real children and given the homogeneous student profile (10 of 12 from the same class, curriculum, teacher, preexperience profile statistically identical), the preexperience knowledge was assumed to be practically the same. All students volunteered in compliance with the US federal regulations that protect human subjects under research.

4.2 Materials

Materials included a preexperience user profile demographic survey, a 2D map for recording personally salient and meaningful information in situ, a field guide reference book [ 44], a posttest following either one real or one virtual experience [ 4], a postexperience interview and survey [ 4], and a follow-up microworld study whose results are not reported here.

In the lab, the students were able to use the software in a typical classroom that was equipped with a standard overhead projector; desks with PCs for each student; and with keyboard, mouse, and a one-earphone headset to facilitate both classroom conversation and private listening to the sounds made by the software. The curriculum was the same in all conditions. One group experienced the Real first and then, a week later, the Virtual, while the other group experienced the Virtual first and then, a week later, the Real, so that a meaningful comparison could be made. The educational material embedded in both the Real and the Virtual field trips was based on the fourth grade Natural Communities curriculum provided by the Audubon Society of Western Pennsylvania, located at Beechwood Farms [ 7].

The 2D map (see Fig. 5) was a critical tool used to record learning activity in situ as free-will, student-initiated annotations. The students were instructed to record anything of interest on their map. The maps were numbered with each student's unique ID, and all annotations were recorded after each field trip—Real and Virtual—so that they could be classified by experience type.

Figure    Fig. 5. An example of the 2D map students used to annotate personally important and meaningful finds.

4.3 Methods and Procedure

The Virtual Trillium Trail was created as a high-fidelity software simulation with the highest degree of accuracy. Every attempt was made to control confounding variables and to make a comparison feasible. However, there are differences. These differences represent the innate differences between the two environments. The two environments had the same curriculum, the same naturalist guide teaching, and the same maps, books, and fact cards.

All students received a preexperience user profile survey before experiencing their first field trip, Real or Virtual. An immediate posttest was administered, which showed no difference in scores [ 4]. A week later, they experienced their second field trip under the opposite condition, Real or Virtual. Each field trip lasted for 1.5 hours. The same expert naturalist guide conducted all the field trips. All parents were welcome to be present at all times.

All students had a 2D map and a wildflower guide book during each field trip. The students were encouraged to use their books for reference and to ask questions at will. In addition to the guide telling stories and pointing to items of interest along the way, students could find fact cards placed next to flowers and plants. They were instructed to mark anything of interest on their map, and in this way, an explicit count of objects and events of personal significance in both environments was captured for all students. This count is the main empirical measurement reported in this paper. It is a proxy for personally meaningful and salient information gathered in the in situ learning activity.

The data comparison methods consisted of measuring, in both the Real and Virtual environments, the student's in situ annotation activity by group, by environment, by order, and by type. As the annotation proxies resulted in a reliable object count, it is our claim that such a method is superior to the automatic data logging so often used, because it reduces the noise and gives the researcher an explicit student-created record of events and objects that the student recorded as perceived, observed and meaningful.

The postexperience survey consisted of 14 questions, representing different affective dimensions rated for each experience, Real and Virtual. So, there are two result sets for each question per student. A 5-point Likert Scale was used: 1 = Not at all, 2 = Somewhat, 3 = Average, 4 = Mostly, and 5 = A great deal. The results were applied to comparing and contrasting the real and virtual experiences.

Research Design

The investigative research design used a one-way, within-subject ANOVA, with repeated measures in counterbalanced order. In this design, the factor is Environment, with two levels, Real and Virtual. This design was required for a meaningful postexperience attitudinal survey that allowed direct comparison and contrast of Real and Virtual. Each subject acted as his/her own control [ 45], which allowed for a powerful and elegant statistical comparison of Real and Virtual. Each student's in situ map annotations resulted in a count per cell, and thus, produced the data required for the repeated measurement. Automatic data-logs were not used as they proved too noisy to be useful. This design was required for a meaningful postexperience attitudinal survey that allowed direct comparison and contrast of Real and Virtual. Additionally, there were two groups, thus allowing for a one-way, between-subjects ANOVA test, with Group 1 experiencing the Real-Virtual order and Group 2 experiencing the Virtual-Real order. Finally, t-tests showed significant transfer effects.

Hypotheses

There are several different hypotheses required for complete and accurate comparison. First, using a one-way, within-subject ANOVA, with repeated measures in counterbalanced order, three hypotheses become viable at a level of significance of $\alpha = 0.05$ . One hypothesis is between the environments, Real and Virtual, and the other is between the Orders, First and Second. A subsequent set of hypotheses, using t-tests, investigates transfer of learning activity and the impact of priming, at a level of significance of $\alpha = 0.01$ . Finally, and to give context to the research results, an empirical comparison using a one-way, within-subject ANOVA is employed to generate attitudinal rankings on the learning environments, at an $\alpha = 0.05$ level of significance.

6.1 Hypotheses for Environment and Order

Total in situ map annotations recorded in the Real environment should be higher than in the Virtual, as there will be more nature-driven events and signals from the forest, and thus, more total learning. Total learning in this study context is defined as a holistic, contextual, complex causal chaining of multidimensional factors, and the activation of complete knowledge ontologies, even though such knowledge may be exogenous to simple in-curriculum tests:

H1: Hypotheses for Environment Impact on Total Activity:

\eqalign{&H1_{0}{:}\; \mu \;Total\; Activity_{(Real)} = \mu \;Total\; Activity_{(Virtual),}\cr &H1_{a}{:}\; \mu \;Total\; Activity_{(Real)} > \mu \;Total\; Activity_{(Virtual).}}

Slicing out the subset of data, which represents in-curriculum, Plant-Only learning activity is expected to be equal in the Real and Virtual environments, as the curriculum and environments are statistically identical:

H2: Hypotheses for Environment Impact on Plant-Only Activity:

\eqalign{&H2_{0}{:}\; \mu \;Plant\hbox{-}Only\; Activity_{(Real)}\cr &\qquad\quad = \mu \;Plant\hbox{-}Only\; Activity_{(Virtual),}\cr &H2_{a}{:}\; \mu \;Plant\hbox{-}Only\; Activity_{(Real)} \cr &\qquad\quad\ne \mu \;Plant\hbox{-}Only\; Activity_{(Virtual).}}

The Second experience will show more learning activity than the First for Total and for Plant-Only in situ map annotations. Expected are order effects, as repetition increases activity, wayfinding, spatial awareness, and ability; thus, the Second experience should outperform the First:

H3: Hypotheses for Order Impact on Activity:

\eqalign{&H3_{0}{:}\; \mu \;First\; Experience = \mu\; Second \;Experience,\cr & H3_{a}{:}\;\mu \;First\; Experience < \mu\; Second \;Experience.}

6.2 Hypotheses for Transfer Effects

Transfer effects should be observed in both directions, from Real to Virtual, and from Virtual to Real:

H4: Virtual Primes for and Transfers to Real:

\eqalign{& H4_{0}{:}\; Total \;Activity{:}\; \mu \;Real_{(First)} = \mu \;Real_{(After\; Virtual),}\cr &H4_{a}{:}\; Total\; Activity{:}\; \mu \;Real_{(First)} < \mu\; Real_{(After \;Virtual).}}

H5: Real Primes for and Transfers to Virtual:

\eqalign{&H5_{0}{:}\; Total\; Activity{:}\; \mu \;Virtual_{(First)} = \mu \;Virtual_{(After\; Real),}\cr &H5_{a}{:}\; Total\; Activity{:}\; \mu \;Virtual_{(First)} < \mu \;Virtual_{(After \;Real).}}

Transfer effects for Plant-Only in situ map annotations should be stronger than the Total in situ map annotations statistical tests, as the content and visualization fidelity for plant life are statistically identical in both modalities:

H6: Virtual Primes for and Transfers to Real:

\eqalign{&H6_{0}{:}\; Plant\hbox{-}Only\; Activity{:}\; \mu \;Real_{(First)} \cr & \qquad\quad= \mu\; Real_{(After\; Virtual),}\cr &H6_{a}{:}\; Plant\hbox{-}Only\; Activity{:}\; \mu\; Real_{(First)}\cr &\qquad\quad < \mu \;Real_{(After\; Virtual).}}

H7: Real Primes for and Transfers to Virtual:

\eqalign{&H7_{0}{:}\; Plant\hbox{-}Only\; Activity{:}\; \mu\; Virtual_{(First)} \cr &\qquad\quad= \mu \;Virtual_{(After\; Real),}\cr &H7_{a}{:}\; Plant\hbox{-}Only\; Activity{:}\; \mu \;Virtual_{(First)} \cr &\qquad\quad< \mu\; Virtual_{(After\; Real).}}

6.3 Hypotheses for Attitudes

Comparisons of attitudinal ranks will show relative perceived strengths and weaknesses of Environments. All attitudes resulting from the Real and Virtual, respectively, will give supporting evidence to be equal to, greater than, or less than, dependent on overall multisignal parity, not including signal convulsion:

\eqalign{&H8_{0}{:}\; \mu \;Attitudes_{(Real)} = \mu\; Attitudes_{(Virtual),}\cr &H8_{a1}{:}\; \mu\; Attitudes_{(Real)} > \mu \;Attitudes_{(Virtual),}\cr &H8_{a2}{:}\; \mu\; Attitudes_{(Real)} < \mu\; Attitudes_{(Virtual).}}

Detailed Empirical Results

7.1 Overview of Results

Empirical learning activity, as measured by the map annotations, is higher in the Total data set for the Real environment (see Table 1). However, learning activity for the data subset of Plant-Only map annotations is identical. Notably, order effects show the addition of a Second experience, Real or Virtual, to result in more learning activity than resulted from the First experience alone (see Fig. 6). Transfers occur in both directions, with the stronger effect shown in the Plant-Only data subset. Overall, students preferred the Real to the Virtual as the more compelling learning environment.

Table 1. Data of the Total and Plant-Only Map Annotation Counts by Environments

Figure    Fig. 6. Bar charts showing environment and order effects on in situ learning activity.

7.2 Results for Environment and Order

$H1_{a}{:}\; \mu \;Total\; Activity_{(Real)} > \mu \;Total\; Activity_{(Virtual).}$

According to the above accepted hypothesis, ${\rm H1}_{\rm a}$ , the Real Environment results in significantly higher Total learning activity than the Virtual. A one-way, within-subject ANOVA, with repeated measures in counterbalanced order, shows a significant and strong effect, ${\rm F}(1,11) = 4.68$ , ${\rm p} = 0.05$ , thus giving supporting evidence that the Real ( ${\rm M} = 4.5$ , ${\rm SD} = 2.71$ ) resulted in more Total learning activity than did the Virtual ( ${\rm M} = 2.83$ , ${\rm SD} = 3.43$ ). However, this relationship did not hold for the subset Plant-Only data set, as shown below in $H2_{0}$ :

\eqalign{&H2_{0}{:}\; \mu \;Plant\hbox{-}Only\; Activity_{(Real)}\cr &\qquad\quad= \mu \;Plant\hbox{-}Only\; Activity_{(Virtual}).}

The Real and Virtual environments are equal when the in-curriculum, Plant-Only learning activity data subset is analyzed separately. For the data subset of Plant-Only map learning activity, a one-way, within-subject ANOVA with repeated measurements in counterbalanced order shows no effect ( ${\rm F}(1,11) = 0.00$ , ${\rm p} = 0.95$ ). Thus, for Plant-Only data subset learning activity, the Real ( ${\rm M} = 2.75$ , ${\rm SD} = 1.96$ ) and the Virtual ( ${\rm M} = 2.83$ , ${\rm SD} = 3.43$ ) are identical.

In terms of Order, the Second experience resulted in higher learning activity than the First, independent of Environment, as shown in ${\rm H3}_{\rm a}$ :

$H3_{a}{:}\; \mu \;First\; Experience < \mu\; Second \;Experience.$

A one-way, within-subject ANOVA with repeated measures gives supporting evidence that the Second experience, independent of the environment, results in higher map annotation learning activities for both the Total map annotations, ${\rm F}(1, 11) = 16.23$ , ${\rm p} = 0.002$ , and the Plant-Only map annotations, ${\rm F}(1, 11) = 49.00$ , ${\rm p} < 0.000$ . The Second Total map annotation mean ( ${\rm M} = 5.75$ , ${\rm SD} = 2.18$ ) was greater than that of the First ( ${\rm M} = 1.58$ , ${\rm SD} = 2.53$ ), and the Second Plant-Only map annotation mean ( ${\rm M} = 4.82$ , ${\rm SD} = 2.12$ ) was also greater than that of the First ( ${\rm M} = 0.75$ , ${\rm SD} = 1.42$ ).

7.3 Results for Transfer Effects

If there is no transfer of skill, in situ learning activity should be the same. If there is a significant difference, then there is evidence of transfer. The t-test for two independent groups (one-tail, posthoc analysis) has been carried out to help explain the results.

For the Total map annotations, there was a significant difference, ${\rm t} \;\!\!=\;\!\! -2.29$ , ${\rm df} \;\!\!= \;\!\!10$ , ${\rm p} \;\!\!= \;\!\!0.023$ (one-tailed, $a \;\!\!= \;\!\!0.01$ ), between the conditions, Real (First) ( ${\rm M} = 2.8$ , ${\rm SD} = 2.6$ ) and Real (After Virtual) ( ${\rm M} = 5.8$ , ${\rm SD} = 1.9$ ), in the Independent Samples t-test. This result gives supporting evidence of the value of priming for the Real first with the Virtual, as the mean value significantly increased from 2.8 to 5.8 counts:

$H4_{a}{:}\; Total\; Activity{:}\; \mu \;Real_{(First)} < \mu\; Real_{(After\; Virtual).}$

If transfer effects behave like order effects, could not the opposite relationship be true? Could we see transfer from Real to Virtual? In the past, such an analysis was impossible for life-critical training, such as pilot flight training, medical student operation simulation, or combat military training. But, for educational and learning applications, the question is valid and viable. The results for Total map learning activity show the Virtual (After Real) ( ${\rm M} = 3.8$ , ${\rm SD} = 2.4$ ) to be significantly higher, ${\rm t} = -3.88$ , ${\rm df} = 10$ , ${\rm p} = 0.00$ (one-tailed, $\alpha = 0.01$ ), than the Virtual (First) ( ${\rm M} = 0.0$ , ${\rm SD} = 0.0$ ) in the Independent Samples t-test. This result gives supporting evidence of the value of reinforcement of the Real with the Virtual, as the mean value significantly increased from 0.0 to 3.8 counts. Note that no learning activity occurred in the Virtual when used First and independently:

$H5_{\rm a}{:}\; Total\; Activity{:}\; \mu\; Virtual_{(First)} < \mu\; Virtual_{(After\; Real).}$

For the Plant-Only map annotations, the Real (After Virtual) ( ${\rm M} = 4.0$ , ${\rm SD} = 1.26$ ) resulted in significantly higher activity ( ${\rm t} = - 2.83$ , ${\rm df} = 10$ , ${\rm p} = 0.00$ (one-tailed, $\alpha = 0.001$ )) than the Real (First) ( ${\rm M} = 1.5$ , ${\rm SD} = 1.8$ ) in the Independent Samples t-test. This result gives supporting evidence of the value of priming for the Real with the Virtual, as the mean value significantly increased from 1.5 to 4.0 counts:

$H6_{a}{:}\; Plant\hbox{-}Only\; Activity{:}\; \mu\; Real_{(First)} < \mu \;Real_{(After\; Virtual).}$

There was a significant difference between the Plant-Only annotations for the Virtual (First) and the Virtual (After Real) ( ${\rm t} = - 5.38$ , ${\rm df} = 10$ , ${\rm p} = 0.000$ (one-tailed, alpha $= 0.01$ )). There were more annotations in the Virtual (After Real) ( ${\rm M} = 5.67$ , ${\rm SD} = 2.58$ ) than in the Virtual (First) ( ${\rm M} = 0.00$ , ${\rm SD} = 0.0$ ) in the Independent Samples t-test. This result gives supporting evidence of the value of reinforcement of the Real with the Virtual, as the mean value significantly increased from 0.0 to 5.67 counts:

\eqalign{&H7_{a}{:}\; Plant\hbox{-}Only\; Activity{:}\; \mu \;Virtual_{(First)} \cr &\qquad\quad< \mu \;Virtual_{(After\; Real).}}

7.4 Results for Attitudes on Environments

Student subjective attitudes were gathered using a postexperience survey [ 4]. The comparative ranks as a natural scale—on usability, attitudes, emotional reactions, esthetic assessments, and subjective reflections for both the Real and Virtual environments—give requisite framing and context to the results. Each environment was experienced in opposite counterbalanced order and directly compared using a one-way, within-subject ANOVA (see Table 2). The survey did not capture Plant-Only data, but allowed the student to compare and contrast the total learning experience.

Table 2. Real and Virtual Attitudinal, Emotional, and Affective Ranks (One-Way ANOVA, Within-Subject, Repeated Measurements)

7.4.1 Results on Attitudes

Attitudes ranked as equal represent an exciting accomplishment in interface design and virtual environment design for intrinsic learning environments, as the Virtual matches the Real. Attitudes ranked the same were: Awe and Wonder, Sense of Calm, Assessment of Beauty, Disinterest, Sense of Excitement, Level of Curiosity, and Desire to Share. Furthermore, the attitude of Desire to Share was statistically identical, ${\rm F}(1,11) = 0.00$ , ${\rm p} = 1.0$ , thus suggesting that programs like the VTT may prove effective for teaching collaboration and team-building:

$H8_{0}{:}\; \mu\; Attitudes_{(Real)} = \mu\; Attitudes_{(Virtual).}$

Conclusion

Significantly higher Total learning activity, combined with higher attitudinal ranks for Learning in the Real, gives supporting evidence to the claim that more Total learning activity occurs in the Real. However, Plant-Only learning activity is identical in Real and Virtual, which gives supporting evidence to the claim that the Virtual may be used for in-curriculum material. Thus, the Real is the superior learning environment overall, but the Virtual shows value for carefully targeted learning objectives of in-curriculum material, especially when the real environment is not available.

This result does not imply that schools should substitute “expensive” real-world field trips with “inexpensive” virtual reality field trips, but it does suggest the value of virtual environments for learning, if used properly. The empirical data show order and transfer effects. Therefore, for schools to maximize the learning impact of real-world field trips, they should prime the students before, and reinforce afterward, with follow-up virtual reality field trips. If only one experience is possible, use the Real. If the Real is impossible, carefully use the Virtual.

Great scientific exploration and discovery can only logically come from real and true environmental awareness, curiosity, a sense of wonder, and the ability to inquire when we recognize that we do not know. We must remember to “know that we do not know,” and therefore, cannot program artifacts, such as virtual environments, that are as authentic as the real world. These new tools require sensitive understanding to maximize their potential while respecting their limitations. We can design and build virtual environments for ecology education to support particular goals of factual, conceptual, and contextual awareness; strategically use features of free navigation and information annotation; and respect the interaction of form and function. At the same time, we should not use these powerful emotional and intellectual tools to mislead, frame, sell, or advance a biased or destructive agenda. In final conclusion, virtual environments present opportunities for research of the child-computer-environment interface. However, much future research is required to understand the complex interactions of learning in virtual worlds and to responsibly harness this powerful new technology as it approaches the truth.

Acknowledgments

The author wishes to thank Dr. Peter Brusilovsky, major PhD dissertation advisor, for his kind assistance and critical guidance; Dr. Susan Kalisz, professor of biology at the University of Pittsburgh; and Jessica Dunn, graduate student researcher in Pitt's Biology Department, for their Trillium Trail data sets. She would also like to thank Ms. Gabi Hughes, coordinator of the Environmental Education Program, Audubon Society of Western Pennsylvania, for her participation and advice. Most importantly recognized are all of the teachers, parents, and students for volunteering.

References

• 1. P. Brusilovsky, “Adaptive Hypermedia,” User Modeling and User-Adapted Interaction, vol. 11, nos. 1/2, pp. 87-110, 2002.
• 2. D.A. Norman, The Psychology of Everyday Things, pp. 187-217. Basic Books Inc., 1986.
• 3. S.K. Card, T.P. Moran, and A. Newell, The Psychology of Human-Computer Interaction. L. Erlbaum Assoc. Inc., 1983.
• 4. M.C.R. Harrington, “An Ethnographic Comparison of Real and Virtual Reality Field Trips to Trillium Trail: The Salamander Effect,” Children, Youth and Environments, Special Issue on Children in Technological Environments, N.G. Freier and P.H. Kahn, eds., vol. 19, no. 1, http://www.colorado.edu/journals/cye, 2009.
• 5. E. Nash, G. Edwards, J. Thompson, and W. Barfield, “A Review of Presence and Performance in Virtual Environments,” Int'l J. Human-Computer Interaction, vol. 12, no 1, pp. 1-41, 2000.
• 6. Myst, http://mystworlds.ubi.com/us/myst10th/index.php, 2008.
• 7. Beechwood Farms Nature Reserve, Beechwood Farms Outdoor Discovery Hike, unpublished manuscript, Audubon Soc. of Western Pennsylvania, 2005.
• 8. Conservation Council of the Fox Chapel Area and Squaw Run Area Watershed Assoc. Inc., Parks in the Fox Chapel, O'Hara, and Indiana Township Area: A Guide to the History and Character of Eight of Them, E. Stehle, ed., 1988.
• 9. S. Kalisz, Plot Study of Trillium Trail Wild Life Reserve, unpublished raw data, Univ. of Pittsburgh, 1996-2006.
• 10. D. Allison, B. Wills, D. Bowman, J. Wineman, and L.F. Hodges, “The Virtual Reality Gorilla Exhibit,” IEEE Computer Graphics and Applications, vol. 17, no. 6, pp. 30-38, Nov./Dec. 1997.
• 11. S. Barab, et al., “Situationally Embodied Curriculum: Relating Formalisms and Contexts,” Science Education, vol. 91, no. 5, pp. 750-782, 2007.
• 12. C. Dede, J. Clarke, D.J. Ketelhut, B. Nelson, and C. Bowman, “Students' Motivation and Learning of Science in a Multi User Virtual Environment,” Proc. Am. Educational Research Assoc. (AERA) Ann. Meeting, 2005.
• 13. A. Johnson, T. Moher, S. Ohlsson, and M. Gillingham, “The Round Earth Project—Collaborative VR for Conceptual Learning,” IEEE Computer Graphics and Applications, vol. 19, no. 6, pp. 60-69, Nov./Dec. 1999.
• 14. C. Dede, J. Clarke, D.J. Ketelhut, B. Nelson, and C. Bowman, “Students' Motivation and Learning of Science in a Multi User Virtual Environment,” Proc. Am. Educational Research Assoc. (AERA) Ann. Meeting, 2005.
• 15. A. Johnson, T. Moher, S. Ohlsson, and M. Gillingham, “The Round Earth Project—Collaborative VR for Conceptual Learning,” IEEE Computer Graphics and Applications, vol. 19, no. 6, pp. 60-69, Nov./Dec. 1999.
• 16. M. Salzman, C. Dede, and B. Loftin, “ScienceSpace: Virtual Realities for Learning Complex and Abstract Scientific Concepts,” Proc. IEEE Virtual Reality Ann. Int'l Symp., pp. 246-253, 1996.
• 17. A. Bobick, S. Intille, J. Davis, F. Baird, C. Pinhanez, L. Campbell, Y. Ivanov, A. Schutte, and A. Wilson, “The KidsRoom: A Perceptually-Based Interactive and Immersive Story Environment,” Presence: Teleoperators and Virtual Environments, vol. 8, pp. 367-391, 1999.
• 18. A. Johnson, T. Moher, Y. Cho, D. Edelson, and E. Russell, “Learning Science Inquiry Skills in a Virtual Field,” Computers & Graphics, vol. 28, pp. 409-416, 2004.
• 19. M. Roussou, “Learning by Doing and Learning through Play: An Exploration of Interactivity in Virtual Environments for Children,” ACM Computers in Entertainment, vol. 2, p. 10, 2004.
• 20. B. Campbell, P. Collins, H. Hadaway, N. Hedley, and M. Stoermer, “Web3D in Ocean Science Learning Environments: Virtual Big Beef Creek,” Proc. Web3D Symp., pp. 85-91, 2002.
• 21. R. Jackson, and E. Fagan, “Collaboration and Learning within Immersive Virtual Reality,” Proc. Third Int'l Conf. Collaborative Virtual Environments (CVE '00), 2000.
• 22. Puget Sound, PRISM, http://www.prism.washington.edu, 2005.
• 23. G.H. Wheless, C.M. Lascara, A. Valle-Levinson, D.P. Brutzman, W. Sherman, W.L. Hibbard, and B.E. Paul, “Virtual Chesapeake Bay: Interacting with a Coupled Physical/Biological Model,” IEEE Computer Graphics and Applications., vol. 16, no. 4, pp. 52-57, July 1996.
• 24. T.A. Mikropoulos, A. Katsikis, E. Nikolou, and P. Tsakalis, “Virtual Environments in Biology Teaching,” J. Biological Education, vol. 37, no. 4, pp. 176-181, 2003.
• 25. J.M. Morie, K. Iyer, D. Luigi, J. Williams, A. Dozois, and A. Rizzo, “Development of a Data Management Tool for Investigating Multivariate Space and Free Will Experiences in Virtual Reality,” Applied Psychophysiology and Biofeedback, vol. 30, no. 3, pp. 319-331, 2005.
• 26. R. Aggarwal, J. Ward, I. Balasundaram, P. Sains, T. Athanasiou, and A. Darzi, “Proving the Effectiveness of Virtual Reality Simulations for Training in Laparoscopic Surgery,” Annals of Surgery, vol. 245, no. 5, pp. 771-779, 2007.
• 27. D.A. Bowman, C. North, J. Chen, N. Polys, P.S. Pyla, and U. Yilmaz, “Information-Rich Virtual Environments: Theory, Tools, and Research Agenda,” Proc. Symp. Virtual Reality Software and Technology (VRST '03), 2003.
• 28. R.P. Darken, and J.L. Sibert, “Navigating Large Virtual Spaces,” Int'l J. Human-Computer Interaction, vol. 8, pp. 49-71, 1996.
• 29. B.G. Witmer, J.H. Bailey, B.W. Knerr, and K.C. Parsons, “Virtual Spaces and Real World Places: Transfer of Route Knowledge,” Int'l J. Human-Computer Studies, vol. 45, no. 4, pp. 413-428, 1996.
• 30. M. Flaxman, “Visual Simulation of the Interaction between Market Demand, Planning Rules, and City Form,” Proc. SIGGRAPH, 2004.
• 31. S. Papert, Mindstorms: Children, Computers, and Powerful Ideas, second ed., p. 162. Basic Books, 1993.
• 32. M. Csikszentmihalyi, Flow the Psychology of Optimal Experience. Harper Perennial, 1991.
• 33. K.E. Dill, and J.C. Dill, “Video Game Violence: A Review of the Empirical Literature,” Aggression and Violent Behavior, vol. 3, pp. 407-428, 1998.
• 34. Secondlife.com, http://www.secondlife.com, 2008.
• 35. UnReal Technology, https://www.epicgames.com, 2008.
• 36. Virtual Williamsburg, http://research.history.org/DHC/VW.cfm, 2008.
• 37. WolfQuest.com, http://www.wolfquest.org, 2008.
• 38. J. Jacobson, and M. Lewis, “Game Engine Virtual Reality with CaveUT,” Computer, vol. 38, no. 4, pp. 79-82, Apr. 2005.
• 39. ESRI, http://www.esri.com, 2008.
• 40. Autodesk Maya, http://usa.autodesk.com/adsk/servlet/index? id=7635018&siteID=123112, 2008.
• 41. Quest Atlantis, http://atlantis.crlt.indiana.edu, 2008.
• 42. River City Project, http://muve.gse.harvard.edu/rivercity project/index.html, 2008.
• 43. ActiveWorlds, http://www.activeworlds.com, 2005.
• 44. J. Thieret, W. Niering, and N. Olmstead, National Audubon Society Field Guide to Wildflowers Eastern Region. Alfred A. K nopf, 2001.
• 45. S. Siegel, Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, 1956.
• 46. M. Bentley, “Carpe Diem: Making the Most of Teachable Moments,” Science Activities, vol. 32, no. 3, pp. 23-27, 1995.
• 47. A. De Lucia, R. Francese, I. Passero, and G. Tortora, “Development and Evaluation of a Virtual Campus on Second Life: The Case of Second DMI,” Computers & Education, vol. 52, no. 1, pp. 220-233, 2009.
• 48. J. Bailenson, N. Yee, J. Blascovich, A. Beall, N. Lundblad, and M. Jin, “The Use of Immersive Virtual Reality in the Learning Sciences: Digital Transformations of Teachers, Students, and Social Context,” The J. Learning Sciences, vol. 17, no 1, pp. 102-141, 2008.
• 49. R. Pausch, D. Proffitt, and G. Williams, “Quantifying Immersion in Virtual Reality,” Proc. 24th Ann. Conf. Computer Graphics and Interactive Techniques, pp 13-18, 1997.
• 50. C. Cruz-Neira, D.J. Sandin, T.A. DeFanti, R.V. Kenyon, and J.C. Hart, “The CAVE: Audio Visual Experience Automatic Virtual Environment,” Comm. ACM, vol. 35, no. 6, pp. 64-72, June 1992.
• 51. M.C.R. Harrington, “Simulated Ecological Environments for Education (SEEE): A Tripartite Model Framework of HCI Design Parameters for Situational Learning in Virtual Environments,” Dissertation Abstracts Int'l, July 2008.