The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - July-September (2011 vol.4)
pp: 222-232
Published by the IEEE Computer Society
Bertrand Schneider , Stanford Univ., Stanford, CA, USA
Patrick Jermann , Rolex Learning Center, EPFL, Lausanne, Switzerland
Guillaume Zufferey , Rolex Learning Center, EPFL, Lausanne, Switzerland
Pierre Dillenbourg , Rolex Learning Center, EPFL, Lausanne, Switzerland
ABSTRACT
We investigated the role that tangibility plays in a problem-solving task by observing logistic apprentices using either a multitouch or a tangible interface. Results showed that tangibility helped them perform the task better and achieve a higher learning gain. In addition, groups using the tangible interface collaborated better, explored more alternative designs, and perceived problem solving as more playful. Mediation analysis revealed that exploration was the only process variable explaining the performance for the problem-solving task. Implications of this study are discussed in terms of the benefits of tangibility for education and directions for future research.
Introduction
The Traditional image of the computer as a keyboard and a mouse attached to a box is changing with the recent development of technologies which embed computational capabilities into clothes, pieces of furniture, and buildings. These technologies open new opportunities for designers to create innovative forms of interaction, based on gestures, body movements, or physical manipulation of real objects. Among them, Tangible User Interfaces (TUIs) allow users of computer systems to interact with digital content through the manipulation of physical objects. The implicit knowledge people have of everyday objects gives a strong feeling of directness to these interfaces and make them intuitive for novices.
A body of literature explores the opportunities offered by TUIs for education, namely by their natural support for collaborative activities, physical interactions, and external representations. Many applications have been developed in a great variety of domains and tasks, such as color-mixing activities [ 1], digitally augmented paper and book [ 2], and literacy development [ 3]. These are only a few examples of the use of TUIs in education, interested readers are directed to the extensive review proposed by O'Malley and Fraser [ 4].
Despite a growing theoretical corpus supporting the use of TUIs in education, there is still a lack of quantitative studies evaluating their educational benefits. In this contribution, we report an empirical experiment which contributes to fill this gap, by comparing the use of a Tangible and a Multitouch User Interface in a problem-solving activity. More precisely, we aim at assessing whether the physicality of TUIs leads to learning benefits, increases performance, and improves the quality of collaboration.
We report a study that was carried out using the Tinker Environment, a tabletop learning environment developed for apprentices in logistics who follow a dual apprenticeship that combines training on the job four days a week and schooling for one day. The Tinker Environment addresses the gap between theory and practice that is inherent to the dual organization of the training. The goal of the system is to contextualize problem solving at school via a small-scale warehouse simulation. Abstract concepts about the organisation of a warehouse (e.g., economics, management) can be illustrated in the concrete setting of the warehouse that is closer to everyday experience at the workplace than traditional teaching approaches. The objective of our study is to identify the benefits of tangibility for logistics apprentices. Does the concrete manipulation of a small-scale warehouse facilitate problem solving and learning? To find out, we compare apprentices' performance using the physical small-scale warehouse with an alternative version of the Tinker environment based on a multitouch interface.
After describing related work in TUIs for education, we introduce the context of this work. We then report the experiment we conducted, present the results and finally discuss them in terms of learning benefits and implications for education.
2. Tangible User Interfaces
The term TUIs was introduced by Ishii and Ullmer [ 5] to describe novel forms of human-computer interactions based on the physical manipulation of everyday objects. The idea of using physical tokens as interfaces to computer systems goes back to the work of Fitzmaurice et al. on Graspable User Interfaces [ 6]. These interfaces build on the intuitive knowledge people have of everyday objects, and take advantage of their rich physical affordances. They offer a tight coupling between real and digital objects, thanks to their similarity in terms of physical shape or types of manipulations that can be applied to them.
The tangible approach allows the creation of intuitive interfaces, supporting collaborative activities, two-handed interactions, and providing users with a strong feeling of directness. Applications of TUIs have been developed in a large variety of domains and took many different forms, from simple tokens to task-specific objects. Jacob et al. [ 7] developed a tangible interface helping office workers in the task of organizing and arranging data objects, represented by magnetic pucks. RFID technology is used to detect the position of the pucks placed on a board. A projector adds digital information which is relevant to the task, e.g., unsatisfied constraints in a planning activity. A good example of an application based on specific physical objects is Tangible Geospace [ 8]. Small-scale models of the MIT campus are used to interact with a map. When objects are placed on an interactive surface, the map becomes aligned to their position and orientation. Jumping to a location is simply done by placing the appropriate object in the environment, and zooming is achieved by moving two objects closer to or further apart from each other. Another noticeable system is Illuminating Clay [ 9] which introduces the concept of Continuous TUIs. A laser coupled with a projector allows the system to capture the shape of a landscape (made of clay) modeled by users real-time geo-spatial information is projected on top of it.
3. Tangible Interfaces and Learning
3.1 Advantages and Promises
Three properties of TUIs are especially promising in a pedagogical context [ 4].
First, tangible interfaces add new physical actions to the repertoire of computer-based learning activities. Not surprisingly the added value of sensori-motor experience is often described in projects involving young children [ 10]. The advantage of graspable and tangible interfaces relies on the idea that they enable an enactive mode of reasoning [ 11] as well as empirical abstractions of sensori-motor schemes [ 12]. Price [ 13] reports qualitative results that suggest that tangible environments support playful learning among children. This view is also supported by Marshall [ 14], who argues that “As interaction with tangible interfaces is assumed to be more natural or familiar than with other types of interface, they might be more accessible to young children, people with learning disabilities or novices, lowering the threshold of participation.” However, the assumption of a facilitated playful learning [ 15] has been poorly studied with experimental methods. The physicality of the interface also favors exploratory behaviors [ 16] because tangibles are natural and intuitive to use (physical familiarity enables exploration) and at the same time constrain the range of configurations that can be made with the material (physical resistance which guides exploration). A similar complementarity of enabling and guiding has been described by Suthers [ 17] for notations and how they affect collaboration as representational guidance. Moreover, it is believed that expressive activities are also promoted: a TUI allows users to externalize his representation of a situation, which permits them to make clear inconsistencies, conflicting beliefs, and incorrect assumptions [ 18].
Second, the coupling of TUIs with augmented reality (the system projects information on top of physical artifacts) allows for a very close mapping between tangible input and digital output, between the physicality of an object, the manipulations it affords, and the abstraction of visualization. Computer augmentations make abstractions “touchable”: they present them as external representations [ 19] for inspection and discussion embedded in a concrete situation. An appropriate representation can be helpful in multiple ways: to reduce the amount of cognitive effort required to solve problems by grouping information or reducing the complexity of a problem, for example with the help of a diagram [ 20]; to alternatively represent reality and thus influence problem solving [ 21]; to constrain the range of inferences that can be made (for example texts permit ambiguity in a way that graphics cannot easily accommodate [ 22]). According to Ainsworth [ 23], Multiple External Representations (MERs) can bring several learning advantages: first, two representations can complement each other, and encourage learners to try more than one strategy to solve a problem. Second, learners' familiarity with one representation can help the interpretation of a less familiar one. In addition, MERs support the construction of more abstract and deeper understanding, which increases the likelihood to achieve insight in a problem-solving task. Furthermore, it seems that it also helps students to transfer their knowledge to new situations [ 24]; some other studies also show that encouraging students to actively integrate different representations and interact with them improve verbal comprehension and learning [ 25]. Typically, a tangible interface can integrate two representations by mixing the input and the output of the software. The input is often a manipulation of physical objects related to familiar objects : for example moving small-scale buildings in the Urp project [ 26]. The output can be displayed on a screen or augment the objects with the help of a projector : for instance the wind's flow around the buildings.
Third, TUIs support face-to-face collaborative activities, allowing multiple users to interact with the system while collaborating with each other [ 27]. At the difference of vertical display, the horizontal display does not interfere with verbal and gaze interactions. New forms of interactions such as exchanging objects to each other outside the field of the camera. Social learning processes could be promoted and enhanced when using a tangible interface, because having access to a shared representation of the problem facilitates interaction and reduces cognitive load [ 28].
3.2 Possible Drawbacks
Some other perspectives on manipulating physical material suggest that TUI can also lead to no or negative learning outcomes. Indeed, manipulatives have been particularly studied in mathematics learning; it was believed that presenting objects rather than abstract numbers could help children to discover and better understand mathematical operations.
Uttal's work [ 29] challenges this view by presenting some metaanalyses in which inconsistent and limited effect have been found; moreover some longitudinal studies show that children do not really acquire mathematical concepts and have difficulties in generalizing from using physical material. Uttal argues that manipulatives are most of the time not interpreted as representing something else (at least for children), and the difficulty to see them as symbols causes these mixed results. Consequently, this way of teaching mathematics cannot be considered as an ideal solution in its actual form. Additional research may be necessary in order to understand which conditions are necessary for positive outcomes.
Furthermore, Clements [ 30] posits that the danger of physical material is to constrain reflection and abstract thinking by blocking the learner in an “action mode.” From his point of view, it is more difficult to take distance from an activity with manipulatives and reflect on it. As a result, it is possible that manipulatives per se are not detrimental, but they must take place in activities where some space for high level thinking is provided.
3.3 Empirical Studies
A few studies have tested the effect of physical manipulatives compared to digital ones. Triona and Klahr [ 31] have compared a task where children had either to point and click or to grab a physical material to build experiments on springs. They found that subjects performed equally well in both conditions, meaning that neither the tangibility nor the computer was more efficient. Olkun [ 32] conducted the same type of experiment comparing computer and concrete manipulatives when solving 2D geometry problems. His results showed that fourth and fifth grade students better performed a tangram task when using a computer or tangible objects compared to a filler activity; however physical manipulation did not bring better results than a digital use. Marshall et al. [ 33] conducted a study using a standard piagetian task where users had to evenly place weights on a balance beam to reach an equilibrium; this task involves the understandings of complex rules (e.g., distance of the weight to the center, different of weight between items) and is widely used within developmental and cognitive psychology. Comparing physical and graphical material, Marshall did not find any learning effect between the two conditions. In another study, Finkelstein et al. [ 34] tested how physical circuits versus simulations affected students' understanding of electronics: after a practice session, students who used the simulation were quicker to build physical circuit compared to the students who used physical material; they were also able to provide better explanations of circuit behavior. In a similar study, Zacharia and Olympiou [ 35] also compared physical versus digital material in a learning exercise about heat and temperature; they found that both material equally supported students' understanding. Finally, Chini et al. [ 36] found that there were advantages for each type of manipulative (e.g., each material supported the understanding of different concepts). When studying pulleys' behavior, students using physical manipulatives better understood the concept of Effort Force, while students using virtual pulleys better grasped the concept of work.
Even if these studies are not enough to prove that tangible material is not improving learning, it indicates that tangibility alone may not be a panacea. It is possible that the learning situation needs a mediatory variable to be really effective (like integrating relevant MERs and therefore a variety of levels of abstraction). And it is also possible that the gain cannot be measured only by learning between pretest and posttest. As Marshall proposed, there can be accessory benefits like enhanced collaborative learning, increased engagement, playful learning, etc. Changes in these variables may bring several benefits in the long run, so it is essential to take a deeper look at them during experiments. Moreover cognitive processes have to be more closely observed, in terms of strategies and level of abstraction. Consequently, methodological improvements have to be made in order to specifically extract the influence of tangibility as it is developed in the next section.
4. Multitouch Interfaces and Learning
Since this paper proposes an empirical study comparing a tangible and a multitouch interface, we will now provide a short review of this field of study. Multitouch interfaces share common characteristics with TUI: they promote collaboration [ 37] compared to vertical surfaces [ 38], for instance by increasing awareness in groups [ 39], and physical interaction equity [ 40]. From a learning perspective, multitouch interfaces are believed to promote explorative and creative activities, as shown by the DigiTile project [ 41].
Compared to tangible interfaces, several advantages are worth noticing. First of all, multitouch displays allow users to visualize and interact with an infinity of objects; those objects may be resized, deformed, or combined in a large variety of ways due to their virtual nature. From this point of view, TUI are severely limited by the physical shape of the objects. Moreover, on a multitouch table the workspace is not cluttered by tangibles and no information is hidden in the third dimension: every piece of information is equally visible for each user at any time.
However, multitouch interfaces may also present a number of disadvantages compared to TUI: tangible items are easier to acquire and manipulate compared to virtual items with a mice or multitouch interfaces [ 42]; furthermore, multitouch displays are limited to 2D, whereas TUI take advantage of the third dimension (and thus can potentially lead to an increased collaboration, if awareness of others' actions is improved by a wider range of gestures); also, the “present at hand” nature of tangibles contributes to balance the level of participation in collaborative settings [ 43]; finally physical directness and familiarity of everyday objects are believed to be well suited for learning in a constructivist approach [ 41]. The comparison between multitouch and tangible interfaces in our context (teaching logistic) is further developed in the next sections.
5. Comparison of Tangible and Multitouch Interfaces
The objective of the current study is to examine the influence of the physicality of the representation manipulated by users. A core difference between tangible and multitouch interfaces is precisely how the objects of interest are represented for the user. Multitouch interfaces allow users to interact with a digital representation (that can be acted upon by finger gestures) whereas tangible interfaces involve the user with a physical instance of the object (that can be grasped). We refer to the central difference between these two types of interfaces as tangibility or physicality.
Both tangible and multitouch interfaces have their strengths and weaknesses; in some contexts users will benefit from the flexibility of the visualization of a multitouch interface (an aritrary number of abstract and concrete representations can be displayed), while physical objects may provide a faster and more direct way to explore a problem space.
The rationale for comparing a tangible with a multitouch interface rather than with a keyboard and mouse is that multitouch maintains a certain level of directness (or embodiment) for the interaction with the system, and allows for quicker trial and error exploration. With a multitouch interface, users interact by touching the problem representation: rather than “grabbing and placing” physical shelves on the table, they use their fingers to “drag and drop” digital representations of the shelves. We believe that this directness is required to enable users to tinker with the problem space.
While the present study focuses on tangibility, other properties of TUIs are worth studying. The level of metaphor of objects (i.e., the ressemblance of interface objects with their real counterparts) might influence how the users interact with the system. For example, in an urban simulation, houses may be represented as detailed small-scale mockups, as 3D monochrome wooden blocks, or as flat 2D paper rectangles. The use of digital augmentations (i.e., additional information is projected on top and around interface objects) in TUIs differentiates these systems from mere manipulatives. The dynamic nature of the augmented interface influences central aspects of the interaction like the provision of help and guidance, as well as the display of feedback.
The influence of tangibility on problem-solving situations and learning has been poorly studied until now; indeed, we lack empirical results for determining how this kind of interface influences problem solving and learning. The only empirical studies show little or no benefits which compromises the use of physical manipulatives instead of digital ones. However, many theoretical points of view make us think that tangibility may positively influence learning by improving several process variables, including collaboration, integration of representations, exploration, and playfulness of the task.
6. Context
In Switzerland, 70 percent of youth 16 years of age, follow vocational training after obligatory school. Concretely, they work in a company four days a week, and spend the fifth day studying in a professional school. This dual training has the advantage of enabling apprentices to practice essential skills in a real workplace while simultaneously providing them with a theoretical education. In reality, however, the articulation between the company and the school is difficult because apprentices find it hard to use concepts from the school at the workplace and because school exercises are too theoretical [ 44]. An attempt to bridge this abstraction gap was made by developing the Tinker Lamp, a tabletop logistics simulation, which allows apprentices to build a small-scale warehouse via a Tangible User Interface and to simulate its operation through Augmented Reality. The Tinker Lamp allows the integration of both concrete and conceptual aspects of logistics.
6.1 The Tinker Lamp
The Tinker Lamp [ 45] is a tabletop learning environment. Users place small-scale shelves as well as loading and expedition docks on the table to layout a warehouse. A camera recognizes the position of the objects via fiducial markers [ 46]. An augmented reality system [ 47] uses this information to project additional information on top of the warehouse via a projector.


Fig. 1. A warehouse built by an apprentice with augmented information (accessibility information displayed on the top of the shelves and forklift simulation).




The augmentations can be manifold. For example, small colored dots projected on top of the shelves indicate whether they are accessible by forklifts (green dot) or not (red dot) from the docks. During simulation, users see forklifts moving through the warehouse to store and retrieve pallets. More abstract information that represents the state of the warehouse is projected on paper windows called Tinker Sheets [ 48]; e.g., number of accessible shelves, graphs showing the remaining goods, efficiency of the warehouse (average time for a forklift to retrieve any merchandise in the warehouse).
The physical aspects of the simulation are controlled by grabbing and placing the shelves and docks on the table (see Fig. 1). Additional controls are provided through Tinker Sheets, which allow to configure the simulation. For example, users can start a simulation, increase or decrease the speed of the forklifts, visualize the inventory, modify the demand, or delivery delay for some goods. The Tinker Sheets are simple pieces of paper featuring a form-based control interface; users can set parameter values just by moving little black tokens on the paper. The simplicity of the interface helps users to quickly control the system, and it allows the teacher to flexibly create suitable situations for learning.
A multitouch version of the TinkerLamp has been created to conduct experimental studies (see Fig. 2). The tangible and the multitouch versions provide the same set of functionalities, namely moving (by dragging with one finger) and rotating (by moving with two fingers) one, or several shelves; in this case, a lasso selection was available to manipulate a large amount of items. The following functionalities were also provided by touching icons on the top of the interface: undo/redo an action, add a shelf. To remove a shelf, the user just had to drag and drop the object outside the screen area. Additional details can be found in a study in [ 49], Lucchi et al. comparing the two interfaces; also, see Fig. 3 for a visual comparison between the two interfaces.


Fig. 2. The Tinker Lamp, which can be used either with a tangible or a multitouch interface.






Fig. 3. The two conditions compared in this study: tangible interface (on the left) and multitouch interface (on the right).




7. Problem Space
7.1 Tinkering Rather than Sketching
During the initial phase of the project, we have observed in real classrooms that apprentices face difficulties when designing warehouses with paper and pencil [ 44]. The goal of the design activity in class is to address the influence of the layout on the efficiency of a warehouse, in terms of the proportion of total surface used for storage, as well as the relationship between the size of alleys and the types of forklifts that can be used (larger alleys lose space, but allow faster trucks). We saw that apprentices were not used to sketching to explore possible layouts and alternative designs. Rather, than sketching, apprentices directly implemented a final solution, which required time consuming attention to details, precise scaling and measuring of distances as well as labeling. As a consequence only one possible solution was implemented. Difficulties also stem from the need to translate the problem from an abstract format (a textual description of the requirements for the warehouse) into a set of constraints in the design space (a figurative representation of the warehouse on an A4 sheet). As a result, the dimensions of the warehouse, and the relative size of the shelves, are sometimes not plausible.
In stark contrast, with tangible small-scale shelves, apprentices quickly start working on layout problems by placing a few shelves, and then progressively propagating rows of shelves across the physical design space. While it takes several hours to design a warehouse with paper and pencil, it takes 10 minutes to implement one with tangibles. Different solutions can be explored quickly by grabbing and placing shelves in various configurations. The scaling of the design is embedded in the dimension of the tangible shelves and does not need to be calculated. Finally, apprentices make frequent references to their everyday experience and working context during the design. For the apprentices, tinkering is much more efficient than sketching.
However, it is worth mentioning that tangibles are not intended to completely supplant sketching in this project. The long term vision is, first, to make classroom exercises more accessible (e.g., to focus on logistic concepts rather than technical skills, like drawing); and then progressively bring the student into more complex representations (either virtual or printed layouts) once students have mastered basic concepts. The rationale behind this approach is that designing warehouse with paper and pencil may be too challenging for apprentices who are not experts in sketching; thus, lowering the level of abstraction for some activities may be beneficial for beginners. However, both sketching and tinkering have their own advantages; as a matter of fact, sketching should be used in a complementary way in the curriculum so that core skills (e.g., scale understanding) can still be mastered by students.
7.2 Tangible or Multitouch
Why are apprentices so much better at designing warehouses with a tangible interface? Is it the ability to explore solutions by trial and error, or the fact that they manipulate objects and configure a 3D warehouse, that leads to this spectacular difference in efficiency?
The present study, conducted with logistic apprentices who are the intended end users of the system, aims at assessing whether the physicality of tangible interfaces is associated with learning benefits, performance increase, and collaboration quality improvements. The task we chose for the experiment is modeled after the classroom activity described above: groups of two to four apprentices have to optimize the layout of a warehouse in order to learn about the tradeoff between storage space and forklift efficiency.
We hypothesize that learners will explore more alternatives and therefore achieve better warehouse designs with a tangible compared to a multitouch interface. The more extensive exploration of the design space might also have a positive impact on the induction of general warehouse design rules, following the idea that more generic rules are inferred from a larger number of examples. It is also probable that the richer representation of the warehouse in the tangible condition will enable apprentices to apply professional expertise in evaluating warehouse designs and facilitate the discovery of general warehouse design principles. This hypothesis relies on the idea that the “psychological distance” between the representation of the problem and the apprentices' everyday working context is smaller with the 3D tangible interface, and hence facilitates the transfer of expertise.
We are finally interested in two additional process variables, namely the quality of collaboration and the perceived playfulness of the interface. It is indeed possible that the interaction with tangible objects leads to a better coordination and a more fluid interaction. With multitouch, fingers need to stay put on the table and moving across each other's field of action might be cumbersome, whereas with tangibles it is easier to reach across the partner's field and build concurrently. Finally, the fun and playfulness of tangibles has been often cited as a potential benefit.
In summary, our hypotheses are:

    1. Outcome variables: logistic apprentices using a tangible interface will better solve a problem-solving task and have a higher learning gain than apprentices using a multitouch interface.

    2. Process variables: collaboration, explorative behaviors and playfulness will be increased when using a TUI compared to a multitouch interface.

8. Method
8.1 Population
The subjects were 82 apprentices (nine female and 73 male) from the CPNV (“Centre Professionnel Nord Vaudois”) aged between 16 and 40 years $({\rm mean} = 20,\; {\rm SD} = 5.4)$ . The dyads were composed by alphabetically following the class list and were randomly assigned to either the tangible or multitouch condition. Among the 41 dyads, two were excluded because of technical problem during the experiment. Thirty apprentices were in their first year ( $N = 16$ in the touch condition, 14 in the tangible condition), and 48 in their second year ( $N = 18$ in the touch condition, 30 in the tangible condition). More second year students used the tangible interface because the multitouch surface had minor technical problems at the end of the experiment (mainly calibration). Finally, all subjects were comfortable with the operation of the Tinker Lamp as they used it at least once during the school year.
8.2 Procedure
The experiments took place in a closed and soundproof room of the multimedia library at the professional school (CPNV in Yverdon, Switzerland). Pairs of apprentices were allowed to leave their class for one hour to participate in the experiment. When the two apprentices arrived at the experiment room, the dyad was randomly assigned either to the “tangible” or the “multitouch” condition. The experimenter welcomed them and thanked them for their participation. Then, they were asked to individually fill a pretest which tested the students' ability to analyze several warehouses' layouts in terms of space management and efficiency during approximately 10 minutes (see the “measures” section for more details about the pre and posttest). After that, the experimenter invited the two apprentices to stand in front of the Tinker Lamp to solve a problem. He told them the following instructions: “you have to build a warehouse in order to include the maximum number of shelves possible. This is your primary goal; moreover the efficiency of the built will also be assessed (meaning the mean distance from each shelf to each dock). You will have approximately 25 minutes to build your warehouse. Try to make most of the shelves accessible and to maximize the space used.” He also explained that they could use a control sheet in order to observe numerical properties of their warehouse. The following information was “augmented” on the paper: number of shelves, number of accessible storage places, average distance to the expedition dock, average distance to the reception dock, and average distance to both docks. Two pillars and a wall were placed at strategic places in the warehouse to make the problem more challenging. Then, as explained to the apprentices, they had 25 minutes maximum to build their warehouse. During the last five minutes, the experimenter repeatedly informed them of the time remaining in order to make them finish promptly; the time allowance was decided on the basis of a pilot study conducted at the Ecole Polytechnique Fédérale de Lausanne (EPFL) with undergraduate students. The actions allowed for the task were: adding a shelf, moving it, removing it, consulting the control sheet, and moving the entry and departure dock. The whole exercise was recorded by two cameras and a boundary microphone placed on the foot of the Tinker Lamp. After completion of the problem-solving task, apprentices were asked to individually fill the posttest, which was identical to the pretest, except that the warehouses' layout varied. They had the same amount of time as the pretest to complete it. Finally, they had to complete a questionnaire, which addressed demographical questions (age, sex, study year) and the flow questionnaire (measuring playfulness). They were allotted all the time they wanted for this last step, but no one exceeded 60 minutes in finishing the whole experiment. This limit was required by the teacher to minimize the time each student would spend outside the classroom. Once they had completed the experiment, they were thanked for their participation and asked to go back to the classroom.
8.3 Material
The setup consisted of a Tinker Lamp placed on a horizontal table (total dimension: 107 x 107 cm). In both conditions, the Tinker Lamp was used to project the warehouse on the table (size of the projection: 38 x 53 cm). In the multitouch condition, the multitouch surface detected the apprentices' fingers and allowed them to build the warehouse by “dragging and dropping” rectangular shapes projected by the lamp. In the tangible condition, apprentices built the warehouse by “grabbing and placing” small-scale plastic shelves on the table.
Each dyad was recorded with one webcam at the top of the table and one video recorder on the side. An additional microphone (AKG C 400 BL) was used to record the apprentices' voices in order to get a proper sound signal.
Several questionnaires and observation grids were employed. First, a pretest/posttest was designed and validated by the teacher to observe learning gain. Second, the collaboration was assessed by an adapted grid of the rating scheme developed by Meier et al. [ 50]; and third, we measured the flow with an adapted questionnaire of Novak and Hoffman [ 51].
8.4 Measures
Primary goal performance was measured by counting the number of accessible shelves in the warehouse, which was automatically computed by the software. We also assessed how efficiently the warehouses were built by the apprentices by analyzing the log files of the experiments. In agreement with teachers, efficiency was determined to be measured by calculating the average distance from both loading and reception docks to every shelf. These two measures were used as a performance score for the task.
A learning gain was calculated by subtracting the posttest score from the pretest score. The pre and posttest questionnaires had two parts: the first part addressed the apprentices' ability to judge the layout of warehouses with regards to space management and efficiency of navigation, and the second part addressed general warehouse design principles (“to gain space, is it more useful to have short/narrow/large/long alleys? Multiple answers are possible”; the same questions were asked concerning time-saving alleys). The second part was identical for the pretest and the posttest, and the first part used different layouts.
In addition to outcomes, we observed motivational, behavioral and cognitive process variables. First, collaboration was assessed by the rating scheme developed by Meier et al. [ 50]. 9D were used to capture the main characteristics of collaboration (communication, joint information processing, coordination, interpersonal relationship, and motivation). Each dimension was rated on a five point scale and the sum of these formed the final collaboration score.
Second, exploration was measured by counting the number of moves for every object (docks or shelves) in the logfiles. This value was adapted with a threshold for each condition, in order to filter out micro movements (e.g., millimeter differences falsely detected by the software as an intentional move).
Finally, we used a flow questionnaire to capture the main characteristics of the playfulness variable. Csikzentmihalyi and Corporation [ 52] has proposed the following definition for the concept of flow: “the state in which people are so involved in an activity that nothing else seems to matter; the experience itself is so enjoyable that people will do it even at great cost, for the sheer sake of doing it.” We would not detail the theory here, but invite the reader to consult the book by Csikzentmihalyi which details every aspects of this theory. In our case, we operationalized flow by adapting the tool developed by Novak and Hoffman [ 51]. This questionnaire can be easily adapted and allowed us to assess how much users enjoy a task. For example, we asked apprentices to rate items like “I forgot about my immediate surroundings when I was building my warehouse,” “building a warehouse challenges me,” “building this warehouse pushed my abilities to their limits,” “I consider myself knowledgeable about good warehouse building techniques,” or “I felt excited during this task,” on a five point Lickert scale (1 = “strongly disagree,” 5 = “strongly agree”). Then, all items were summed up for each participant to compute a final “flow” score.
The choice of using a flow questionnaire to measure playfulness was motivated by the fact that some authors (e.g., Novak et al. [ 53]) believe that playfulness is central component of flow and highly correlates with this construct. As a consequence, this measure does not only reflect the playfulness of the task, but also the some other constructs (perceived skills, control, challenge, $\ldots$ See [ 53] for more details). Therefore, our measure is not unidimensional. However, since those two variables correlates, we make the assumption that a high flow state is related to a strong feeling of playfulness.
9. Results
9.1 Main Hypothesis: The Performance
The main hypothesis was that participants would better perform in a problem-solving task with a tangible interface than with a multitouch interface. This was mainly measured by the number of shelves accessible in the warehouse, and secondarily by the efficiency of the layout (e.g., the mean distance from each dock to each shelf).


Fig. 4. Boxplots of the performance (number of shelves) between the multitouch and tangible conditions $(p <.001)$ .






Fig. 5. Boxplots of the learning gain between the multitouch and tangible conditions $(p <.05)$ .




As expected, participants in the tangible condition $({\rm N} = 22,\; {\rm mean} = 32.2,\; {\rm SD} = 2.6)$ performed better than in the multitouch condition $(\rm N = 17,\; {\rm mean} = 27.1,\; {\rm SD} = 3.8){,}$$t(32) = 4.873,\; p <.001$ (two tailed), meaning that they placed significantly more accessible shelves on the available surface (see Fig. 4). Moreover, we also assessed the efficiency of the warehouse (meaning the mean distance from every dock to every shelf). Five dyads (three in the tangible condition; two in the multitouch condition) were excluded due to dock inaccessibility. We found that compared to the touch group $(\rm N = 15,\; {\rm mean} = 17.3,\; {\rm SD} = 4.51)$ , the tangible group $(\rm N = 19,\; \rm mean = 14.8,\; {\rm SD} = 3.96)$ tendentially built warehouses with shorter and more direct alleys, $t(32) = -1.73, p = .09$ (two tailed).
Moreover, it was hypothesized that using a tangible interface would provide a better learning gain than using a multitouch interface. Measured by an appropriate pretest/posttest, we computed a score for every subject by subtracting pretest performance from the posttest. Then, we followed the procedure described by Kenny et al. [ 54] for dyadic analysis with undistinguishable members and independence within the dyad. These authors propose a multilevel analysis in order to analyze data from dyads.
Descriptive data are as follows: for tangible condition, ${\rm m} = .43\; ({\rm SD} = 5.4)$ and ${\rm N} = 44;$ for touch condition, ${\rm m} = -2.5\; ({\rm SD} = 5.9)$ and $\rm N = 34$ . The multilevel analysis with the group ID as a random factor yielded a significant effect, $F(1,37) = 6.68,\; p <.05$ , meaning that the tangible group better succeeded than the touch group (see Fig. 5).
9.2 Secondary Results: Mediator Variables
The secondary hypothesis proposed that several variables (e.g., the collaboration between the members of the dyad, a state of flow and more exploration) played a mediating role for explaining the performance. Referring to Table 1, apprentices explored more alternative solutions in the tangible than in the touch condition $(p <0.001)$ . They also found the tangible system more playful $(p <.05)$ , and collaborated more $(p <.01)$ . Concerning collaboration rating, a second judge rated 20 percent of the dyads; interreliability analysis using Krippendorff's alpha was 0.93 [ 55].

Table 1. Mediatory Variables


Then, we estimate the influence of the mediator variables with the method described in the article of Preacher and Hayes [ 56]. Unfortunately, this procedure does not take into account multilevel designs, but considering that the intraclass correlation is not significant for the only individual variable among mediators (playfulness, measured by the flow questionnaire: $r = -.344,\; p = .879$ ), it is quite reasonable to conduct analysis on an individual level [ 54].
We tested for multiple mediation using Preacher and Hayes' bootstrapping methodology for indirect effects based on 5,000 bootstrap resamples to describe the confidence intervals of indirect effects in a manner that makes no assumptions about the distribution of the indirect effects. Interpretation of the bootstrap data is accomplished by determining whether zero is contained within the 95 percent CIs (thus indicating the lack of significance) [ 57]. Results for multiple mediation showed that only exploration (CI: [0.02; 1.13]) was a mediator for performance (see Fig. 6 for a graphical visualization of the model).


Fig. 6. Graphic representation of the two models tested (either with performance or learning gain as a dependant variable). Note: the total effects (weight c) are composed of a direct effect (weight c') and the indirect effect (sum of all a × b weights).




We also tested which variable had a positive effect depending on the condition. Pearson's correlations showed that collaboration, $r = .57$ , $p <.05 (\rm N = 17)$ , and playfulness, $r = .53,p <.05 (\rm N = 17)$ , were strongly related to a better performance in the touch condition. In the tangible condition, the only significant correlation with performance was exploration, $r = .47, p <.05 (\rm N = 22)$ .
10. Discussion
The purpose of this study was to identify how a Tangible User Interface could influence the performance of pairs in a problem-solving task (and more broadly learning). Following an interaction paradigm [ 26], a secondary focus of interest was identified to discriminate which kind of behavioral and cognitive changes were necessary for a positive outcome. Hence, three process variables have been studied in order to answer this question: to which extent apprentices explored different warehouses layouts, how playful the task was for them, and to what extend they collaborated.
Results provided evidence that logistic apprentices better solve a warehouse design task with a tangible interface than with a multitouch interface. This means that tangibility is well suited for understanding and seeking out solutions in a logistic problem. Our results also suggest that tangibility may enhance learning compared to a multitouch interface: indeed, apprentices using the former interface had a better learning gain than the apprentices using the second one. Surprisingly, the “touch” group performed the posttest worse than the pretest. This can be explained by a difference of complexity between the two tests, since the warehouse layouts in the posttest were more complex and more difficult to analyze than those in the pretest (this was noticed afterwards, because the time frame for the study did not allow us to test retest the questionnaire). However, apprentices performed equally well between the pre/posttest for the general questions (identical in the pre and posttest), while they improved or lowered their scores for questions related to the warehouses' layouts. So we can hypothesize that if they had “unlearned” how to analyze a warehouse and how to optimize it, we would have observed losses on every question; consequently, it is more likely that those differences are due to a difference of complexity between layouts rather than students being confused by the activity. Globally, these results support the hypothesis that tangibility improves learning for logistics apprentices, at least for a simple problem-solving task. Our results differ from the study of Triona and Klahr [ 31], in which they found no advantages of either a physical or virtual manipulation of artefacts.
Moreover the study of cognitive and behavioral changes observed during the task brought us many insights into how the apprentices solved the problem. The only variable that yielded a significant intermediary effect was the exploration variable (measured by the number of shelves moved). This means that the most important behavior predicting performance in this task was to explore as many layouts as possible, and that the apprentices who used the tangible interface benefited from this fact. It implies that tangibility has the potential to increase exploration, and by this medium to enhance performance for a simple problem-solving task. The results may have been different in an unlimited time session since dragging and dropping a shelf is slower to carry out in the multitouch condition; however, those results make sense in our context and also from an ecological point of view: the Tinker Lamp and the activity proposed is based on classroom requirements and meant to be used in the classroom, where students only have a limited time to carry out an exercise. Alternatively, a multitouch interface may also increase students' cognitive load and be responsible for those differences in performance; future work should answer this question.
Collaboration quality was also increased by tangibility, even if this variable was correlated with success only in the touch condition. This can be explained by the fact that in the tangible group it was easier to coordinate the work: monitoring each other's activity was probably faster and more natural. In the touch group it was not so simple, because the actions were less diversified (two fingers moving a rectangle are more difficult to track than a hand moving a shelf); consequently, the apprentices felt a stronger need to plan their actions and coordinate their efforts. Our assumption is that the more coordination is required, the more collaboration quality becomes a predictor. A difficult interface forces people to talk to each other more.
Results also showed that playfulness was increased in the tangible condition, which indicates that the task was more adapted to the apprentices' ability and that they felt excited about the task. However, this measure was only correlated with success in the touch condition, meaning that every apprentice in the tangible condition already got a very high flow score; ergo, it did not affect their performance. Hence, the more apprentices were in the flow in the touch condition, the more they had good results. However, an important remark has to be made here: as the flow is defined to be “ a state in which people are so involved in an activity that nothing else seems to matter; the experience itself is so enjoyable that people will do it even at great cost, for the sheer sake of doing it” [ 52], it would be more appropriate to call this variable “playfulness,” because the involvement of the apprentices was not intense enough to meet the criteria of the original theory of Csikzentmihalyi. Also playfulness is believed to be a core component of the flow construct (or at least to highly correlates with it) [ 53].
10.1 Conclusion and Future Work
From our results it appears that the main impact of tangible interfaces is to promote constructive behavior (exploration, collaboration, and playfulness of the task). The link between these process variables and problem-solving performance however, is not systematic. Only increased exploration leads to better performance.
The most important question still needs to be answered: what caused the learning gains observed? Mediation analyses suggest that neither collaboration, exploration nor playfulness were responsible for this improvement. Clearly, more experiments are needed to systematically control the complexity of such learning arrangements.
One possibility is that mediatory variables have a longer-term effect which we were not able to measure with the experimental setting. Indeed, since the problem-solving task was pretty short and simple, we need to observe interactions for a longer period (for instance the effect of this system after several months of utilization). Another possibility, is that uncontrolled variables explain these differences, for instance, the group composition or the strategy adopted by subjects. Finally, it might be that the richer representation offered by the tangible interface (3D small-scale model of shelves versus 2D digital representations) favors apprentices' comprehension. A comparison of the tangible shelves used in this experiment with more abstract tangibles (e.g., 2D paper rectangles) would allow to better identify the importance of the concreteness of representations [ 58].
This study has, however, several important limitations. The main drawback is the limited scope of implication of these results: a more complete factorial plan would have been more appropriate to determine the influence of tangibility. Therefore, future researches are needed to explore at least 5D: first, how can users benefit from a tangible environment for more complex problem-solving task (1)? Because this study details a very simple learning activity, it would be interesting to see how apprentices react to more abstract tasks. Do they stay in an “action mode” as suggested by Clements [ 30], or are they able to reflect on their experiences during the activity? Second, what is the influence of users' expertise when using the tangible interface (2)? Are they advantaged or disadvantaged by the concreteness of the interface? Does it limit the understanding of a concept, or to the contrary improve it? And linked to that question, what is the influence of moving on a concrete-abstract continuum for designing interfaces (3)? Are experts more comfortable being on the abstract side (e.g., multitouch interface), or do they equally or more greatly benefit from a concrete interface (e.g., tangible interface)? Our hypothesis is that the more abstract an interface is, the better it will be suited to expert users. However, this assumption needs to be assessed by additional empirical studies. Fourth, we believe that the size of the group (4) plays a role in the way the group works with a tangible interface, since every learning situation is not appropriate for collaboration. Finally, one critical variable still needs to be studied from an ecological point of view: the effect of time (5) when using the Tinker Lamp. Indeed, it is crucial to determine if the benefits found in this study are caused by the novelty of the multitouch interface; tests in the long run (for several months for instance) are needed in order to answer this specific question.

Acknowledgments

This project is part of Dual-T, a leading house on technologies for vocational training, funded by the Swiss Department of Economic Affairs. The authors wish to thank the logistic teachers Jacques Kurzo and André Ryser for their deep involvement in the project and their perfect organization regarding the experiment management.

    B. Schneider is with Stanford University, Stanford, CA 94305.

    E-mail: schneibe@stanford.edu.

    P. Jermann, G. Zufferey, and P. Dillenbourg are with EPFL - CRAFT, Rolex Learning Center, Station 20, Lausanne CH - 1015.

    E-mail: {patrick.jermann, guillaume.zufferey, pierre.dillenbourg}@epfl.ch.

Manuscript received 11 Mar. 2010; revised 20 July 2010; accepted 17 Aug. 2010; published online 23 Nov. 2010.

For information on obtaining reprints of this article, please send e-mail to: lt@computer.org, and reference IEEECS Log Number TLT-2010-03-0023.

Digital Object Identifier no. 10.1109/TLT.2010.36.

References



Bertrand Schneider received two master's degrees, one in psychology and one in learning and teaching technologies, from the Geneva University. Currently, he is working toward the PhD degree at Stanford University in the learning sciences and technology design (LSTD) program. He was working at the École Polytechnique Fédérale de Lausanne (EPFL) in the Center for Research and Support of Training and its Technologies (CRAFT) as a research assistant for the DUAL-T project. His research interests include the development of new and original interfaces (tangible, multitouch, and paper-based) for collaborative learning.



Patrick Jermann received the MA degree in educational technology in 1996 and the PhD degree in psychology in 2004 from the University of Geneva. He started his career in the TECFA team of the University of Geneva designing and studying online environments featuring innovative collaboration scripts. From 2000 to 2003, he was a visiting researcher in the Learning Research and Development Center (LRDC) at the University of Pittsburgh. Since 2003, he has been a researcher and a lecturer of computer science in the Center for Research and Support of Training and its Technologies (CRAFT) team at the École Polytechnique Fédérale de Lausanne (EPFL). His research interests include the regulation of collaborative activity through a broad range of approaches ranging from educational design to interaction mining and modeling.



Guillaume Zufferey received the MA degree in 2005 and the PhD degree in 2010 in computer science from the École Polytechnique Fédérale de Lausanne (EPFL). His research focused on tangible and paper-based user interfaces for tabletop environments. He is now the cofounder and the CEO of Simpliquity Sàrl, a startup specialized in learning and communication environments based on novel interfaces and augmented reality techniques.



Pierre Dillenbourg received the graduate degree in educational science from the University of Mons, Belgium. He received the PhD degree in computer science from the University of Lancaster, United Kingdom, in the field of educational applications of artificial intelligence. He was a teacher in a elementary school. Currently, he is a professor of computer science at the École Polytechnique Fédérale de Lausanne (EPFL). He started to conduct research in learning technologies in 1984. He has been involved in the CSCL community since the first meeting in 1989 and is the past president of the International Society for the Learning Sciences. His work covers various domain of CSCL, ranging from the design and experimentation of collaboration scripts and interactive furniture to more cognitive projects on dual eye tracking and mutual modeling.
29 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool