Issue No. 05 - September/October (2006 vol. 26)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MCG.2006.107
Stacey D. Scott , Massachusetts Institute of Technology
Sheelagh Carpendale , University of Calgary
A digital tabletop is sufficiently different from the standard desktop computer in that it invites researchers to challenge traditional perspectives on human-computer interaction, such as rethinking the use of a computer as a solitary act, the kinds of activities people want to do with computers, the types of social interactions that can be part of digital interactions, and how the interplay between digital and social needs can be supported. These new perspectives come in part from the historical use of tables to support group activities and in part from the growing interest in supporting collaborative activities digitally. Researchers in both academia and industry have stepped enthusiastically up to the plate. As a result, digital tabletop interaction has become an active and varied research area.
Although digital tabletops have been around for nearly 20 years, a serious obstacle to their widespread research is that most sites had to invent and build their own digital table, often at considerable cost and effort. Thus tables varied—often widely—in basic technologies, physical appearance and hardware capabilities, size, display, and input. These often led to idiosyncratic software capabilities, such as display and input-specific interfaces and interaction techniques. It also meant that each site could not easily leverage hardware and software developed by others, requiring constant reinvention of the wheel.
Fortunately, this is changing due to the availability of digital tabletop platforms. Mitsubishi Electric Research Laboratories, for example, developed the DiamondTouch multiuser input surface. 1 Rather than keep it inhouse, they donated this technology, along with a basic tabletop software toolkit called the DiamondSpin, to selected university research laboratories across the globe. 2 Similarly, technologies intended for touch-sensitive vertical displays—such as those produced by Smart Technologies and Mimeo—can be purchased and repurposed as a table simply by laying them flat. This readily available technology encouraged new research on digital tabletops, where groups could concentrate on table interaction instead of low-level hardware and software design. This not only led to a relatively recent renewal of interest in this interaction environment, but the formation of a community, that is, the organization of the first IEEE workshop (held in Adelaide, Australia, in January 2006) devoted specifically to tabletop research. Even as we write this introduction, new tabletop hardware and software platforms are being showcased at the Emerging Technologies exhibit at the ACM Siggraph 2006 conference. 3-5
Of course, these existing technologies provide particular hardware and software combinations, which not only influence but also limit the kinds of digital tabletop systems that can be created. Many laboratories are still investing considerable research efforts on experimentations with new hardware assemblies or advanced programming abilities. It's safe to say that we have not yet found the ideal tabletop hardware solution, research that will take significant effort by the community.
Tabletop displays are part of a general trend of increased research on large-format displays. Thus, this special issue continues the IEEE Computer Graphics and Applications' tradition of showcasing these new interaction environments: for example, in the special issues on large displays in 2005—edited by Kurtenbach and Fitzmaurice (vol. 25, no. 4)—and 2000—edited by Funkhouser and Li (vol. 20, no. 4). Articles have highlighted advances in large-display hardware, as well as advanced interface and interaction techniques that address the unique issues of supporting interaction in large-format workspaces. Most of the interface technologies discussed in these large-display issues were vertical, wall-style displays. While digital tabletops share many of the same interaction design challenges as wall displays, such as physically large surface areas and direct-touch input, a table's horizontal orientation introduces additional challenges for system designers. The articles in this special issue discuss these table-specific issues and present advances in interfaces and interaction techniques that address these issues. This issue also highlights many of the remaining challenges that need to be overcome before digital tabletops can become commonly available commercial technology.
New interaction spaces and issues
An exciting aspect of conducting research related to digital tabletops is that both the hardware and software aspects of tabletop workspaces continue to evolve and branch, providing a wide variety of interaction possibilities and many opportunities to explore alternative design approaches.
The large size, horizontal orientation, and inherently collaborative nature of digital tabletops render many of the standard software interface design approaches—including the traditional desktop metaphor and commonly used interface components (for example, buttons and anchored menus)—unsuitable for supporting user interaction in this digital workspace. Interface items can be physically difficult to reach, textual information can be difficult to read when viewed upside down or at an angle, and the state of standard interface components can become ambiguous when viewed from different angles. For example, due to standard shading techniques, the same button can look ready to press when viewed right way up, and look depressed when viewed upside down.
Additionally, as Geller discusses in the Applications article in this issue ("Interactive Tabletop Exhibits in Museums and Galleries"), digital tabletops encourage a "more-familiar, collaborative atmosphere" than traditional computer technologies, making them well suited for not only the public art and history exhibits described in his article, but also for supporting a wide range of colocated collaboration activities. It's not surprising then that much of the research on digital tabletops is focused on designing workspaces that facilitate multiuser interaction and small-group behaviors, in addition to supporting normal task-oriented functionality. Designing for collaboration adds an additional level of complexity to the design process, over and above the challenges of supporting single-user interaction on a large, horizontal display surface.
These types of difficulties present a fundamental challenge to the development of effective user interfaces for digital tabletops.
Interface challenges related to display hardware
Digital tabletops vary widely in size and shape. Geller's article describes large, executive-boardroom-sized tables that enable simultaneous interaction from dozens of museum visitors, as well as small, round tabletop exhibits that allow only a handful of people to share the experience together. The physical composition of a tabletop surface impacts the types of individual and collaborative interactions possible on the table surface, which, in turn, influences the types of tasks that tabletops might afford.
The amount of digital workspace available to tabletop users also impacts the types of digital interactions that can be provided. And while many tabletop surfaces are significantly large compared to the standard desktop computer display, the pixel-per-inch (ppi) resolution of tabletop displays is still remarkably low in comparison. Although extremely high-resolution (100 ppi or better) desktop LCD displays are readily available, these displays are not currently available at the size of many tabletop surfaces. It's certainly possible to create a digital tabletop by tiling these high-resolution desktop display; however, this approach currently provides about a half an inch of black space between the tiles. Moderately high-resolution tabletops (for example, 46 ppi) can be created through tiled projection, but are still in the minority due to the high cost of projectors with native resolution over XVGA (1,024 × 768 pixels—the current industry standard). Thus, researchers continue to explore the basic tabletop research issues on fairly low-resolution tabletops (for example, 26–35 ppi), with the (likely realistic) expectation that the continued demand for higher resolution, large-screen, home consumer displays will eventually drive down the cost of high-resolution, large-screen displays.
Interface challenges related to accessing distant tabletop areas
Since many digital tabletops have physically large workspaces, it's often difficult for someone to reach an interface item displayed across the table. To help extend the physical reach of tabletop users, Parker et al. (in this issue) consider the TractorBeam technique, which lets a tabletop user select a distant tabletop workspace item by simply pointing a tracked stylus at that item and subsequently pressing a button located on the stylus. Their article describes a series of user studies aimed at exploring the accuracy and usability of their TractorBeam technique in combination with various selection mechanisms. Shen et al. (this issue) discuss alternative interaction approaches for addressing the reach issue. They consider two techniques, called the Context-Rooted Rotatable Draggables and puppetry techniques, which provide alternate interaction methods for viewing and manipulating distant tabletop items using mobile, rotatable, virtual extensions of the distant items.
Interface challenges beyond single point interaction
The need to support simultaneous multiuser interaction is one of the main drivers behind the variations in input devices contained in existing tabletop systems: to date, no single system has provided truly sufficient interaction for every desired usage scenario. The vision systems favored by the art and history tabletop exhibits and the combined acoustic and infrared input tracking technique used by the TViews system (Mazalek et al., in this issue) tend to lack the fine input resolution and robustness to support the fine-grained interaction required by more complex, real-world applications. Existing input technologies that enable higher resolution input, such as the DiamondTouch's capacitive touch sensing surface (see the articles in this issue by Shen et al., Ryall et al., and Morris et al.) and the magnetically tracked styli used by Parker et al. (this issue), don't typically scale well to large table surfaces, often require tethered interaction, or can experience environmental interference.
In recent years, various tabletop systems have been designed to enable the use of tangible user interfaces on the tabletop surface. This interaction approach exploits our natural tendency to use physical objects during traditional tabletop activities and provides opportunities for alternate forms of individual and collaborative interactions. In this issue, Mazalek et al. present an extendible method to support the use of physical interaction objects on a table that enables a large number of physical inputs and the mapping of these objects to digital media items. Their article explores different mapping approaches, including providing physical objects that serve as generic controls or that support more special-purpose functionality in the digital workspace.
Many tabletop systems also provide the capability of recognizing different hand gestures. Some of the art and history exhibits surveyed in Geller's article used vision-based input systems capable of identifying various user hand or body gestures, thus enabling exhibit visitors to use gestures such as grasping or pushing to select or move digital media items. The DiamondTouch 1 capacitive touch system is also capable of identifying various hand postures on its input surface, along with multipoint input from a single user. Shen et al. (this issue) present a number of interaction techniques that exploit this capability, including bimanual interaction techniques for mode switching, manipulating view lenses on a map, and cutting and pasting virtual workspace items. They also discuss design issues related to the use of gestural interaction, including gesture registration, gesture reuse, and proving appropriate user feedback for modal interactions.
Interface challenges in enabling group interaction
One issue that arises with supporting simultaneous multiuser input is whether it's essential to keep track of who is touching where on the table surface. Whether or not the input technology provides this capability affects the possible functionalities, interfaces, and interaction techniques that can be implemented in the system. For instance, a system that cannot distinguish between different users interacting with the tabletop cannot provide independent modes of interaction via the standard mechanism of first selecting a mode and then performing actions corresponding to that mode. Consider a drawing application. If one person selects "draw circle" in a system that cannot distinguish one person from another, this "draw circle" selection would affect everyone's drawing. However, people often wish to simultaneously engage in different types of interactions during tabletop collaboration—for example, one person might wish to draw a circle while the other is creating annotations. One way of addressing this issue is to develop new mechanisms for enabling different users to engage in unrelated task activities. Another way is to use specialty tabletop input technologies that enable the system to know who is touching where. These tabletop systems enable a variety of unique interface design approaches. The article in this issue by Ryall et al. describes the concept of identity-differentiating widgets (iDwidgets), which are multiuser widgets enabled by user-differentiating input technologies. These iDwidgets can be overloaded in a variety of ways depending on which user is interacting with them, including the overloading of associated functionality and visual appearance.
An additional challenge of supporting group interaction at a digital tabletop is enabling users to both independently access and share system functionality. Shen et al. present possible approaches for providing multiple menu bars and multiple areas of focus during collaboration activities. They also explore various interaction methods for adjusting the orientation of individual and groups of items. Morris et al. consider how to mitigate access to shared resources and how the software can enhance and influence the collaboration process. They explore the use of various interface and interaction designs to try to encourage equal participation during group work.
The interplay of tabletop interface challenges
We can discuss the interaction challenges triggered by digital tabletop research at several levels. At the conceptual level, the existence of an interactive tabletop opens up new possibilities of different task types that a computer might support. For the groups using these systems, we should also consider the various group processes (for example, chairing, voting, and consensus building) that they might use to perform their tasks. These processes also impact the appropriateness of the software interaction metaphors used in a given usage scenario. For example, system-based control mechanisms such as data locks and hand-offs might be appropriate for supporting certain group processes and usage situations, while an embodied interaction approach striving to support social minutiae and interpersonal awareness might be more appropriate for others. 6-7 At the media level, we need to consider what will be needed for the intended tasks: documents, images, and/or calculations? Will there be many objects, a few shared objects, a single shared object (such as a large map), or some combination of these?
In this chain of levels the last one is the physical table itself. As discussed earlier, there are, and will likely continue to be, wide variations in the physical designs of available digital tabletop systems. This variation makes digital tabletops well suited for a wide range of task activities. This then brings our discussion of levels full circle as new designs in the physical aspects of digital tables will impact the conceptual level, affecting what types of activities might be supported, and potentially increasing the role that digital tabletops can play in our computer-based activities.
Stacey D. Scott is a postdoctoral researcher in the Humans and Automation Lab at the Massachusetts Institute of Technology, Cambridge, Massachusetts. Her research interests include colocated and distributed computer-supported collaboration; novel interface technologies for interactive, multiuser display systems; and information visualization. Scott has a PhD in computer science on the topic of "territoriality in collaborative tabletop workspaces" from the University of Calgary, Canada. Contact her at email@example.com.
Sheelagh Carpendale is a Canada Research Chair in Information Visualization and directs the Innovations in Visualization group in the Interactions Lab at the University of Calgary. As well as information visualization, her research interests include large displays—particularly tabletops, colocated collaboration, graph layout, interaction design, human-computer interaction, and nonphotorealistic rendering. Her research draws on her dual background in computer science (BSc and PhD from Simon Fraser University) and visual arts (Sheridan College, School of Design and Emily Carr, College of Art). Contact her at firstname.lastname@example.org.