Over the last decade or so, the related fields of virtual reality, augmented reality, and haptics have seen tremendous progress and expanded application domains. New graphics and haptic displays, faster and distributed computing hardware, better authoring software, and human factors studies lead to systems that are closer to meeting the user's high expectations. The new awareness in the visualization and graphics community lead to a call for papers for this special iisue on haptics, virtual, and augmented reality of our transactions. The four guest editors and the editorial office had the demanding task of selecting 11 papers out of nearly 100 manuscripts that were received. We relied heavily on the many reviewers of this special issue and we would like to thank them warmly for the diligent work they did.

One of the current challenges in virtual and augmented reality is omnidirectional stereo graphics, specifically the way to capture video that can then be seen in stereo, regardless of the user's viewing direction. The special issue starts with the paper written by Tanaka and Tachi on "TORNADO: Omnistereo Video Imaging with Rotating Optics." The system uses two cameras and an optics assembly inside a rotating cylindrical shutter surface. Depending on the rotating speed, the stereo image is synthesized through optics (low rpm) or postprocessing (high rpm).

The second paper, "Development of Anthropomorphic Multi-D.O.F. Master-Slave Arm for Mutual Telexistence" by Tadakuma, Asahara, Kajimoto, Kawakami, and Tachi, describes a novel robotic arm. The authors are working toward teleexistence, which places a physical avatar of the operator—a robot—in a remote location. Such technology enables humans to work in remote or hostile environments, such as contaminated sites, unstable mines, burning structures, or battlefields. Here, the authors describe the mechanical design and control algorithms for an anthropomorphic arm on such a robot. The operator controls the robot's arm using an exoskeleton around his own arm and sees what the rest of the robot sees by using a head-mounted display coupled with cameras on the robot's head. Tadakuma et al. compared three control methods and found that impedance control was the most appropriate for their system.

Another issue related to telepresence and distributed virtual environments is fast and smooth communication between remote locations. This is the subject of the paper "Data Streaming in Telepresence Environments" by Lamboray, Würmlin, and Gross. Data streams in networked virtual environments are analyzed and classified according to their traffic characteristics. Special emphasis is placed on geometry enhanced (3D) video. The paper presents a simulated analysis of network latency and bandwidth occurring in a system that connects two CAVE-like environments and displays the user's entire real body as an avatar. The 3D avatar model is constructed in real-time using computer vision techniques applied to the images of a collection of cameras that view the user. Remote collaboration in virtual environments is an important application area; the paper's combination of a detailed system description and careful analysis will provide a solid basis for application development and future research in this area.

A way to improve simulation response in distributed virtual environments is to reduce network load by incorporating more "intelligence" into object models (including avatars). This is the subject of the paper "Dynamic Interactions in Physically Realistic Collaborative Virtual Environments" by Jorissen, Wijnants, and Lamotte. Their approach uses inverse kinematics to provide more realism in avatar movements and make object interactions application independent.

One way to further improve user-object interaction is to model its companion haptic feedback. In the paper "Six-Degree-of-Freedom Haptic Rendering Using Spatialized Normal Cone Search," Johnson, Willemsen, and Cohen present a novel haptic rendering algorithm for displaying interaction forces and torques between two polygonal objects. By using spatialized normal cones, the algorithm detects collision robustly. It maintains local distance extrema between the virtual environment and the object moved by the haptic device to compute the resulting haptic feedback. The approach is demonstrated on models consisting of tens of thousands of triangles, tested on several complex geometric scenes, and applied to a virtual prototyping application.

A special case of user-object interaction is associated with simulations where object surface topology is permanently changed. The paper "Cutting on Triangle Mesh: Local Model-Based Haptic Display for Dental Preparation Surgery Simulation" by Wang, Zhang, Wang, Lee, Lu, and Wang describes a new method to realize stable, realistic cutting simulation with force feedback for dental preparation surgery simulation. A piecewise contact force model along with a dynamic local model-based multirate simulation cutting architecture for decoupling the simulation loop from the haptic updates is proposed. the cutting process is described as a transition between adjacent contact statuses. Vertex deformation of triangle mesh is presented for simulating material removal on the surface of a tooth. Stability analyses have been carried out using passive dynamics criteria and a force-filtering method is used to ensure stability without detailed knowledge of the haptic device.

New modeling approaches, hardware, or proposed applications need to be evaluated from the user's perspective to determine their effectiveness. The paper "Psychophysical Evaluation of In-Situ Ultrasound Visualization" by Wu, Klatzky, Shelton, and Stetten evaluates the "Sonic Flashlight." This is a new device that uses augmented reality techniques to superimpose scanned ultrasound imagery directly onto patients. The authors find that the Sonic Flashlight provides more accurate localization of the body features being sought. More accurate localization in turn results in improved accuracy during biopsy. The stereoscopic display used by the Sonic Flashlight proved to be particularly important in providing users with this accuracy.

Travel in immersive virtual environments is the subject of the paper "Comparison of Path Visualizations and Cognitive Measures Relative to Travel Technique in a Virtual Environment," by Zanbaka, Lok, Babu, Ulinski, and Hodges. The authors study the user's cognition and paths taken in a between-subjects experiment. The study compares four different methods of travel:

The authors found that, for applications where problem solving and information evaluation is important, a large tracked space offers benefits over common virtual travel techniques (such as the use of a joystick).

The next paper in this special issue is "Real-Time 3D Human Capture System for Mixed-Reality Art and Entertainment" by Nguyen, Qui, Xu, Cheok, Teo, Zhou, Mallawaarachchi, Lee, Liu, Teo, Thang, Li, and Kato. The authors present a complete mixed-reality system in which 3D human avatars are displayed on a tabletop virtual environment using a head mounted see-through display. A set of cameras capture the humans, and computer vision techniques enable 3D reconstruction, in real time. The user can tangibly manipulate the avatars by using physical objects on the table. The avatars can interact with each other and also with computer generated characters. This work brings together research from several different areas to produce a robust and efficient system. This system, with its ability to do live reconstruction, will be quite helpful in the development of future systems and applications.

Another artificial/mixed reality system is presented by Broll, Lindt, Ohlenburg, Herbst, Wittkämper, and Novotny in the paper "An Infrastructure for Realizing Custom-Tailored Augmented Reality User Interfaces." The authors describe a three-part development infrastructure for AR user interfaces. MORGAN is the most basic component (a low-level API). It supports access to several input and output devices, distributed and multi-user AR interaction, and also includes a rendering engine.The second part is an interaction prototyping facility implemented in XML on top of predefined interaction components. Users describe a dataflow graph across the components to define the interaction. The XML is loaded and executed at runtime, allowing users to alter interactions while the AR application is running. The final part abstracts successful interactions completely away from the application environment, allowing the same user interfaces to be used in desktop or mobile devices, for example. The authors illustrate the use of these devices in three stage prototyping, urban planning, and educational visualization applications.

The last paper in this special issue is "Achieving Dialogue with Children with Severe Autism in an Adaptive Multisensory Interaction: The "MEDIATE" Project," by Parés, Masri, van Wolferen, and Creed. The authors present a system designed to emphasize interaction rather than synthetic environment content, something novel in therapeutic uses of virtual environments. Specifically, the researchers have created a hexagonal interaction enclosure where autistic children interact with the computer multimodally while receiving abstract feedback. The enclosure has a floor and walls that detect pressure, back-projection display walls, vision sensing, and sound capture and sound generation. At a higher level, there is a Decision Maker program which modifies the behavior of MEDIATE in order to establish and maintain a dialogue with the user. Initial testing on more than 90 autistic children showed that, for the first time, they were able to interact completely on their own and make choices without parental input. Ongoing trials are aimed at gathering information on the impact of play in MEDIATE on the children's general development.

For the first time, this issue of the transactions brings together original articles on haptics, virtual, and augmented reality in a special issue of TVCG. It is our hope that this special issue will result in increased collaboration among researchers in these areas and help facilitate exploration of novel cross-disciplinary ideas for their fields.

Grigore C. Burdea

Ming C. Lin

William Ribarsky

Benjamin Watson

Guest Editors

Grigore C. Burdea received the PhD degree in robotics from New York University in 1987 and joined Rutgers University in 1988. Currently, he is professor in the Department of Electrical and Computer Engineering and director of the Human-Machine Interface Laboratory. His current research interests include haptics, virtual reality, and virtual rehabilitation. He authored more than 100 peer-reviewed publication and several textbooks on virtual reality technology. He served as general chair of the IEEE Virtual Reality International Conference in 2000 and of the Second International Workshop on Virtual Rehabilitation in 2003. He is a senior member of the IEEE

Ming C. Lin received the PhD degree in electrical engineering and computer science from the University of California, Berkeley. She is currently a full professor in the Department of Computer Science at the University of North Carolina (UNC), Chapel Hill. Her research interests include haptics, physically-based modeling, robotics, real-time 3D graphics for virtual environments, and geometric computing. She received several honors and awards, including the US National Science Foundation Young Faculty Career Award, Honda Research Initiation Award, UNC/IBM Junior Faculty Development Award, UNC Hettleman Award for Scholarly Achievements, and several best paper awards. She was the organizer, general chair, and/or program chair of several international conferences. She is a member of the Steering Committee of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation. She has served as an associate editor or guest editor of several journals and magazines. She also coedited the book Applied Computation Geometry. She is a member of the IEEE.

William Ribarsky received the PhD dgree in physics from the University of Cincinnati. He is the Bank of America Endowed Chair in Information Technology at the University of North Carolina Charlotte and the founding director of the Charlotte Visualization Center. His research interests include visual analytics, 3D multimodal interaction, bioinformatics visualization, virtual environments, visual reasoning, and interactive visualization of large-scale information spaces. He has published 100 scholarly papers, book chapters, and books. He is the former chair and a current director of the IEEE Visualization and Graphics Technical Committee (VGTC). He also chairs the Steering Committees for the IEEE Visualization Conference and the IEEE Virtual Reality Conference, the leading international conferences in their fields. Dr. Ribarsky cofounded the Eurographics/IEEE visualization conference series (now called EG/IEEE EuroVis) and the Virtual Reality Conference series was established during his tenure as chair of VGTC. He is a member of the IEEE.

Benjamin Watson received the doctorate degree from the Georgia Institute of Technology's GVU Center. He is currently an assistant professor of computer science at Northwestern University and will soon become an associate professor of computer science at North Carolina State University. His research focuses on human graphics, graphics for and about people. His interests include adaptive display, as well as the intersections between graphics and perception, design, and interaction. His work has been applied to digital entertainment, computer security, financial analysis, medical therapy and assessment, and education. He cochaired the Graphics Interface 2001 and IEEE Virtual Reality 2004 Conferences, and he will cochair the ACM Interactive 3D Graphics and Games 2006 conference hosted at Electronic Arts. He is a coauthor of Level of Detail for 3D Graphics (Morgan Kaufman). He is a member of the ACM and a senior member of the IEEE.