The Community for Technology Leaders

Tracking: How Hard Can It Be?

Simon Julier, ITT Advanced Engineering Systems/Naval Research Laboratory
Gary Bishop, University of North Carolina at Chapel Hill

Pages: pp. 22-23


This special issue of IEEE Computer Graphics and Applications includes a sampling of the current state of the art in tracking technology. In the first article, Welch and Foxlin define the "silver bullet" of tracking, the tracker-on-a-chip: a small, self-contained device capable of tracking itself with unprecedented accuracy. Regrettably, this device has yet to be realized. However, their article describes a wide variety of approaches used in today's tracking systems.

The rest of the issue covers what we believe to be a representative sample of current research work. The first three articles discuss real-time viewpoint tracking. Prince et al. provide a clear introduction to a vision-based tracking system that uses a homography (transformation of a plane) to estimate camera movement. Simon et al. describe a tracking system that's capable of using multiple homographies. Ribo et al. describe a tracking system, specifically tuned for outdoor environments, which uses point-like features such as the corners of windows. They implemented their system in a self-contained, wearable computer. The fourth article considers tracking for natural human-computer interaction. Oka et al. describe a finger tracking and gesture recognition system for an augmented desk. It tracks users' hands in infrared and uses a variety of recognition and vision operators to identify palms, fingers, and gestures. Lin et al. describe a face tracking system for animation. Once many small markers are placed on a user's face, a camera views the face and its reflection in two mirrors. It then tracks a "cloud" of points that can be used as control points in animation software.


Two noteworthy trends emerge from the articles collected here. First, the articles that focus on viewpoint tracking emphasize the needs of augmented reality, perhaps the most challenging application for trackers. Second, all the authors use real-time vision-based tracking as a core technology. We find these trends encouraging—after years of hinting at great promise, it seems that computer vision is finally becoming practical. There are two explanations for this. The first is the dramatic increase in computer performance. CPUs have become faster and high-quality cameras can be readily purchased for even the humblest of laptops. Second, it appears that a number of vision-based algorithms have matured and are rapidly becoming widely accepted in the community. Several authors in this issue adopt the same general approach. For example, they used

  • a homography to estimate the translation and rotation from frame to frame of a plane,
  • the Harris feature detector to identify target points, and
  • the random sample consensus (Ransac) algorithm to validate matching.

We hope this special issue will help 3D tracker users appreciate just how difficult the tracking problem is, show that some solutions exist, and inspire researchers to work on new approaches. After all, we really would like the tracker-on-a-chip, and the sooner the better.


We'd like to thank everyone who helped make this issue possible. First, we thank the authors who submitted 19 high-quality articles for this issue—choosing only six was difficult. Second, we thank the small army of reviewers who helped with the selection. Finally, we thank the staff at CG&A for helping to prepare this issue.

About the Authors

Bio Graphic
Simon Julier is a research scientist for ITT Industries at the Naval Research Lab, Washington, D.C. He received a DPhil from the Robotics Research Group, Oxford University, UK. He is a technical lead on the Battlefield Augmented Reality System (BARS) Project. His research interests include mobile augmented reality and large-scale distributed data fusion.
Bio Graphic
Gary Bishop has been thinking about 3D tracking for more than 20 years and he still hasn't grown tired of it. He is an associate professor in the Department of Computer Science at the University of North Carolina at Chapel Hill. His research interests include assistive technology, hardware and software for man-machine interaction, tracking technologies, and image-based rendering. He completed his PhD in computer science at UNC-Chapel Hill in 1984.
62 ms
(Ver 3.x)