2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR) (2014)
Sept. 10, 2014 to Sept. 12, 2014
Christoph Resch , EXTEND3D GmbH
Peter Keitler , EXTEND3D GmbH
Gudrun Klinker , TU München
Shader lamps can augment physical objects with projected virtual replications using a camera-projector system, provided that the physical and virtual object are well registered. Precise registration and tracking has been a cumbersome and intrusive process in the past. In this paper, we present a new method for tracking arbitrarily shaped physical objects interactively. In contrast to previous approaches our system is mobile and makes solely use of the projection of the virtual replication to track the physical object and “stick” the projection to it. Our method consists of two stages, a fast pose initialization based on structured light patterns and a non-intrusive frame-by-frame tracking based on features detected in the projection. In the initialization phase a dense point cloud of the physical object is reconstructed and precisely matched to the virtual model to perfectly overlay the projection. During the tracking phase, a radiometrically corrected virtual camera view based on the current pose prediction is rendered and compared to the captured image. Matched features are triangulated providing a sparse set of surface points that is robustly aligned to the virtual model. The alignment transformation serves as an input for the new pose prediction. Quantitative experiments show that our approach can robustly track complex objects at interactive rates.
Three-dimensional displays, Cameras, Tracking, Iterative closest point algorithm, Feature extraction, Graphics processing units, Mobile communication
C. Resch, P. Keitler and G. Klinker, "Sticky projections — A new approach to interactive shader lamp tracking," 2014 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), Munich, Germany, 2014, pp. 151-156.