This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Stable Real-Time 3D Tracking Using Online and Offline Information
October 2004 (vol. 26 no. 10)
pp. 1385-1391
We propose an efficient real-time solution for tracking rigid objects in 3D using a single camera that can handle large camera displacements, drastic aspect changes, and partial occlusions. While commercial products are already available for offline camera registration, robust online tracking remains an open issue because many real-time algorithms described in the literature still lack robustness and are prone to drift and jitter. To address these problems, we have formulated the tracking problem in terms of local bundle adjustment and have developed a method for establishing image correspondences that can equally well handle short and wide-baseline matching. We then can merge the information from preceding frames with that provided by a very limited number of keyframes created during a training stage, which results in a real-time tracker that does not jitter or drift and can deal with significant aspect changes.

[1] A.W. Fitzgibbon and A. Zisserman, Automatic Camera Recovery for Closed or Open Image Sequences Proc. European Conf. Computer Vision, pp. 311-326, June 1998.
[2] M. Pollefeys, R. Koch, and L. Van Gool, Self Calibration and Metric Reconstruction in Spite of Varying and Unknown Internal Camera Parameters Proc. Sixth Int'l Conf. Computer Vision, pp. 90-96, 1998.
[3] A. Azarbayejani and A. Pentland, "Recursive Estimation of Motion, Structure and Focal Length," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 6, pp. 562-575, June 1995.
[4] P.A. Beardsley, A. Zisserman, and D.W. Murray, Sequential Update of Projective and Affine Structure from Motion Proc. Int'l J. Computer Vision, vol. 23, no. 3, pp. 235-259, 1997.
[5] G. Simon, A. Fitzgibbon, and A. Zisserman, Markerless Tracking Using Planar Structures in the Scene Proc. Int'l Symp. Mixed and Augmented Reality, pp. 120-128, Oct. 2000.
[6] U. Neumann and S. You, "Natural Feature Tracking for Augmented Reality," IEEE Trans. Multimedia, vol. 1, no. 1, Mar. 1999, pp. 53-64.
[7] O.D. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, 1993.
[8] D. Nistér, An Efficient Solution to the Five-Point Relative Pose Problem Proc. Computer Vision and Pattern Recognition, pp. 195-202, 2003.
[9] T. Drummond and R. Cipolla, Real-Time Visual Tracking of Complex Structures IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 7, pp. 932-946, July 2002.
[10] E. Marchand, P. Bouthemy, F. Chaumette, and V. Moreau, “Robust Real-Time Visual Tracking Using a 2D-3D Model-Based Approach,” Proc. Int'l Conf. Computer Vision ,'99, vol. 1, pp. 262-268, Sept. 1999.
[11] D. G. Lowe, Robust Model-Based Motion Tracking through the Integration of Search and Estimation Proc. Int'l J. Computer Vision, vol. 8, no. 2, pp. 113-122, 1992.
[12] D. DeCarlo and D. Metaxas, The Integration of Optical Flow and Deformable Models with Applications to Human Face Shape and Motion Estimation Proc. Conf. Computer Vision and Pattern Recognition, pp. 231-238, 1996.
[13] M. La Cascia, S. Sclaroff, and V. Athitsos, “Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture Mapped 3D Models,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 4, pp. 322-336, Apr. 2000.
[14] F. Jurie and M. Dhome, Hyperplane Approximation for Template Matching IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 7, pp. 996-100, July 2002.
[15] G. Hager and P. Belhumeur, “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, Oct. 1998.
[16] S. Ravela, B. Draper, J. Lim, and R. Weiss, Adaptive Tracking and Model Registration Across Distinct Aspects Proc. Int'l Conf. Intelligent Robots and Systems, pp. 174-180, 1995.
[17] K.W. Chia, A.D. Cheok, and S.J.D. Prince, Online 6 DOF Augmented Reality Registration from Natural Features Proc. Int'l Symp. Mixed and Augmented Reality, 2002.
[18] G. Simon, V. Lepetit, and M.-O. Berger, Computer Vision Methods for Registration: Mixing 3D Knowledge and 2D Correspondences for Accurate Image Composition Proc. Int'l Workshop Augmented Reality, 1998.
[19] L. Vacchetti, V. Lepetit, and P. Fua, Fusing Online and Offline Information for Stable 3 D Tracking in Real-Time Proc. Conf. Computer Vision and Pattern Recognition, June 2003.
[20] C.G. Harris and M.J. Stephens, A Combined Corner and Edge Detector Proc. Fourth Alvey Vision Conf., 1988.
[21] P.J. Huber, Robust Statistics. New York: Wiley, 1981.
[22] Y. Shan, Z. Liu, and Z. Zhang, Model-Based Bundle Adjustment with Application to Face Modeling Proc. Int'l Conf. Computer Vision, July 2001.
[23] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, Cambridge Univ. Press, 2000.
[24] D. Lowe, Distinctive Image Features from Scale-Invariant Keypoints Proc. Int'l J. Computer Vision, 2004, (Accepted for publication).
[25] V. Lepetit, L. Vacchetti, D. Thalmann, and P. Fua, Fully Automated and Stable Registration for Augmented Reality Applications Proc. Int'l Symp. Mixed and Augmented Reality, Sept. 2003.

Index Terms:
Computer vision, real-time systems, tracking.
Citation:
Luca Vacchetti, Vincent Lepetit, Pascal Fua, "Stable Real-Time 3D Tracking Using Online and Offline Information," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1385-1391, Oct. 2004, doi:10.1109/TPAMI.2004.92
Usage of this product signifies your acceptance of the Terms of Use.