This Article 
 Bibliographic References 
 Add to: 
Drift Detection and Removal for Sequential Structure from Motion Algorithms
October 2004 (vol. 26 no. 10)
pp. 1249-1259
In sequential Structure from Motion algorithms for extended image or video sequences, error build up caused by drift poses a problem as feature tracks that normally represent a single scene point will have distinct 3D reconstructions. For the final bundle adjustment to remove this drift, it must be told about these 3D-3D correspondences through a change in the cost function. However, as a bundle adjustment is a nonlinear optimization technique, the drift needs to be removed from the supplied initial solution to allow for convergence of the bundle adjustment to the real global optimum. Before drift can be removed, it has to be detected. This is accomplished through understanding of the long term behavior of drift which leaves 3D reconstructions from short sequences intact. Drift detection boils down to identifying reconstructions of the same scene part that only differ up to a projective transformation. After detection, the drift can be removed from future processed images and an Adapted Bundle Adjustment using correspondences supplied by the drift detection can remove the drift from previous images. Several experiments on real video sequences demonstrate the merit of drift detection and removal.

[1] F. Aherne, N. Thacker, and P. Rockett, The Bhattacharyya Metric as an Absolute Similarity Measure for Frequency Coded Data Kybernetika, vol. 32, no. 4, pp. 1-7, 1997.
[2] M. Bajura, Merging Real and Virtual Environments with Video See-through Head-Mounted Displays Technical Report TR98-036, 17, 1998.
[3] M. Bajura and U. Neumann, "Dynamic Registration Correction in Video-Based Augmented Reality Systems," IEEE Computer Graphics and Applications, vol. 15, no. 5, Sept. 1995, pp. 52-61.
[4] K. Cornelis, M. Pollefeys, and L. Van Gool, Tracking Based Structure and Motion Recovery for Augmented Video Productions Proc. Symp. Virtual Reality Software and Technology, pp. 17-24, Nov. 2001.
[5] K. Cornelis, M. Pollefeys, and L. Van Gool, Lens Distortion Recovery for Accurate Sequential Structure and Motion Recovery Proc. European Conf. Computer Vision, A. Heyden, ed., vol. 2, pp. 186-200, May 2002.
[6] O. Faugeras, Q.T. Luong, and S. Maybank, Camera Self-Calibration: Theory and Experiments Proc. European Conf. Computer Vision, vol. 588, pp. 321-334, 1992.
[7] M. Fischler and R. Bolles, Random Sampling Consensus: A Paradigm for Model Fitting with Application to Image Analysis and Automated Cartography Comm. ACM, vol. 24, pp. 381-395, 1981.
[8] A.W. Fitzgibbon and A. Zisserman, Automatic Camera Recovery for Closed or Open Image Sequences Proc. European Conf. Computer Vision, pp. 311-326, 1998.
[9] R.M. Haralick, H. Joo, C.-N. Lee, X. Zhuang, and M.B. Kim, “Pose Estimation from Corresponding Point Data,” IEEE Trans. Systems, Man, and Cybernetics, vol. 19, no. 6, p. 1426, 1989.
[10] R.I. Hartley, L. de Agapito, I.D. Reid, and E. Hayman, Camera Calibration and the Search for Infinity Proc. Int'l Conf. Computer Vision, pp. 510-515, 1999.
[11] R. Koch et al., "Calibration of Hand-Held Camera Sequences for Plenoptic Modeling," Proc. IEEE Int'l Conf. Computer Vision (ICCV 99), IEEE CS Press, 1999, pp. 585-591.
[12] D. Nister, Calibration with Robust Use of Cheirality by Quasi-Affine Reconstruction of the Set of Camera Projection Centres Proc. Int'l Conf. Computer Vision, pp. 116-123, 2001.
[13] D. Nister, An Efficient Solution to the Five-Point Relative Pose Problem Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 195-202, June 2003.
[14] M. Pollefeys and L. Van Gool, Some Issues on Self-Calibration and Critical Motion Sequences Proc. Asian Conf. Computer Vision, pp. 893-898, 2000.
[15] J. Shi and C. Tomasi, Good Features to Track Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[16] H.Y. Shum and R. Szeliski, Systems and Experiment Paper: Construction of Panoramic Mosaics with Global and Local Alignment Int'l J. Computer Vision, vol. 2, no. 36, pp. 101-130, Feb. 2000.
[17] R. Smith, A. Fitzgibbon, and A. Zisserman, Improving Augmented Reality Using Image and Scene Constraints Proc. British Machine Vision Conf., pp. 295-304, 1999.
[18] C. Tomasi and T. Kanade, Shape and Motion from Image Streams under Orthography: A Factorization Approach Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992.
[19] B. Triggs, “Autocalibration and the Absolute Quadric,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 609-614, June 1997.
[20] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, Bundle Adjustment A Modern Synthesis Vision Algorithms: Theory and Practice, pp. 298-372, 1999.
[21] T. Tuytelaars and L. Van Gool, Content-Based Image Retrieval Based on Local Affinely Invariant Regions Proc. Third Int'l Conf. Visual Information Systems, pp. 493-500, June 1999.

Index Terms:
Geometric correction, registration.
Kurt Cornelis, Frank Verbiest, Luc Van Gool, "Drift Detection and Removal for Sequential Structure from Motion Algorithms," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1249-1259, Oct. 2004, doi:10.1109/TPAMI.2004.85
Usage of this product signifies your acceptance of the Terms of Use.