This Article 
 Bibliographic References 
 Add to: 
Handling Motion-Blur in 3D Tracking and Rendering for Augmented Reality
Sept. 2012 (vol. 18 no. 9)
pp. 1449-1459
V. Lepetit, KAIST GSCT UVR Lab., Daejeon, South Korea
Youngmin Park, Qualcomm Austria Res. Center, Wien, Austria
Woontack Woo, CVLab., ISIM, Lausanne, Switzerland
The contribution of this paper is two-fold. First, we show how to extend the ESM algorithm to handle motion blur in 3D object tracking. ESM is a powerful algorithm for template matching-based tracking, but it can fail under motion blur. We introduce an image formation model that explicitly consider the possibility of blur, and shows its results in a generalization of the original ESM algorithm. This allows to converge faster, more accurately and more robustly even under large amount of blur. Our second contribution is an efficient method for rendering the virtual objects under the estimated motion blur. It renders two images of the object under 3D perspective, and warps them to create many intermediate images. By fusing these images we obtain a final image for the virtual objects blurred consistently with the captured image. Because warping is much faster than 3D rendering, we can create realistically blurred images at a very low computational cost.

[1] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004.
[2] M. Ozuysal, P. Fua, and V. Lepetit, "Fast Keypoint Recognition in Ten Lines of Code," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[3] B. Okumura, M. Kanbara, and N. Yokoya, "Augmented Reality Based on Estimation of Defocusing and Motion Blurring from Captured Images," Proc. Fifth IEEE and ACM Int'l Symp. Mixed and Augmented Reality (ISMAR '06), 2006.
[4] D. Claus and A. Fitzgibbon, "Reliable Fiducial Detection in Natural Scenes," Proc. Eighth European Conf. Computer Vision (ECCV '04), 2004.
[5] G. Klein and D. Murray, "Improving the Agility of Keyframe-Based SLAM," Proc. European Conf. Computer Vision (ECCV), 2008.
[6] J. Fischer, D. Bartz, and W. Strasser, "Enhanced Visual Realism by Incorporating Camera Image Effects," Proc. IEEE/ACM Int'l Symp. Mixed and Augmented Reality (ISMAR), 2006.
[7] G. Klein and D. Murray, "Simulating Low-Cost Cameras for Augmented Reality Compositing," IEEE Trans. Visualization and Computer Graphics, vol. 16, no. 3, pp. 369-380, May 2010.
[8] E. Malis, "Improving Vision-Based Control Using Efficient Second-Order Minimization Techniques," Proc. IEEE Int'l Conf. Robotics and Automation (ICRA '04), 2004.
[9] S. Lieberknecht, S. Benhimane, P. Meier, and N. Navab, "A Dataset and Evaluation Methodology for Template-based Tracking Algorithms," Proc. IEEE Eighth Int'l Symp. Mixed and Augmented Reality (ISMAR '09), 2009.
[10] Y. Park, V. Lepetit, and W. Woo, "ESM-Blur: Handling & Rendering Blur in 3D Tracking and Augmentation," Proc. IEEE Eighth Int'l Symp. Mixed and Augmented Reality (ISMAR '09), 2009.
[11] G. Klein and T. Drummond, "Tightly Integrated Sensor Fusion for Robust Visual Tracking," Image and Vision Computing, vol. 22, no. 10, pp. 769-776, Sept. 2004.
[12] S. Baker and I. Matthews, "Lucas-Kanade 20 Years On: A Unifying Framework," Int'l J. Computer Vision, vol. 56, no. 3, pp. 221-255, Feb./Mar. 2004.
[13] S. Benhimane and E. Malis, "Homography-Based 2D Visual Tracking and Servoing," Int'l J. Robotics Research, vol. 26, no. 7, pp. 661-676, 2007.
[14] C. Mei and I. Reid, "Modeling and Generating Complex Motion Blur for Real-Time Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2008.

Index Terms:
solid modelling,augmented reality,image matching,image motion analysis,image restoration,object tracking,rendering (computer graphics),3D rendering,motion-blur,augmented reality,ESM algorithm,3D object tracking,template matching-based tracking,image formation model,virtual objects,intermediate images,image fusion,Tracking,Rendering (computer graphics),Three dimensional displays,Cameras,Jacobian matrices,Robustness,Computational modeling,efficient second-order minimization.,Augmented reality,computer vision,object tracking,object detection,motion-blur
V. Lepetit, Youngmin Park, Woontack Woo, "Handling Motion-Blur in 3D Tracking and Rendering for Augmented Reality," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 9, pp. 1449-1459, Sept. 2012, doi:10.1109/TVCG.2011.158
Usage of this product signifies your acceptance of the Terms of Use.