This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Calibration-Free Augmented Reality in Perspective
October-December 2000 (vol. 6 no. 4)
pp. 346-359

Abstract—This paper deals with video-based augmented reality and proposes an algorithm for augmenting a real video sequence with views of graphics objects without metric calibration of the video camera by representing the motion of the video camera in projective space. A virtual camera, by which views of graphics objects are generated, is attached to a real camera by specifying image locations of the world coordinate system of the virtual world. The virtual camera is decomposed into calibration and motion components in order to make full use of graphics tools. The projective motion of the real camera recovered from image matches has the function of transferring the virtual camera and makes the virtual camera move according to the motion of the real camera. The virtual camera also follows the change of the internal parameters of the real camera. This paper shows the theoretical and experimental results of our application of nonmetric vision to augmented reality.

[1] R.T. Azuma, “A Survey of Augmented Reality,” PRESENCE: Teleoperations and Virtual Environments, vol. 6, pp. 355-385, Aug. 1997.
[2] Mixed Reality—Merging Real and Virtual Worlds, Y. Ohta and H. Tamura, eds., Springer-Verlag, 1999.
[3] M. Bajura and U. Neumann, "Dynamic Registration Correction in Video-Based Augmented Reality Systems," IEEE Computer Graphics and Applications, vol. 15, no. 5, Sept. 1995, pp. 52-61.
[4] W. Grimson, G. Ettinger, S. White, P. Gleason, T. Lozano-Perez, W. Wells, and R. Kikinis, “Evaluating and Validating an Automated Registration System for Enhanced Reality Visualization in Surgery,” Proc. Computer Vision, Virtual Reality, and Robotics in Medicine '95 (CVRMed '95), pp. 3-12, Apr. 1995.
[5] M. Tuceryan, D.S. Greer, R.T. Whitaker, D.E. Breen, C. Crampton, E. Rose, and K.H. Ahlers, “Calibration Requirements and Procedures for a Monitor-Based Augmented Reality System,” IEEE Trans. Visualization and Computer Graphics, vol. 1, no. 3, pp. 255-273, 1995.
[6] R. Sharma and J. Molineros, “Computer Vision-Based AR for Guiding Manual Assembly,” PRESENCE: Teleoperators and Virtual Environments, vol. 6, pp. 292-317, June 1997.
[7] M.-O. Berger, G. Simon, S. Petitjean, and B. Wrobel-Dautcourt, “Mixing Synthesis and Video Images of Outdoor Environments: Application to the Bridges of Paris,” Proc. Int'l Conf. Pattern Recognition (ICPR '96), pp. 90-94, 1996.
[8] D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker, and M. Tuceryan, “Real-Time Vision-Based Camera Tracking for Augmented Reality Applications,” Proc. ACM Symp. Virtual Reality Software and Technology (VRST-97), pp. 87-94, Sept. 1997.
[9] D. Koller, G. Klinker, E. Rose, D. Breen, R. Whitaker, and M. Tuceryan, “Automated Camera Calibration and 3D Egomotion Estimation for Augmented Reality Applications,” Proc. Seventh Int'l Conf. Computer Analysis of Images and Patterns (CAIP-97), pp. 199-206, Sept. 1997.
[10] H. Schumann, S. Burtescu, and F. Siering, “Applying Augmented Reality Techniques in the Field of Interactive Collaborative Design,” Proc. SMILE Workshop 3D Structure from Multiple Images of Large-Scale Environments, in Conjunction with ECCV '98, June 1998.
[11] S.-W. Park, Y. Seo, and K.S. Hong, “Real-Time Camera Calibration for Virtual Studio,” J. Real-Time Imaging, to appear.
[12] M.-O. Berger, “Resolving Occlusion in Augmented Reality: A Contour Based Approach without 3D Reconstruction,” Proc. Computer Vision and Pattern Recognition (CVPR '97), 1997.
[13] S. Grosskopf and P. Neugebauer, “The Use of Reality Models in Augmented Reality Applications,” Proc. SMILE Workshop on 3D Structure from Multiple Images of Large-Scale Environments, in Conjunction with ECCV '98, June 1998.
[14] A. Heyden and K. Åström, “Euclidean Reconstruction from Image Sequences with Varying and Unknown Focal Length and Principal Point,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 438-443, 1997.
[15] A. Heyden and K. Åström, “Minimal Conditions on Intrinsic Parameters for Euclidean Reconstruction,” Proc. 22nd Asian Conf. Computer Vision, 1998.
[16] M. Pollefeys, R. Koch, and L. Van Gool, “Self-Calibration and Metric Reconstruction in Spite of Varying and Unknown Internal Camera Parameters,” Proc. Int'l Conf. Computer Vision, pp. 90-95, Jan. 1998.
[17] K.N. Kutulakos and J. Vallino, “Calibration-Free Augmented Reality,” IEEE Trans. Visualization and Computer Graphics, vol. 4, no. 1, pp. 1-20, Jan.-Mar. 1998.
[18] O. Faugeras, “From Geometry to Variational Calculus: Theory and Applications of Three-Dimensional Vision,” Proc. IEEE and ATR Workshop Computer Vision for Virtual Reality Based Human Comm., Jan. 1998.
[19] Y. Seo, M. Ahn, and K.S. Hong, “Video Augmentation by Image-Based Rendering under the Perspective Camera Model,” Proc. Int'l Conf. Pattern Recognition, pp. 1,694-1,696, Aug. 1998.
[20] C.-S. Chen, C.-K. Yu, and Y.-P. Hung, “New Calibration-Free Approach for Augmented Reality Based on Parameterized Cuboid Structure,” Proc. Seventh Int'l Conf. Computer Vision, 1999.
[21] R. Cipolla and N. Hollinghurst, “Human-Robot Interface by Pointing with Uncalibrated Stereo Vision,” Image and Vision Computing, vol. 14, no. 3, pp. 171-178, 1996.
[22] G. Hager, “Calibration-Free Visual Control Using Projective Invariance,” Proc. Fifth Int'l Conf. Computer Vision, June 1995.
[23] C. Zeller and O. Faugeras, "Applications of Non-Metric Vision to Some Visual Guided Tasks," Proc. 12th Int'l Conf. Pattern Recognition,Jerusalem, Israel, 1994.
[24] P. Beardsley, I. Reid, A. Zisserman, and D. Murray, “Active Visual Navigation Using Non-Metric Structure,” Proc. Fifth Int'l Conf. Computer Vision, 1995.
[25] L. Robert, M. Buffa, and M. Herbert, “Weakly-Calibrated Stereo Perception for Rover Navigation,” Proc. Fifth Int'l Conf. Computer Vision, 1995.
[26] D. Forsyth, J. Mundy, A. Zisserman, and C. Rothwell, “Using Global Consistency to Recognise Euclidean Objects with an Uncalibrated Camera,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, June 1994.
[27] A. Shashua,“Projective depth: A geometric invariant for 3D reconstruction from two perspective/orthographic views and for visual recognition,” Proc. Int’l Conf. Computer Vision, pp. 583-590,Berlin, May 1993.
[28] O. Faugeras, "What can be seen in three dimensions with an uncalibrated stereo rig?" Second European Conf. Computer Vision, pp. 563-578, 1992.
[29] R. Hartley, R. Gupta, and T. Chang, “Stereo from Uncalibrated Cameras,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 761-764, June 1992.
[30] Q.-T. Luong and T. Vieville, “Canonic Representation for the Geometry of Multiple Projective Views,” Proc. European Conf. Computer Vision ECCV'94, pp. 589-600, 1994.
[31] R.I. Hartley, "A Linear Method for Reconstruction From Lines and Points," Proc. Int'l Conf. Computer Vision, 1995, pp. 882-887.
[32] A. Heyden, “Reconstruction from Image Sequences by Means of Relative Depths,” Proc. Fifth Int'l Conf. Computer Vision, June 1995.
[33] B. Triggs, “Factorisation Methods for Projective Structure and Motion,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 845-851, 1996.
[34] A. Heyden, R. Berthilsson, and G. Sparr, “An Iterative factorization Method for Projective Structure and Motion from Image Sequences,” Image and Vision Computing, vol. 17, no. 5, 1999.
[35] O.D. Faugeras, Three-Dimensional Computer Vision: A Geometric Viewpoint.Cambridge, Mass.: MIT Press, 1993.
[36] O. Faugeras, “Stratification of Three-Dimensional Vision: Projective, Affine, and Metric Representations,” J. Optical Soc. Am. A, vol. 12, pp. 465-484, Mar. 1995.
[37] B. Triggs, “Autocalibration and the Absolute Quadric,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 609-614, June 1997.
[38] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C. Cambridge Univ. Press, 1992.
[39] OpenGL-ARB, OpenGL Programming Guide. Addison-Wesley, 1993.
[40] Z. Zhang, “Determining the Epipolar Geometry and Its Uncertainty: A Review,” Technical Report 2079, INRIA, France, July 1996.

Index Terms:
Augmented reality, Euclidean geometry, virtual camera, perspective projection, projective geometry, calibration-free method.
Citation:
Yongduek Seo, Ki Sang Hong, "Calibration-Free Augmented Reality in Perspective," IEEE Transactions on Visualization and Computer Graphics, vol. 6, no. 4, pp. 346-359, Oct.-Dec. 2000, doi:10.1109/2945.895879
Usage of this product signifies your acceptance of the Terms of Use.