The Community for Technology Leaders
RSS Icon
Issue No.07 - July (2011 vol.17)
pp: 875-887
Sehwan Kim , UCSB Four Eyes Lab, Santa Barbara
Christopher Coffin , UCSB Four Eyes Lab, Santa Barbara
Tobias Höllerer , UCSB Four Eyes Lab, Santa Barbara
The acquisition of surround-view panoramas using a single hand-held or head-worn camera relies on robust real-time camera orientation tracking and relocalization. This paper presents robust methodology and evaluation for camera orientation relocalization, using virtual keyframes for online environment map construction. In the case of tracking loss, incoming camera frames are matched against known-orientation keyframes to re-estimate camera orientation. Instead of solely using real keyframes from incoming video, the proposed approach employs virtual keyframes which are distributed strategically within completed portions of an environment map. To improve tracking speed, we introduce a new variant of our system which carries out relocalization only when tracking fails and uses inexpensive image-patch descriptors. We compare different system variants using three evaluation methods to show that the proposed system is useful in a practical sense. To improve relocalization robustness against lighting changes in indoor and outdoor environments, we propose a new approach based on illumination normalization and saturated area removal. We examine the performance of our solution over several indoor and outdoor video sequences, evaluating relocalization rates based on ground truth from a pan-tilt unit.
Environment map, virtual keyframe, vision-based tracking, camera pose relocalization, illumination changes.
Sehwan Kim, Christopher Coffin, Tobias Höllerer, "Robust Relocalization and Its Evaluation for Online Environment Map Construction", IEEE Transactions on Visualization & Computer Graphics, vol.17, no. 7, pp. 875-887, July 2011, doi:10.1109/TVCG.2010.243
[1] T. Höllerer, J. Wither, and S. DiVerdi, "Anywhere Augmentation: Towards Mobile Augmented Reality in Unprepared Environments," Location Based Services and TeleCartography, Series: Lecture Notes in Geoinformation and Cartography, G. Gartner, M. Peterson, and W. Cartwright, eds., pp. 393-416, Springer-Verlag, Feb. 2007.
[2] M. Uyttendaele, A. Criminisi, S.B. Kang, S. Winder, R. Szeliski, and R. Hartley, "Image-Based Interactive Exploration of Real-World Environments," IEEE Computer Graphics and Applications, vol. 24, no. 3, pp. 52-63, May/June 2004.
[3] K. Agusanto, L. Li, Z. Chuangui, and N.W. Sing, "Photorealistic Rendering for Augmented Reality Using Environment Illumination," Proc. Second IEEE/ACM Int'l Symp. Mixed and Augmented Reality (ISMAR '03), p. 208, 2003.
[4] T. Grosch, "PanoAR: Interactive Augmentation of Omni—Directional Images with Consistent Lighting," Proc. Computer Vision / Computer Graphics Collaboration Techniques and Applications (Mirage '05), pp. 25-34, 2005.
[5] S. DiVerdi, J. Wither, and T. Höllerer, "Envisor: Online Environment Map Construction for Mixed Reality," Proc. IEEE Int'l Conf. Virtual Reality, pp. 19-26, 2008.
[6] Thompson, 96repulsion, June 2009.
[7] S. Kim, C. Coffin, and T. Höllerer, "Relocalization Using Virtual Keyframes for Online Environment Map Construction," Proc. 16th ACM Symp. Virtual Reality Software and Technology (VRST '09), pp. 127-134, 2009.
[8] R. Fernando and M.J. Kilgard, The Cg Tutorial: The Definitive Guide to Programmable Real—Time Graphics. Addison—Wesley Longman Publishing Co. Inc., 2003.
[9] FLIR Motion Control Systems Inc., http:/, June 2009.
[10] M. Pupilli and A. Calway, "Real—Time Camera Tracking Using a Particle Filter," Proc. British Machine Vision Conf. (BMVC '05), pp. 519-528, 2005.
[11] S. Se, D.G. Lowe, and J.J. Little, "Vision—Based Global Localization and Mapping for Mobile Robots," IEEE Trans. Robotics, vol. 21, pp. 364-375, June 2005.
[12] D.G. Lowe, "Distinctive Image Features from Scale—Invariant Keypoints," Int'l J. Computer Vision (IJCV '04), vol. 60, pp. 91-110, 2004.
[13] M. Fischler and R. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. the ACM, vol. 24, no. 6, pp. 381-395, 1981.
[14] G. Reitmayr and T.W. Drummond, "Going Out: Robust Tracking for Outdoor Augmented Reality," Proc. Fifth IEEE and ACM Int'l Symp. Mixed and Augmented Reality (ISMAR '06), pp. 109-118, Oct. 2006.
[15] B. Williams, G. Klein, and I. Reid, "Real—Time SLAM Relocalisation," Proc. Int'l Conf. in Computer Vision (ICCV '07), pp. 1-8, 2007.
[16] G. Klein and D. Murray, "Parallel Tracking and Mapping for Small AR Workspaces," Proc. Sixth IEEE and ACM Int'l Symp. Mixed and Augmented Reality (ISMAR '07), Nov. 2007.
[17] G. Klein and D. Murray, "Improving the Agility of Keyframe—Based SLAM," Proc. European Conf. Computer Vision (ECCV '08), pp. 802-815, Oct. 2008.
[18] S. Benhimane and E. Malis, "Homography—Based 2D Visual Tracking and Serving," Special Joint Issue on Robotics and Vision J. Robotics Research, vol. 26, no. 7, pp. 661-676, July 2007.
[19] J.M.M. Montiel and A.J. Davison, "A Visual Compass Based on SLAM," Proc. IEEE Int'l Conf. Robotics and Automation, pp. 1917-1922, May 2006.
[20] D. Wagner, A. Mulloni, T. Langlotz, and D. Schmalstieg, "Real—Time Panoramic Mapping and Tracking on Mobile Phones," Proc. IEEE Int'l Conf. Virtual Reality, 2010.
[21] A. Irschara, C. Zach, J.-M. Frahm, and H. Bischof, "From Structure—from—Motion Point Clouds to Fast Location Recognition," Proc. IEEE Computer Soc. Conf. Computer Vision and Pattern Recognition , vol. 0, pp. 2599-2606, 2009.
[22] J. Pilet, V. Lepetit, and P. Fua, "Retexturing in the Presence of Complex Illumination and Occlusions," Proc. Sixth IEEE and ACM Int'l Symp. Mixed and Augmented Reality (ISMAR '07), pp. 1-8, 2007.
[23] J. Pilet, V. Lepetit, and P. Fua, "Augmenting Deformable Objects in Real—Time," Proc. Fourth IEEE/ACM Int'l Symp. Mixed and Augmented Reality (ISMAR '05), pp. 134-137, 2005.
[24] G.F. Silveira and E. Malis, "Real—Time Visual Tracking Under Arbitrary Illumination Changes," Proc. Conf. Computer Vision and Pattern Recognition (CVPR '07), 2007.
[25] J. Tian, J. Sun, and Y. Tang, "Tricolor Attenuation Model for Shadow Detection," Trans. Image Processing Archive, vol. 18, no. 10, pp. 2355-2363, 2009.
[26] S. DiVerdi, J. Wither, and T. Höllerer, "All Around the Map: Online Spherical Panorama Construction," Computers and Graphics, vol. 33, no. 1, pp. 835-846, 2009.
[27] S. DiVerdi, "Towards Anywhere Augmentation," PhD dissertation, pp. 835-846, 2007.
[28] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '94), pp. 593-600, 1994.
[29] B. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981.
[30] H. Bay, A. Ess, T. Tuytelaars, and L.V. Gool, "Surf: Speeded Up Robust Features," Computer Vision and Image Understanding (CVIU '08), vol. 110, no. 3, pp. 346-359, 2008.
[31] Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Trans. Pattern Analysis and Machine Intelligence (PAMI '00), vol. 22, no. 11, pp. 1330-1334, Nov. 2000.
[32] Intel Open Source Computer Vision Library, com/technology/computing opencv/, May 2007.
[33] InterSense, http:/, June 2009.
[34] R.C. Gonzalez and R.E. Woods, Digital Image Processing. Addison-Wesley, 1992.
[35] G. Reitmayr, personal comm., June 2010.
21 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool