Subscribe
Issue No.02 - February (2012 vol.34)
pp: 387-401
O. Kahler , Dept. of Eng. Sci., Univ. of Oxford, Oxford, UK
J. Denzler , Lehrstuhl fur Digitale Bildverarbeitung, Friedrich-Schiller-Univ. Jena, Jena, Germany
ABSTRACT
We present a novel approach to the structure-from-motion problem which combines the search for correspondences and geometric reconstruction, rather than treating these as separate steps. Through the combination of the two steps, we achieve an implicit feedback of 3D information to aid the correspondence search, and at the same time we avoid an explicit model for tracking errors. The reconstruction results are therefore optimal in case of, for example, Gaussian noise on image intensities. We also present an efficient online framework for structure-from-motion with our combined approach, thoroughly evaluate the method in experiments and compare the results to state-of-the-art methods.
INDEX TERMS
solid modelling, image reconstruction, optimisation, combined optimization approach, structure from motion problem, geometric reconstruction, 3D information, tracking errors, Gaussian noise, image intensities, online framework, Three dimensional displays, Image reconstruction, Tracking, Cameras, Mathematical model, Estimation, local minima., Structure-from-motion, direct reconstruction, constrained tracking
CITATION
O. Kahler, J. Denzler, "Tracking and Reconstruction in a Combined Optimization Approach", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 2, pp. 387-401, February 2012, doi:10.1109/TPAMI.2011.141
REFERENCES
 [1] M. Pollefeys, L.V. Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, "Visual Modeling with a Hand-Held Camera," Int'l J. Computer Vision, vol. 59, no. 3, pp. 207-232, Sept. 2004. [2] A.J. Davison, I.D. Reid, N.D. Molton, and O. Stasse, "Monoslam: Real-Time Single Camera Slam," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 6, pp. 1052-1067, June 2007. [3] A.N. Netravali and J. Salz, "Algorithms for Estimation of 3D Motion," AT&T Bell Laboratories Technical J., vol. 64, no. 2, pp. 335-346, Feb. 1985. [4] C. Mei, S. Benhimane, E. Malis, and P. Rives, "Constrained Multiple Planar Template Tracking for Central Catadioptric Cameras," Proc. British Machine Vision Conf., vol. 2, pp. 619-628, 2006. [5] G. Silveira, E. Malis, and P. Rives, "An Efficient Direct Approach to Visual Slam," IEEE Trans. Robotics, vol. 24, no. 5, pp. 969-979, Oct. 2008. [6] L. Zelnik-Manor and M. Irani, "Multiview Constraints on Homographies," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 2, pp. 214-223, Feb. 2002. [7] B. Triggs, P.F. McLauchlan, R.I. Hartley, and A.W. Fitzgibbon, "Bundle Adjustment—A Modern Synthesis," Proc. Int'l Workshop Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds., pp. 298-372, 2000. [8] C. Rother, S. Carlsson, and D. Tell, "Projective Factorization of Planes and Cameras in Multiple Views," Proc. Int'l Conf. Pattern Recognition, pp. 737-740, Aug. 2002. [9] O. Kähler and J. Denzler, "Implicit Feedback between Reconstruction and Tracking in a Combined Optimization Approach," Proc. 30th Ann. Symp. the German Assoc. for Pattern Recognition, G. Rigoll, ed., pp. 274-283, June 2008. [10] J. Shi and C. Tomasi, "Good Features to Track," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, June 1994. [11] G.D. Hager and P.N. Belhumeur, "Efficient Region Tracking with Parametric Models of Geometry and Illumination," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, Oct. 1998. [12] B. Triggs, "Plane + Parallax, Tensors and Factorization," Proc. European Conf. Computer Vision, pp. 522-538, 2000. [13] H. Jin, P. Favaro, and S. Soatto, "A Semi-Direct Approach to Structure from Motion," The Visual Computer, vol. 19, no. 6, pp. 377-394, Oct. 2003. [14] C. Engels, H. Stewenius, and D. Nister, "Bundle Adjustment Rules," Proc. Photogrammetric Computer Vision, pp. 266-271, Sept. 2006. [15] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2003. [16] Y. Ma, S. Soatto, J. Kosecka, and S.S. Sastry, An Invitation to 3D Vision, Interdisciplinary Applied Mathematics, first ed., vol. 26. Springer, 2004. [17] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981. [18] R. Koch, "Dynamic 3D Scene Analysis through Synthesis Feedback Control," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 556-568, June 1993. [19] M. Irani and P. Anandan, "About Direct Methods," Proc. Int'l Workshop Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds., pp. 267-277, Sept. 2000. [20] V. Lepetit and P. Fua, "Monocular Model-Based 3D Tracking of Rigid Objects: A Survey," Foundations and Trends in Computer Graphics and Vision, vol. 1, no. 1, pp. 1-89, Oct. 2005. [21] D. Cobzas and P. Sturm, "3D ssd Tracking with Estimated 3D Planes," Proc. Second Canadian Conf. Computer and Robot Vision, pp. 129-134, May 2005. [22] A. Ladikos, S. Benhimane, and N. Navab, "A Real-Time Tracking System Combining Template-Based and Feature-Based Approaches," Proc. Int'l Conf. Computer Vision Theory and Applications, vol. 2, pp. 325-332, Mar. 2007. [23] M. Habbecke and L. Kobbelt, "Iterative Multi-View Plane Fitting," Proc. Vision, Modeling, and Visualization Conf., pp. 73-80, Nov. 2006. [24] A. Chiuso, R. Brockett, and S. Soatto, "Optimal Structure from Motion: Local Ambiguities and Global Estimates," Int'l J. Computer Vision, vol. 39, no. 3, pp. 195-228, Sept. 2000. [25] T. Xiang and L.-F. Cheong, "Understanding the Behavior of SFM Algorithms: A Geometric Approach," Int'l J. Computer Vision, vol. 51, no. 2, pp. 111-137, Feb. 2003. [26] J. Oliensis, "The Least-Squares Error for Structure from Infinitesimal Motion," Int'l J. Computer Vision, vol. 61, no. 3, pp. 259-299, Feb. 2005. [27] G. Schweighofer and A. Pinz, "Robust Pose Estimation from a Planar Target," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2024-2030, Dec. 2006. [28] O. Kähler and J. Denzler, "Rigid Motion Constraints for Tracking Planar Objects," Proc. 29th Ann. Symp. German Assoc. for Pattern Recognition, F. Hamprecht, C. Schnörr, and B. Jähne, eds., pp. 102-111, Sept. 2007. [29] O. Kähler and J. Denzler, "Robust Real-Time SFM in a Combined Formulation of Tracking and Reconstruction," Proc. Vision, Modeling, and Visualization Conf., 2008. [30] A. Shashua and S. Avidan, "The Rank 4 Constraint in Multiple ($\ge 3$ ) View Geometry," Proc. Fourth European Conf. Computer Vision, B.F. Buxton and R. Cipolla, eds., pp. 196-206, Apr. 1996. [31] O. Kähler and J. Denzler, "Detecting Coplanar Feature Points in Handheld Image Sequences," Proc. Conf. Computer Vision Theory and Applications, vol. 2, pp. 447-452, Mar. 2007. [32] M.D. Wheeler and K. Ikeuchi, "Iterative Estimation of Rotation and Translation Using the Quaternions," Technical Report CMU-CS-95-215, Computer Science Dept., Carnegie Mellon Univ., Dec. 1995. [33] S. Negahdaripour, "Closed-Form Relationship between the Two Interpretations of a Moving Plane," J. Optical Soc. Am. A: Optics, Image Science, and Vision, vol. 7, no. 2, pp. 279-285, Feb. 1990. [34] S. Maybank, Theory of Reconstruction from Image Motion. Springer-Verlag, 1993. [35] C. Sminchisescu and B. Triggs, "Building Roadmaps of Minima and Transitions in Visual Models," Int'l J. Computer Vision, vol. 61, no. 1, pp. 81-101, Jan. 2005. [36] B.K. P. Horn, "Closed-form Solution of Absolute Orientation Using Unit Quaternions," J. Optical Soc. Am. A: Optics, Image Science, and Vision, vol. 4, no. 4, pp. 629-642, Apr. 1987. [37] B. Triggs, "Autocalibration from Planar Scenes," Proc. Fifth European Conf. Computer Vision, vol. 1, pp. 89-105, 1998. [38] V. Kolmogorov and R. Zabih, "Multi-Camera Scene Reconstruction via Graph Cuts," Proc. European Conf. Computer Vision, pp. 82-96, 2002.