Subscribe
Issue No.09 - Sept. (2013 vol.35)
pp: 2238-2251
Yuchao Dai , Sch. of Electron. & Inf., Northwestern Polytech. Univ., Xi'an, China
Hongdong Li , NICTA, ANU, Canberra, ACT, Australia
Mingyi He , ShaanXi Key Lab. of Inf. Acquisition & Process., Northwestern Polytech. Univ., Xi'an, China
ABSTRACT
The Sturm-Triggs type iteration is a classic approach for solving the projective structure-from-motion (SfM) factorization problem, which iteratively solves the projective depths, scene structure, and camera motions in an alternated fashion. Like many other iterative algorithms, the Sturm-Triggs iteration suffers from common drawbacks, such as requiring a good initialization, the iteration may not converge or may only converge to a local minimum, and so on. In this paper, we formulate the projective SfM problem as a novel and original element-wise factorization (i.e., Hadamard factorization) problem, as opposed to the conventional matrix factorization. Thanks to this formulation, we are able to solve the projective depths, structure, and camera motions simultaneously by convex optimization. To address the scalability issue, we adopt a continuation-based algorithm. Our method is a global method, in the sense that it is guaranteed to obtain a globally optimal solution up to relaxation gap. Another advantage is that our method can handle challenging real-world situations such as missing data and outliers quite easily, and all in a natural and unified manner. Extensive experiments on both synthetic and real images show comparable results compared with the state-of-the-art methods.
INDEX TERMS
Cameras, Minimization, Iterative methods, Matrix decomposition, Educational institutions, Indexes, Image reconstruction,outlier, Element-wise factorization, projective structure and motion, semidefinite programming, missing data
CITATION
Yuchao Dai, Hongdong Li, Mingyi He, "Projective Multiview Structure and Motion from Element-Wise Factorization", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 9, pp. 2238-2251, Sept. 2013, doi:10.1109/TPAMI.2013.20
REFERENCES
 [1] C. Tomasi and T. Kanade, "Shape and Motion from Image Streams under Orthography: A Factorization Method," Int'l J. Computer Vision, vol. 9, no. 2, pp. 137-154, 1992. [2] P. Sturm and B. Triggs, "A Factorization Based Algorithm for Multi-Image Projective Structure and Motion," Proc. European Conf. Computer Vision, pp. 709-720, 1996. [3] B. Triggs, "Factorization Methods for Projective Structure and Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 845-851, 1996. [4] T. Ueshiba and F. Tomita, "A Factorization Method for Projective and Euclidean Reconstruction from Multiple Perspective Views via Iterative Depth Estimation," Proc. European Conf. Computer Vision, pp. 296-310, 1998. [5] A. Heyden, R. Berthilsson, and G. Sparr, "An Iterative Factorization Method for Projective Structure and Motion from Image Sequences," Image Vision Computing, vol. 17, no. 13, pp. 981-991, 1999. [6] Q. Chen and G. Medioni, "Efficient Iterative Solution to M-View Projective Reconstruction Problem," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 55-61, 1999. [7] M. Han and T. Kanade, "Perspective Factorization Methods for Euclidean Reconstruction," Technical Report CMU-RI-TR-99-22, Robotics Inst., Carnegie Mellon Univ., Aug. 1999. [8] S. Mahamud and M. Hebert, "Iterative Projective Reconstruction from Multiple Views," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 430-437, 2000. [9] J. Oliensis and R. Hartley, "Iterative Extensions of the Sturm/Triggs Algorithm: Convergence and Nonconvergence," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2217-2233, Dec. 2007. [10] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004. [11] M.A. Fischler and R.C. Bolles, "Random Sample Consensus: A Paradigm for Model Fitting with Applications to Image Analysis and Automated Cartography," Comm. ACM, vol. 24, no. 6, pp. 381-395, June 1981. [12] Y. Dai, H. Li, and M. He, "Element-Wise Factorization for n-View Projective Reconstruction," Proc. European Conf. Computer Vision, pp. 396-409, 2010. [13] B. Triggs, P.F. McLauchlan, R.I. Hartley, and A.W. Fitzgibbon, "Bundle Adjustment—A Modern Synthesis," Proc. Int'l Workshop Vision Algorithms, pp. 298-372, 2000. [14] N. Snavely, S.M. Seitz, and R. Szeliski, "Modeling the World from Internet Photo Collections," Int'l J. Computer Vision, vol. 80, no. 2, pp. 189-210, 2008. [15] S. Agarwal, N. Snavely, I. Simon, S. Seitz, and R. Szeliski, "Building Rome in a Day," Proc. IEEE Int'l Conf. Computer Vision, pp. 72-79, Oct. 2009. [16] J.-M. Frahm, P. Fite-Georgel, D. Gallup, T. Johnson, R. Raguram, C. Wu, Y.-H. Jen, E. Dunn, B. Clipp, S. Lazebnik, and M. Pollefeys, "Building Rome on a Cloudless Day," Proc. European Conf. Computer Vision, pp. 368-381, 2010. [17] D. Crandall, A. Owens, N. Snavely, and D. Huttenlocher, "Discrete-Continuous Optimization for Large-Scale Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 3001-3008, June 2011. [18] C. Olsson and O. Enqvist, "Stable Structure from Motion for Unordered Image Collections," Proc. 17th Scandinavian Conf. Image Analysis, pp. 524-535, 2011. [19] T. Ding, M. Sznaier, and O.I. Camps, "A Rank Minimization Approach to Video Inpainting," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007. [20] Y. Peng, A. Ganesh, J. Wright, W. Xu, and Y. Ma, "RASL: Robust Alignment by Sparse and Low-Rank Decomposition for Linearly Correlated Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 763-770, 2010. [21] M. Ayazoglu, M. Sznaier, and O. Camps, "Euclidean Structure Recovery from Motion in Perspective Image Sequences via Hankel Rank Minimization," Proc. European Conf. Computer Vision, pp. 71-84, 2010. [22] B. Recht, M. Fazel, and P.A. Parrilo, "Guaranteed Minimum-Rank Solutions of Linear Matrix Equations via Nuclear Norm Minimization," SIAM Rev., vol. 52, no. 3, pp. 471-501, 2010. [23] M. Fazel, H. Hindi, and S. Boyd, "A Rank Minimization Heuristic with Application to Minimum Order System Approximation," Proc. Am. Control Conf., pp. 4734-4739, 2001. [24] M. Fazel, "Matrix Rank Minimization with Applications," PhD dissertation, Stanford Univ., 2002. [25] R. Angst, C. Zach, and M. Pollefeys, "The Generalized Trace-Norm and Its Application to Structure-from-Motion Problems," Proc. IEEE Int'l Conf. Computer Vision, pp. 2502-2509, 2011. [26] J. Sturm, "Using SeDuMi 1.02, a MATLAB Toolbox for Optimization over Symmetric Cones," Optimization Methods and Software, vol. 11, pp. 625-653, 1999. [27] K. Toh, M. Todd, and R. Tutuncu, "SDPT3—A Matlab Software Package for Semidefinite Programming," Optimization Methods and Software, vol. 11, pp. 545-581, 1999. [28] J.-F. Cai, E.J. Candès, and Z. Shen, "A Singular Value Thresholding Algorithm for Matrix Completion," SIAM J. Optimization, vol. 20, no. 4, pp. 1956-1982, 2010. [29] S. Ma, D. Goldfarb, and L. Chen, "Fixed Point and Bregman Iterative Methods for Matrix Rank Minimization," Math. Programming, Series A, vol. 128, nos. 1/2, pp. 321-353, 2011. [30] E. Candès, X. Li, Y. Ma, and J. Wright, "Robust Principal Component Analysis?" J. ACM, vol. 58, no. 3, pp. 11:1-37, 2011. [31] Z. Lin, M. Chen, and Y. Ma, "The Augmented Lagrange Multiplier Method for Exact Recovery of Corrupted Low-Rank Matrices," ArXiv e-prints, Sept. 2010. [32] R. Liu, Z. Lin, F. De la Torre, and Z. Su, "Fixed-Rank Representation for Unsupervised Visual Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 598-605, 2012. [33] S. Burer and R.D. Monteiro, "A Nonlinear Programming Algorithm for Solving Semidefinite Programs via Low-Rank Factorization," Math. Programming, vol. 95, pp. 329-357, 2003. [34] K. Mitra, S. Sheorey, and R. Chellappa, "Large-Scale Matrix Factorization with Missing Data under Additional Constraints," Proc. Advances in Neural Information Processing System, vol. 23, pp. 1642-1650, 2010. [35] R. Hartley and F. Schaffalitzky, "PowerFactorization: 3D Reconstruction with Missing or Uncertain Data," Proc. Japan-Australia Workshop Computer Vision, pp. 1-9, 2004. [36] J. Haldar and D. Hernando, "Rank-Constrained Solutions to Linear Matrix Equations Using Powerfactorization," IEEE Signal Processing Letters, vol. 16, no. 7, pp. 584-587, July 2009. [37] A.M. Buchanan and A.W. Fitzgibbon, "Damped Newton Algorithms for Matrix Factorization with Missing Data," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 316-322, 2005. [38] A. Del Bue, J. Xavier, L. Agapito, and M. Paladini, "Bilinear Modeling via Augmented Lagrange Multipliers (BALM)," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 8, pp. 1496-1508, Aug. 2012. [39] C. Olsson and M. Oskarsson, "A Convex Approach to Low Rank Matrix Approximation with Missing Data," Proc. Scandinavian Conf. Image Analysis, pp. 301-309, 2009. [40] S. Mahamud, M. Hebert, Y. Omori, and J. Ponce, "Provably-Convergent Iterative Methods for Projective Structure from Motion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1018-1025, 2001. [41] D. Martinec and T. Pajdla, "Structure from Many Perspective Images with Occlusions," Proc. European Conf. Computer Vision, pp. 355-369, 2002. [42] H. Jia and A.M. Martinez, "Low-Rank Matrix Fitting Based on Subspace Perturbation Analysis with Applications to Structure from Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 841-854, May 2009. [43] D. Martinec and T. Pajdla, "3D Reconstruction by Fitting Low-Rank Matrices with Missing Data," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 198-205, 2005. [44] Q. Ke and T. Kanade, "Robust ${L}_1$ Norm Factorization in the Presence of Outliers and Missing Data by Alternative Convex Programming," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 739-746, 2005. [45] A. Eriksson and A. van den Hengel, "Efficient Computation of Robust Low-Rank Matrix Approximations in the Presence of Missing Data Using the ${L}_1$ Norm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 771-778, 2010. [46] V. Chandrasekaran, S. Sanghavi, P. Parrilo, and A. Willsky, "Rank-Sparsity Incoherence for Matrix Decomposition," SIAM J. Optimization, vol. 21, no. 2, pp. 572-596, 2011. [47] M. Tao and X. Yuan, "Recovering Low-Rank and Sparse Components of Matrices from Incomplete and Noisy Observations," SIAM J. Optimization, vol. 21, no. 1, pp. 57-81, 2011. [48] R. Hartley, "In Defense of the Eight-Point Algorithm," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 6, pp. 580-593, June 1997. [49] Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, Nov. 2000.