This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Pushing the Envelope of Modern Methods for Bundle Adjustment
Aug. 2012 (vol. 34 no. 8)
pp. 1605-1617
D. Steedly, Microsoft Corp., Redmond, WA, USA
D. Nister, Microsoft Corp., Redmond, WA, USA
Yekeun Jeong, Microsoft Corp., Redmond, WA, USA
R. Szeliski, Microsoft Corp., Redmond, WA, USA
In-So Kweon, Dept. of Electr. Eng., KAIST, Daejeon, South Korea
In this paper, we present results and experiments with several methods for bundle adjustment, producing the fastest bundle adjuster ever published in terms of computation and convergence. From a computational perspective, the fastest methods naturally handle the block-sparse pattern that arises in a reduced camera system. Adapting to the naturally arising block-sparsity allows the use of BLAS3, efficient memory handling, fast variable ordering, and customized sparse solving, all simultaneously. We present two methods; one uses exact minimum degree ordering and block-based LDL solving and the other uses block-based preconditioned conjugate gradients. Both methods are performed on the reduced camera system. We show experimentally that the adaptation to the natural block sparsity allows both of these methods to perform better than previous methods. Further improvements in convergence speed are achieved by the novel use of embedded point iterations. Embedded point iterations take place inside each camera update step, yielding a greater cost decrease from each camera update step and, consequently, a lower minimum. This is especially true for points projecting far out on the flatter region of the robustifier. Intensive analyses from various angles demonstrate the improved performance of the presented bundler.

[1] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, "Bundle Adjustment—A Modern Synthesis," Proc. Int'l Workshop Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds.,, pp. 298-372, 2000.
[2] M. Lourakis and A. Argyros, "The Design and Implementation of a Generic Sparse Bundle Adjustment Software Package Based on the Levenberg-Marquardt Algorithm," Technical Report 340, Inst. of Computer Science-FORTH, Heraklion, Crete, Greece, Aug. 2004.
[3] M. Lourakis and A. Argyros, "Is Levenberg-Marquardt the Most Efficient Optimization Algorithm for Implementing Bundle Adjustment?" Proc. IEEE Int'l Conf. Computer Vision, pp. 1526-1531, 2005.
[4] K. Ni, D. Steedly, and F. Dellaert, "Out-of-Core Bundle Adjustment for Large-Scale 3D Reconstruction," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[5] M. Byröd and K. Åström, "Bundle Adjustment Using Conjugate Gradients with Multiscale Preconditiong," Proc. British Machine Vision Conf., 2009.
[6] S. Agarwal, N. Snavely, I. Simon, S.M. Seitz, and R. Szeliski, "Building Rome in a Day," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[7] S. Agarwal, N. Snavely, S.M. Seitz, and R. Szeliski, "Bundle Adjustment in the Large," Proc. European Conf. Computer Vision, vol. 6312, pp. 29-42, 2010.
[8] M. Byröd and K. Åström, "Conjugate Gradient Bundle Adjustment," Proc. European Conf. Computer Vision, Part II, vol. 6312, pp. 114-127, 2010.
[9] K. Levenberg, "A Method for the Solution of Certain Non-Linear Problems in Least Squares," The Quarterly of Applied Math., vol. 2, pp. 164-168, 1944.
[10] H.-Y. Shum, Z. Zhang, and Q. Ke, "Efficient Bundle Adjustment with Virtual Key Frames: A Hierarchical Approach to Multi-Frame Structure from Motion," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 2538-2543, 1999.
[11] D. Steedly and I.A. Essa, "Propagation of Innovative Information in Non-Linear Least-Squares Structure from Moion," Proc. IEEE Int'l Conf. Computer Vision, pp. 223-229, 2001.
[12] D. Steedly, I.A. Essa, and F. Dellaert, "Spectral Partitioning for Structure from Motion," Proc. IEEE Int'l Conf. Computer Vision, pp. 996-1003, 2003.
[13] E. Mouragnon, M. Lhuillier, M. Dhome, F. Dekeyser, and P. Sayd, "Real Time Localization and 3D Reconstruction," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 363-370, 2006.
[14] C. Engels, H. Stewénius, and D. Nistér, "Bundle Adjustment Rules," Proc. Conf. Photogrammetric Computer Vision, Sept. 2006.
[15] G. Klein and D. Murray, "Parallel Tracking and Mapping for Small AR Workspaces," Proc. Int'l Symp. Mixed and Augmented Reality, 2007.
[16] S. Holmes, G. Sibely, G. Klein, and D.W. Murray, "A Relative Frame Representation for Fixed-Time Bundle Adjustment in SfM," Proc. IEEE Int'l Conf. Robotics and Automation, 2009.
[17] A. Eudes and M. Lhuillier, "Error Propagations for Local Bundle Adjustment," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2009.
[18] "Photosynth," http:/photosynth.net, 2012.
[19] Y. Jeong, D. Nistér, D. Steedly, R. Szeliski, and I.-S. Kweon, "Pushing the Envelope of Modern Methods for Bundle Adjustment," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2010.
[20] R.I. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[21] D.C. Brown, "The Bundle Adjustment—Progress and Prospects," Int'l Archives of Photogrammetry, vol. 21, no. 3, pp. 3-33, 1976.
[22] F. Dellaert and M. Kaess, "Square Root Sam: Simultaneous Location and Mapping via Square Root Information Smoothing," Int'l J. Robotics Research, vol. 25, pp. 1181-1203, 2006.
[23] G. Golub and C.V. Loan, Matrix Computations, Johns Hopkins Studies in Math. Sciences, third ed. JHU Press, 1996.
[24] N. Snavely, S. Seitz, and R. Szeliski, "Photo Tourism: Exploring Photo Collections in 3D," ACM Trans. Graphics, vol. 25, no. 3, pp. 835-846, 2006.
[25] T. Davis, Direct Methods for Sparse Linear Systems. SIAM, 2006.
[26] Y. Saad, Iterative Methods for Sparse Linear Systems, second ed. SIAM, 2000.
[27] M.J. Black and A. Rangarajan, "The Outlier Process: Unifying Line Processes and Robust Statistics," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 1994.

Index Terms:
iterative methods,cameras,conjugate gradient methods,image motion analysis,image reconstruction,computer vision,bundle adjustment,computational perspective,convergence persepctive,block-sparse pattern,reduced camera system,block-sparsity,BLAS3,memory handling,variable ordering,sparse solving,exact minimum degree ordering method,block-based LDL solving method,block-based preconditioned conjugate gradient method,embedded point iteration,camera update step,structure from motion,3D reconstruction,Cameras,Convergence,Sparse matrices,Jacobian matrices,Linear systems,Memory management,Barium,point iterations.,Computer vision,bundle adjustment,structure from motion,block-based,sparse linear solving
Citation:
D. Steedly, D. Nister, Yekeun Jeong, R. Szeliski, In-So Kweon, "Pushing the Envelope of Modern Methods for Bundle Adjustment," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 8, pp. 1605-1617, Aug. 2012, doi:10.1109/TPAMI.2011.256
Usage of this product signifies your acceptance of the Terms of Use.