CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2013 vol.35 Issue No.10 - Oct.

Subscribe

Issue No.10 - Oct. (2013 vol.35)

pp: 2357-2370

A. Censi , Control & Dynamical Syst. Dept., California Inst. of Technol., Pasadena, CA, USA

D. Scaramuzza , Dept. of Inf., Univ. of Zurich, Zurich, Switzerland

DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2013.34

ABSTRACT

This paper presents a new intrinsic calibration method that allows us to calibrate a generic single-view point camera just by waving it around. From the video sequence obtained while the camera undergoes random motion, we compute the pairwise time correlation of the luminance signal for a subset of the pixels. We show that if the camera undergoes a random uniform motion, then the pairwise correlation of any pixels pair is a function of the distance between the pixel directions on the visual sphere. This leads to formalizing calibration as a problem of metric embedding from nonmetric measurements: We want to find the disposition of pixels on the visual sphere from similarities that are an unknown function of the distances. This problem is a generalization of multidimensional scaling (MDS) that has so far resisted a comprehensive observability analysis (can we reconstruct a metrically accurate embedding?) and a solid generic solution (how do we do so?). We show that the observability depends both on the local geometric properties (curvature) as well as on the global topological properties (connectedness) of the target manifold. We show that, in contrast to the euclidean case, on the sphere we can recover the scale of the points distribution, therefore obtaining a metrically accurate solution from nonmetric measurements. We describe an algorithm that is robust across manifolds and can recover a metrically accurate solution when the metric information is observable. We demonstrate the performance of the algorithm for several cameras (pin-hole, fish-eye, omnidirectional), and we obtain results comparable to calibration using classical methods. Additional synthetic benchmarks show that the algorithm performs as theoretically predicted for all corner cases of the observability analysis.

INDEX TERMS

Cameras, Calibration, Robot vision systems, Correlation, Visualization,fish-eye cameras, Intrinsic camera calibration, metric embedding, catadioptric cameras, pin-hole cameras

CITATION

A. Censi, D. Scaramuzza, "Calibration by Correlation Using Metric Embedding from Nonmetric Similarities",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.35, no. 10, pp. 2357-2370, Oct. 2013, doi:10.1109/TPAMI.2013.34REFERENCES

- [1] T.A. Clarke and J.G. Fryer, "The Development of Camera Calibration Methods and Models,"
The Photogrammetric Record, vol. 16, no. 91, pp. 51-66, 1998, doi:10.1111/0031-868X.00113. - [2] P. Sturm, S. Ramalingam, J.-P. Tardif, S. Gasparini, and J. Barreto, "Camera Models and Fundamental Concepts Used in Geometric Computer Vision,"
Foundations and Trends in Computer Graphics and Vision, vol. 6, no. 1-2, pp. 1-183, 2011, doi:10.1561/0600000023. - [3] J.-Y. Bouguet,
The Matlab Calibration Toolbox, http://www.vision. caltech.edu/bouqueticalib_doc /, 2013.- [4] Z. Zhang, "A Flexible New Technique for Camera Calibration,"
IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, Nov. 2000, doi:10.1109/34.888718. - [5] D. Gennery, "Generalized Camera Calibration Including Fish-Eye Lenses,"
Int'l J. Computer Vision, vol. 68, no. 3, pp. 239-266, 2006, doi:10.1007/s11263-006-5168-1. - [6] D. Scaramuzza, A. Martinelli, and R. Siegwart, "A Flexible Technique for Accurate Omnidirectional Camera Calibration and Structure from Motion,"
Proc. IEEE Int'l Conf. Vision Systems, 2006, doi:10.1109/ICVS.2006.3. - [7] D. Scaramuzza, A. Martinelli, and R. Siegwart, "A Toolbox For Easy Calibrating Omnidirectional Cameras,"
Proc. Int'l Conf. Intelligent Robots and Systems, 2006, doi:10.1109/IROS.2006.282372. - [8] H. Li and R. Hartley, "Plane-Based Calibration and Auto-Calibration of a Fish-Eye Camera,"
Proc. Seventh Asian Conf. Computer Vision, P. Narayanan, S. Nayar, and H.-Y. Shum, eds., pp. 21-30, 2006, doi:10.1007/11612032. - [9] D. Scaramuzza and R. Siegwart, "A Practical Toolbox for Calibrating Omnidirectional Cameras,"
Vision Systems: Applications, G. Obinata and A. Duttain, eds., I-Tech Education and Publishing, 2007.- [10] C. Mei and P. Rives, "Single View Point Omnidirectional Camera Calibration From Planar Grids,"
Proc. Int'l Conf. Robotics and Automation, pp. 3945-3950, 2007, doi:10.1109/ROBOT.2007.364084. - [11] C. Mei,
Omnidirectional Camera Calibration Toolbox for MATLAB, http://homepages.laas.fr/~cmei/index.php Toolbox, 2013.- [12] J. Kannala and S.S. Brandt, "A Generic Camera Model and Calibration Method for Conventional, Wide-Angle and Fish-Eye Lenses,"
IEEE Trans Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1335-1340, Aug. 2006, doi:10.1109/TPAMI. 2006.153. - [13] J. Kannala,
Camera Calibration Toolbox for Generic Lenses For MATLAB, http://www.ee.oulu.fi/~jkannalacalibration , 2013.- [14] D. Scaramuzza,
Ocamcalib: Omnidirectional Camera Calibration Toolbox for Matlab, http://sites.google.com/site/scarabotixocamcalib-toolbox , 2013.- [15] L. Puig, J. Bermúdez, P. Sturm, and J.J. Guerrero, "Calibration of Omnidirectional Cameras in Practice: A Comparison of Methods,"
Computer Vision and Image Understanding, vol. 116, no. 1, pp. 120-137, 2012, doi:10.1016/j.cviu.2011.08.003. - [16] M. Rufli, D. Scaramuzza, and R. Siegwart, "Automatic Detection of Checkerboards on Blurred and Distorted Images,"
Proc. Int'l Conf. Intelligent Robots and Systems, pp. 3121-3126, 2008, doi:10.1109/IROS.2008.4650703. - [17] J. Barreto, J. Roquette, P. Sturm, and F. Fonseca, "Automatic Camera Calibration Applied to Medical Endoscopy,"
Proc. British Machine Vision Conf., 2009.- [18] J. Barreto,
Omnidirectional Camera Calibration Toolbox for MATLAB, http://www2.isr.uc.pt/~jpbar/CatPackmain.htm , 2013.- [19] F. Kahl and A. Heyden, "Robust Self-Calibration and Euclidean Reconstruction via Affine Approximation,"
Proc. 14th Int'l Conf. Pattern Recognition, pp. 56-58, 1998.- [20] M.K. Chandraker, S. Agarwal, F. Kahl, D. Nistér, and D.J. Kriegman, "Autocalibration via Rank-Constrained Estimation of the Absolute Quadric,"
Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007, doi:10.1109/CVPR.2007.383067. - [21] M. Grossberg and S. Nayar, "The Raxel Imaging Model and Ray-Based Calibration,"
Int'l J. Computer Vision, vol. 61, pp. 119-137, 2005, doi:10.1023/B:VISI.0000043754.56350.10. - [22] R.I. Hartley, "Self-Calibration from Multiple Views with a Rotating Camera,"
Proc. Third European Conf. Computer Vision, vol. 1, pp. 471-478, 1994, doi:10.1007/3-540-57956-7_52. - [23] F. Espuny and J.B. Gil, "Generic Self-Calibration of Central Cameras from Two 'Real' Rotational Flows,"
Proc. Eighth Workshop Omnidirectional Vision, Camera Networks and Non-Classical Cameras, 2008.- [24] S. Ramalingam, P. Sturm, and S.K. Lodha, "Generic Self-Calibration of Central Cameras,"
Computer Vision and Image Understanding, vol. 114, no. 2, pp. 210-219, 2010, doi:10.1016/j.cviu.2009.07.007. - [25] A. Martinelli, D. Scaramuzza, and R. Siegwart, "Automatic Self-Calibration of a Vision System during Robot Motion,"
Proc. Int'l Conf. Robotics and Automation, pp. 43-48, 2006, doi:10.1109/ROBOT.2006.1641159. - [26] G. Antonelli, F. Caccavale, F. Grossi, and A. Marino, "Simultaneous Calibration of Odometry and Camera for a Differential Drive Mobile Robot,"
Proc. Int'l Conf. Robotics and Automation, 2010, doi:10.1109/ROBOT.2010.5509954. - [27] E. Grossmann, J.A. Gaspar, and F. Orabona, "Discrete Camera Calibration from Pixel Streams,"
Computer Vision and Image Understanding, vol. 114, no. 2, pp. 198-209, 2010, doi:10.1016/j.cviu.2009.03.009. - [28] D. Pierce and B. Kuipers, "Map Learning with Uninterpreted Sensors and Effectors,"
Artificial Intelligence, vol. 92, no. 1/2, 1997, doi:10.1016/S0004-3702(96)00051-3. - [29] M. Boerlin, T. Delbruck, and K. Eng, "Getting to Know Your Neighbors: Unsupervised Learning of Topography from Real-World, Event-Based Input,"
Neural Computation, vol. 21, no. 1, pp. 216-238, 2009, doi:10.1162/neco.2009.06-07-554. - [30] J. Stober, L. Fishgold, and B. Kuipers, "Sensor Map Discovery for Developing Robots,"
Proc. AAAI Fall Symp. Manifold Learning and Its Applications, 2009.- [31] J. Modayil, "Discovering Sensor Space: Constructing Spatial Embeddings that Explain Sensor Correlations,"
Proc. Int'l Conf. Development and Learning, 2010, doi: 10.1109/DEVLRN.2010. 557885. - [32] J.W. Sammon, "A Nonlinear Mapping for Data Structure Analysis,"
IEEE Trans. Computers, vol. 18, no. 5, pp. 401-409, May 1969, doi:10.1109/T-C.1969.222678. - [33] R.C.T. Lee, J.R. Slagle, and H. Blum, "A Triangulation Method for the Sequential Mapping of Points from N-Space to Two-Space,"
IEEE Trans. Computers, vol. 26, no. 3, pp. 288-292, Mar. 1977, doi:10.1109/TC.1977.1674822. - [34] R. Shepard, "The Analysis of Proximities: Multidimensional Scaling with an Unknown Distance Function (Part I),"
Psychometrika, vol. 27, no. 3, pp. 125-140, 1962, doi:10.1007/BF02289630. - [35] R. Shepard, "The Analysis of Proximities: Multidimensional Scaling with an Unknown Distance Function (Part II),"
Psychometrika, vol. 27, no. 3, pp. 219-246, 1962, doi:10.1007/BF02289621. - [36] J.B. Kruskal, "Multidimensional Scaling by Optimizing Goodness of Fit to a Nonparametric Hypothesis,"
Psychometrika, vol. 29, no. 1, pp. 1-27, 1964, doi:10.1007/BF02289565. - [37] S.L. France and J.J. Carroll, "Two-Way Multidimensional Scaling: A Review,"
IEEE Trans. Systems, Man, and Cybernetics, Part C: Applications and Rev., vol. 41, no. 5, pp. 644-661, Sept. 2011, doi:10.1109/TSMCC.2010.2078502. - [38] T. Cox and M. Cox,
Multidimensional Scaling. Chapman & Hall/CRC, 2001.- [39] Y. Shang, W. Rumi, Y. Zhang, and M. Fromherz, "Localization from Connectivity in Sensor Networks,"
IEEE Trans. Parallel and Distributed Systems, vol. 15, no. 11, pp. 961-974, Nov. 2004, doi:10.1109/TPDS.2004.67. - [40] D.L. Ruderman and W. Bialek, "Statistics of Natural Images: Scaling in the Woods,"
Physical Rev. Letters, vol. 73, pp. 814-817, Aug. 1994, doi:10.1103/PhysRevLett.73.814. - [41] S. Agarwal, J. Wills, L. Cayton, G. Lanckriet, D. Kriegman, and S. Belongie, "Generalized Non-Metric Multidimensional Scaling,"
Proc. 11th Int'l Conf. Artificial Intelligence and Statistics, 2007.- [42] J.C. Gower and G.B. Dijksterhuis,
Procrustes Problems, vol. 30 of Oxford Statistical Science Series. Oxford Univ. Press, 2004.- [43] J.C. Platt, "FastMap, MetricMap, and Landmark MDS Are All Nystrom Algorithms,"
Proc. 10th Int'l Workshop Artificial Intelligence and Statistics, pp. 261-268, 2005. |