This Article 
 Bibliographic References 
 Add to: 
On Pose Recovery for Generalized Visual Sensors
July 2004 (vol. 26 no. 7)
pp. 848-861

Abstract—With the advances in imaging technologies for robot or machine vision, new imaging devices are being developed for robot navigation or image-based rendering. However, to satisfy some design criterion, such as image resolution or viewing ranges, these devices are not necessarily being designed to follow the perspective rule and, thus, the imaging rays may not pass through a common point. Such generalized imaging devices may not be perspective and, therefore, their poses cannot be estimated with traditional techniques. In this paper, we propose a systematic method for pose estimation of such a generalized imaging device. We formulate it as a nonperspective n point (NPnP) problem. The case with exact solutions, n=3, is investigated comprehensively. Approximate solutions can be found for n>3 in a least-squared-error manner by combining an initial-pose-estimation procedure and an orthogonally iterative procedure. This proposed method can be applied not only to nonperspective imaging devices but also perspective ones. Results from experiments show that our approach can solve the NPnP problem accurately.

[1] K.S. Arun, T.S. Huang, and S.D. Blostein, Least-Squares Fitting of Two 3-D Point Sets IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 5, pp. 698-700, 1987.
[2] S. Baker and S.K. Nayar, A Theory of Catadioptric Image Formation Proc. IEEE Int'l Conf. Computer Vision, pp. 35-42, 1998.
[3] P.J. Besl and N.D. McKay, "A Method for Registration of 3D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[4] C.S. Chen and W.Y. Chang, Pose Estimation for Generalized Imaging Device via Solving Non-PerspectiveNPoint Problem Proc. IEEE Int'l Conf. Robotics and Automation, pp. 2931-2937, 2002.
[5] C.S. Chen, Y.P. Hung, and J.B. Cheng, “Ransac-Based Darces: A New Approach to Fast Automatic Registration of Partially Overlapping Range Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 11, pp. 1229-1234, Nov. 1999.
[6] C.S. Chen et al., Integrating Virtual Objects into Real Images for Augmented Reality Proc. ACM Virtual Reality Software and Technology Conf., pp. 1-8, 1998.
[7] D. Chetverikov, D. Svirko, D. Stepanov, and P. Krsek, The Trimmed Iterative Closest Point Algorithm Proc. IEEE Int'l Conf. Pattern Recognition, pp. 545-548, 2002.
[8] D.F. DeMenthon and L.S. Davis, Model-Based Object Pose in 25 Line of Code Int'l J. Computer Vision, vol. 15, pp. 123-141, 1995.
[9] M.A. Fischler and R.C. Bolls, Random Sample Consensus: A Paradigm for Model Fitting with Application to Image Analysis and Automated Cartography Comm. ACM, vol. 24, pp. 381-395, 1981.
[10] M.D. Grossberg and S.K. Nayar, A General Imaging Model and a Method for Finding Its Parameters Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 108-115, July 2001.
[11] F. Huang, S.K. Wei, and R. Klette, Geometrical Fundamentals of Polycentric Panoramas Proc. IEEE Int'l Conf. Computer Vision, vol. 1, pp 560-565, July 2001.
[12] X.S. Gao et al., Complete Solution Classification for the Perspective-Three-Point Problem IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 930-943, Aug. 2003.
[13] R. Gupta and R. Hartley, Linear Pushbroom Cameras IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 9, pp. 963-975, Sept. 1997.
[14] J. Huang, J.A. Stankovic, K. Ramamritham, D. Towsley, and B. Purimetla, “On Using Priority Inheritance in Real-Time Databases,” Special Issue of Real-Time Systems J., vol. 4. no. 3, Sept. 1992.
[15] R. Horaud, B. Canio, and O. Leboullenx, An Analytic Solution for the Perspective 4-Point Problem Computer Vision, Graphics, and Image Understanding, no. 1, pp. 33-44, 1989.
[16] B.K.P. Horn, Closed-Form Solution of Absolute Orientation Using Unit Quaternions J. Optical Soc. Am., A, vol. 4, pp. 629-642, 1987.
[17] B.K.P. Horn, H.M. Hilden, and S. Negahdaripour, Closed-form Solution of Absolute Orientation Using Orthonormal Matrices J. Optical Soc. Am., A, vol. 5, no. 7, pp. 1127-1135, 1988.
[18] M.A. Jenkins, Algorithm 493: Zeros of a Real Polynomial ACM Trans. Math. Software, vol. 1, p. 178, June 1975.
[19] M. Lang and B.-C. Frenzel, Polynomial Root Finding IEEE Signal Processing Letter, 1994.
[20] M. Levoy and P. Hanrahan, Light Field Rendering Proc. SIGGRAPH'96, pp. 31-42, Aug. 1996.
[21] A. Lorusso, D.W. Eggert, and R.B. Fisher, A Comparison of Four Algorithms for Estimating 3-D Rigid Transformations Proc. Sixth British Machine Vision Conf., pp. 237-246, 1995.
[22] D.G. Lowe, Robust Model-Based Motion Tracking through the Integration of Search and Estimation Int'l J. Computer Vision, vol. 8, no. 2, pp. 113-122, 1992.
[23] C.P. Lu, G.D. Hager, and E. Mjolsness, Fast and Globally Convergent Pose Estimation from Video Images IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 6, pp. 610-622, June 2000.
[24] K. Miyamoto, Fish Eyes Lens J. Optical Soc. of Am., vol. 54, pp. 1060-1061, 1994.
[25] W.H. Press et al., Numerical Recipes in C: The Art of Scientific Computing, second ed., chapter 3 and chapter 9, Cambridge Univ. Press, 1999.
[26] P. Rademacher and G. Bishop, Multiple-Center-of-Projection Images Proc. SIGGRAPH'98, pp. 199-206, July 1998.
[27] S. Rusinkiewicz and M. Levoy, Efficient Variant of the ICP Algorithm Proc. Third Int'l Conf. 3D Digital Imaging and Modeling, pp. 145-152, 2001.
[28] G.C. Sharp, S.W. Lee, and D.K. Wehe, ICP Registration Using Invariant Features IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 90-102, Jan. 2002.
[29] B.T. Smith et al., Matrix Eigensystem Routines EISPACK Guide, vol. 6, Lecture Notes in Computer Science, Springer, 1976.
[30] R. Swaminathan, M.D. Grossberg, and S.K. Nayar, Caustics of Catadioptric Cameras Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 2-9, July 2001.
[31] W.M. Walker, L. Shao, and R.A. Volz, Estimating 3-D Location Parameters Using Dual Number Quaternions CVGIP: Image Understanding, vol. 54, no. 4, pp. 358-367, 1991.
[32] P. Wunsch and G. Hirzinger, “Registration of CAD-Models to Images by Iterative Inverse Perspective Matching,” Proc. Int'al Conf. Pattern Recognition (ICPR), pp. 77-83, 1996.
[33] Y. Yagi, S. Kaato, and T. Tsuji, Collision Avoidance Using Omnidirectional Image Sensor (COPIS) Proc. IEEE Int'l Conf. Robotics and Automation, pp. 910-915, Apr. 1991.
[34] K. Yamazawa, Y. Yagi, and M. Yachida, Obstacle Detection with Omnidirectional Image Sensor: HyperOmni Vision Proc. IEEE Int'l Conf. Robotics and Automation, pp. 1062-1067, 1995.
[35] J.S.C. Yuan, “A General Phogrammetric Solution for the Determining Object Position and Orientation,” IEEE Trans. Robotics and Automation, vol. 5, no. 2, pp. 129–142, Apr. 1989.

Index Terms:
Computer vision, camera pose estimation, generalized imaging device (GID), perspective n point problem (PnP), nonperspective n point problem (NPnP).
Chu-Song Chen, Wen-Yan Chang, "On Pose Recovery for Generalized Visual Sensors," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 848-861, July 2004, doi:10.1109/TPAMI.2004.34
Usage of this product signifies your acceptance of the Terms of Use.