Issue No.08 - Aug. (2013 vol.35)
pp: 1994-2007
Ziyan Wu , Rensselaer Polytechnic Institute, Troy
Richard J. Radke , Rensselaer Polytechnic Institute, Troy
Pan-tilt-zoom (PTZ) cameras are pervasive in modern surveillance systems. However, we demonstrate that the (pan, tilt) coordinates reported by PTZ cameras become inaccurate after many hours of operation, endangering tracking and 3D localization algorithms that rely on the accuracy of such values. To solve this problem, we propose a complete model for a PTZ camera that explicitly reflects how focal length and lens distortion vary as a function of zoom scale. We show how the parameters of this model can be quickly and accurately estimated using a series of simple initialization steps followed by a nonlinear optimization. Our method requires only 10 images to achieve accurate calibration results. Next, we show how the calibration parameters can be maintained using a one-shot dynamic correction process; this ensures that the camera returns the same field of view every time the user requests a given (pan, tilt, zoom), even after hundreds of hours of operation. The dynamic calibration algorithm is based on matching the current image against a stored feature library created at the time the PTZ camera is mounted. We evaluate the calibration and dynamic correction algorithms on both experimental and real-world datasets, demonstrating the effectiveness of the techniques.
Cameras, Calibration, Lenses, Computational modeling, Heuristic algorithms, Surveillance, dynamic correction, Pan-tilt-zoom, calibration
Ziyan Wu, Richard J. Radke, "Keeping a Pan-Tilt-Zoom Camera Calibrated", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 8, pp. 1994-2007, Aug. 2013, doi:10.1109/TPAMI.2012.250
[1] N. Ashraf and H. Foroosh, "Robust Auto-Calibration of a PTZ Camera with Non-Overlapping FOV," Proc. Int'l Conf. Pattern Recognition, Dec. 2008.
[2] H. Bay, A. Ess, T. Tuytelaars, and L. Van Gool, "Speeded-Up Robust Features (SURF)," Computer Vision and Image Understanding, vol. 110, no. 3, pp. 346-359, 2008.
[3] R.T. Collins and Y. Tsin, "Calibration of an Outdoor Active Camera System," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[4] J. Davis and X. Chen, "Calibrating Pan-Tilt Cameras in Wide-Area Surveillance Networks," Proc. IEEE Int'l Conf. Computer Vision, 2003.
[5] L. de Agapito, R. Hartley, and E. Hayman, "Linear Calibration of a Rotating and Zooming Camera," Proc. Int'l Conf. Computer Vision and Pattern Recognition, 1999.
[6] L. de Agapito, E. Hayman, and I. Reid, "Self-Calibration of Rotating and Zooming Cameras," Int'l J. Computer Vision, vol. 45, no. 2, pp. 107-127, Nov. 2001.
[7] A. Fitzgibbon, "Simultaneous Linear Estimation of Multiple View Geometry and Lens Distortion," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2001.
[8] R. Hartley and A. Zisserman, Multiple View Geometry in Computer Vision, second ed. Cambridge Univ. Press, 2004.
[9] R.I. Hartley, "Self-Calibration of Stationary Cameras," Int'l J. Computer Vision, vol. 22, no. 1, pp. 5-23, Feb. 1997.
[10] E. Hayman and D. Murray, "The Effects of Translational Misalignment When Self-Calibrating Rotating and Zooming Cameras," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 1015-1020, Aug. 2003.
[11] B. He and Y. Li, "Camera Calibration with Lens Distortion and from Vanishing Points," Optical Eng., vol. 48, no. 1, p. 013603, Jan. 2009.
[12] Z. Kukelova and T. Pajdla, "A Minimal Solution to Radial Distortion Autocalibration." IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 12, pp. 2410-2422, Dec. 2011.
[13] R. Lenz and R. Tsai, "Techniques for Calibration of the Scale Factor and Image Center for High Accuracy 3D Machine Vision Metrology," Proc. IEEE Int'l Conf. Robotics and Automation, 1987.
[14] M. Li and J.-M. Lavest, "Some Aspects of Zoom Lens Camera Calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 11, pp. 1105-1110, Nov. 1996.
[15] S.-N. Lim, A. Elgammal, and L. Davis, "Image-Based Pan-Tilt Camera Control in a Multi-Camera Surveillance Environment," Proc. Int'l Conf. Multimedia and Expo, 2003.
[16] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, Nov. 2004.
[17] J. Luo and O. Gwun, "A Comparison of SIFT, PCA-SIFT and SURF," Int'l J. Image Processing, vol. 3, no. 4, pp. 143-152, Oct. 2009.
[18] R. Nevatia, "Camera Calibration from Video of a Walking Human," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 9, pp. 1513-1518, Sept. 2006.
[19] E. Rosten and R. Loveland, "Camera Distortion Self-Calibration Using the Plumb-Line Constraint and Minimal Hough Entropy," Machine Vision and Applications, vol. 22, no. 1, pp. 77-85, Apr. 2009.
[20] M. Sarkis, C. Senft, and K. Diepold, "Calibrating an Automatic Zoom Camera with Moving Least Squares," IEEE Trans. Automation Science and Eng., vol. 6, no. 3, pp. 492-503, July 2009.
[21] T. Schoepflin and D. Dailey, "Dynamic Camera Calibration of Roadside Traffic Management Cameras for Vehicle Speed Estimation," IEEE Trans. Intelligent Transportation Systems, vol. 4, no. 2, pp. 90-98, June 2003.
[22] S.N. Sinha and M. Pollefeys, "Pan-Tilt-Zoom Camera Calibration and High-Resolution Mosaic Generation," Computer Vision and Image Understanding, vol. 103, no. 3, pp. 170-183, Sept. 2006.
[23] K.-T. Song and J.-C. Tai, "Dynamic Calibration of Pan-Tilt-Zoom Cameras for Traffic Monitoring," IEEE Trans. Systems, Man, and Cybernetics. Part B, Cybernetics, vol. 36, no. 5, pp. 1091-1103, Oct. 2006.
[24] R. Steele, C. Jaynes, A. Leonardis, H. Bischof, and A. Pinz, "Overconstrained Linear Estimation of Radial Distortion and Multi-View Geometry," Proc. European Conf. Computer Vision, 2006.
[25] B. Tordoff and D. Murray, "The Impact of Radial Distortion on the Self-Calibration of Rotating Cameras," Computer Vision and Image Understanding, vol. 96, no. 1, pp. 17-34, Oct. 2004.
[26] Z. Wu and R.J. Radke, "Using Scene Features to Improve Wide-Area Video Surveillance," Proc. IEEE Workshop Camera Networks and Wide Area Scene Analysis, 2012.
[27] Z. Zhang, "A Flexible New Technique for Camera Calibration," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, Nov. 2000.