The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2013 vol.35)
pp: 2498-2512
Yu-Wing Tai , Dept. of Electr. Eng., Korea Adv. Inst. of Sci. & Technol., Daejeon, South Korea
Xiaogang Chen , Inst. of Image Process. & Pattern Recognition, Shanghai Jiaotong Univ., Shanghai, China
Sunyeong Kim , Dept. of Electr. Eng., Korea Adv. Inst. of Sci. & Technol., Daejeon, South Korea
Seon Joo Kim , Dept. of Comput. Sci., Yonsei Univ., Seoul, South Korea
Feng Li , Mitsubishi Electr. Res. Labs., Cambridge, MA, USA
Jie Yang , Inst. of Image Process. & Pattern Recognition, Shanghai Jiaotong Univ., Shanghai, China
Jingyi Yu , Dept. of CIS, Univ. of Delaware, Newark, DE, USA
Y. Matsushita , Microsoft Res. Asia, Beijing, China
M. S. Brown , Sch. of Comput., Nat. Univ. of Singapore, Singapore, Singapore
ABSTRACT
This paper investigates the role that nonlinear camera response functions (CRFs) have on image deblurring. We present a comprehensive study to analyze the effects of CRFs on motion deblurring. In particular, we show how nonlinear CRFs can cause a spatially invariant blur to behave as a spatially varying blur. We prove that such nonlinearity can cause large errors around edges when directly applying deconvolution to a motion blurred image without CRF correction. These errors are inevitable even with a known point spread function (PSF) and with state-of-the-art regularization-based deconvolution algorithms. In addition, we show how CRFs can adversely affect PSF estimation algorithms in the case of blind deconvolution. To help counter these effects, we introduce two methods to estimate the CRF directly from one or more blurred images when the PSF is known or unknown. Our experimental results on synthetic and real images validate our analysis and demonstrate the robustness and accuracy of our approaches.
INDEX TERMS
Image edge detection, Kernel, Deconvolution, Cameras, Image restoration, Estimation, Shape,CRF estimation, Nonlinear camera response functions (CRFs), motion deblurring
CITATION
Yu-Wing Tai, Xiaogang Chen, Sunyeong Kim, Seon Joo Kim, Feng Li, Jie Yang, Jingyi Yu, Y. Matsushita, M. S. Brown, "Nonlinear Camera Response Functions and Image Deblurring: Theoretical Analysis and Practice", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 10, pp. 2498-2512, Oct. 2013, doi:10.1109/TPAMI.2013.40
REFERENCES
[1] M. Ben-Ezra and S. Nayar, "Motion-Based Motion Deblurring," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 689-698, June 2004.
[2] J. Cai, H. Ji, C. Liu, and Z. Shen, "Blind Motion Deblurring from a Single Image Using Sparse Approximation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[3] A. Chakrabarti, D. Scharstein, and T. Zickler, "An Empirical Camera Model for Internet Color Vision," Proc. British Machine Vision Conf., 2009.
[4] Y. Chang and J. Reid, "RGB Calibration for Color Image-Analysis in Machine Vision," IEEE Trans. Image Processing, vol. 5, no. 10, pp. 1414-1422, Oct. 1996.
[5] J. Chen and C.K. Tang, "Robust Dual Motion Deblurring," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[6] X. Chen, X. He, J. Yang, and Q. Wu, "An Effective Document Image Deblurring Algorithm," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[7] X. Chen, F. Li, J. Yang, and J. Yu, "A Theoretical Analysis of Camera Response Functions in Image Deblurring," Proc. 12th European Conf. Computer Vision, 2012.
[8] S. Cho and S. Lee, "Fast Motion Deblurring," Proc. ACM Siggraph ASIA, 2009.
[9] S. Cho, Y. Matsushita, and S. Lee, "Removing Non-Uniform Motion Blur from Images," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[10] S. Cho, J. Wang, and S. Lee, "Handling Outliers in Non-Blind Image Deconvolution," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[11] T.-S. Cho, S. Paris, B. Freeman, and B. Horn, "Blur Kernel Estimation Using the Radon Transform," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[12] P. Debevec and J. Malik, "Recovering High Dynamic Range Radiance Maps from Photographs," Proc. ACM Siggraph, 1997.
[13] H. Farid, "Blind Inverse Gamma Correction," IEEE Trans. Image Processing, vol. 10, no. 10, pp. 1428-1433, Oct. 2001.
[14] R. Fergus, B. Singh, A. Hertzmann, S.T. Roweis, and W.T. Freeman, "Removing Camera Shake from a Single Photograph," ACM Trans. Graphics, vol. 25, no. 3, pp. 787-794, 2006.
[15] M. Grossberg and S. Nayar, "Modeling the Space of Camera Response Functions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1272-1282, Oct. 2004.
[16] M.D. Grossberg and S.K. Nayar, "What Is the Space of Camera Response Functions?" Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[17] M. Hirsch, C. Schuler, S. Harmeling, and B. Schölkopf, "Fast Removal of Non-Uniform Camera Shake," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[18] J. Jia, "Single Image Motion Deblurring Using Transparency," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[19] N. Joshi, S. Kang, L. Zitnick, and R. Szeliski, "Image Deblurring with Inertial Measurement Sensors," ACM Trans. Graphics, vol. 29, no. 3, 2010.
[20] N. Joshi, R. Szeliski, and D. Kriegman, "PSF Estimation Using Sharp Edge Prediction," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[21] S. Kim, J. Frahm, and M. Pollefeys, "Radiometric Calibration with Illumination Change for Outdoor Scene Analysis," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[22] S. Kim, Y.-W. Tai, S. Kim, M.S. Brown, and Y. Matsushita, "Nonlinear Camera Response Functions and Image Deblurring," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
[23] S.J. Kim, H.T. Lin, Z. Lu, S. Susstrunk, S. Lin, and M.S. Brown, "A New In-Camera Imaging Model for Color Computer Vision and Its Application," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 12, pp. 2289-2302, Dec. 2012.
[24] S.J. Kim and M. Pollefeys, "Robust Radiometric Calibration and Vignetting Correction," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 562-576, Apr. 2008.
[25] D. Krishnan and R. Fergus, "Fast Image Deconvolution Using Hyper-Laplacian Priors," Proc. Conf. Neural Information Processing Systems, 2009.
[26] D. Krishnan, T. Tay, and R. Fergus, "Blind Deconvolution Using a Normalized Sparsity Measure," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[27] J. Lagarias, J. Reeds, M. Wright, and P. Wright, "Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions," SIAM J. Optimization, vol. 9, no. 1, pp. 112-147, 1998.
[28] J.-Y. Lee, B. Shi, Y. Matsushita, I. Kweon, and K. Ikeuchi, "Radiometric Calibration by Transform Invariant Low-Rank Structure," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[29] A. Levin, "Blind Motion Deblurring Using Image Statistics," Proc. Conf. Neural Information Processing Systems, 2006.
[30] A. Levin, R. Fergus, F. Durand, and W.T. Freeman, "Image and Depth from a Conventional Camera with a Coded Aperture," ACM Trans. Graphics, vol. 26, no. 3, 2007.
[31] H. Lin, S.J. Kim, S. Susstrunk, and M.S. Brown, "Revisiting Radiometric Calibration for Color Computer Vision," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[32] S. Lin, J. Gu, S. Yamazaki, and H.-Y. Shum, "Radiometric Calibration from a Single Image," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2004.
[33] S. Lin and L. Zhang, "Determining the Radiometric Response Function from a Single Grayscale Image," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[34] P.-Y. Lu, T.-H. Huang, M.-S. Wu, Y.-T. Cheng, and Y.-Y. Chuang, "High Dynamic Range Image Reconstruction from Hand-Held Cameras," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[35] L. Lucy, "An Iterative Technique for the Rectification of Observed Distributions," Astronomical J., vol. 79, p. 754, 1974.
[36] Y. Matsushita and S. Lin, "Radiometric Calibration from Noise Distributions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[37] T. Mitsunaga and S. Nayer, "Radiometric Self Calibration," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1999.
[38] T.-T. Ng, S.-F. Chang, and M.-P. Tsui, "Using Geometry Invariants for Camera Response Function Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[39] A. Rav-Acha and S. Peleg, "Two Motion Blurred Images Are Better Than One," Pattern Recognition Letters, vol. 26, pp. 311-317, 2005.
[40] W. Richardson, "Bayesian-Based Iterative Method of Image Restoration," J. Optical Soc. Am., vol. 62, no. 1, 1972.
[41] Q. Shan, J. Jia, and A. Agarwala, "High-Quality Motion Deblurring from a Single Image," ACM Trans. Graphics, vol. 27, 2008.
[42] Y.-W. Tai, H. Du, M. Brown, and S. Lin, "Image/Video Deblurring Using a Hybrid Camera," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[43] Y.-W. Tai, H. Du, M.S. Brown, and S. Lin, "Correction of Spatially Varying Image and Video Motion Blur Using a Hybrid Camera," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 6, pp. 1012-1028, June 2010.
[44] Y.-W. Tai, N. Kong, S. Lin, and S. Shin, "Coded Exposure Imaging for Projective Motion Deblurring," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[45] Y.-W. Tai and S. Lin, "Motion-Aware Noise Filtering for Deblurring of Noisy and Blurry Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2012.
[46] Y.-W. Tai, P. Tan, and M. Brown, "Richardson-Lucy Deblurring for Scenes under Projective Motion Path," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 8, pp. 1603-1618, Aug. 2011.
[47] O. Whyte, J. Sivic, A. Zisserman, and J. Ponce, "Non-Uniform Deblurring for Shaken Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[48] N. Wiener, Extrapolation Interpolation, and Smoothing of Stationary Time Series. Wiley, 1949.
[49] B. Wilburn, H. Xu, and Y. Matsushita, "Radiometric Calibration Using Temporal Irradiance Mixtures," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[50] L. Xu and J. Jia, "Two-Phase Kernel Estimation for Robust Motion Deblurring," Proc. 11th European Conf. Computer Vision, 2010.
[51] L. Yuan, J. Sun, L. Quan, and H. Shum, "Image Deblurring with Blurred/Noisy Image Pairs," ACM Trans. Graphics, vol. 26, no. 3, 2007.
47 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool