The Community for Technology Leaders
RSS Icon
Issue No.01 - Jan. (2013 vol.35)
pp: 157-170
Xiang Zhu , Dept. of Electr. Eng., Univ. of California, Santa Cruz, Santa Cruz, CA, USA
P. Milanfar , Dept. of Electr. Eng., Univ. of California, Santa Cruz, Santa Cruz, CA, USA
To correct geometric distortion and reduce space and time-varying blur, a new approach is proposed in this paper capable of restoring a single high-quality image from a given image sequence distorted by atmospheric turbulence. This approach reduces the space and time-varying deblurring problem to a shift invariant one. It first registers each frame to suppress geometric deformation through B-spline-based nonrigid registration. Next, a temporal regression process is carried out to produce an image from the registered frames, which can be viewed as being convolved with a space invariant near-diffraction-limited blur. Finally, a blind deconvolution algorithm is implemented to deblur the fused image, generating a final output. Experiments using real data illustrate that this approach can effectively alleviate blur and distortions, recover details of the scene, and significantly improve visual quality.
Image restoration, Vectors, Imaging, Deconvolution, Noise, Kernel, Estimation,sharpness metric, Image restoration, atmospheric turbulence, nonrigid image registration, point spread function
Xiang Zhu, P. Milanfar, "Removing Atmospheric Turbulence via Space-Invariant Deconvolution", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 1, pp. 157-170, Jan. 2013, doi:10.1109/TPAMI.2012.82
[1] W.E.K. Middleton, Vision through the Atmosphere. Univ. of Toronto Press, 1958.
[2] M.C. Roggemann and B.M. Welsh, Imaging through Turbulence. CRC Press, 1996.
[3] D. Li, R.M. Mersereau, and S. Simske, "Atmospheric Turbulence-Degraded Image Restoration Using Principal Components Analysis," IEEE Geoscience and Remote Sensing Letters, vol. 4, no. 3, pp. 340-344, July 2007.
[4] M. Shimizu, S. Yoshimura, M. Tanaka, and M. Okutomi, "Super-Resolution from Image Sequence under Influence of Hot-Air Optical Turbulence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[5] R.N. Tubbs, "Lucky Exposures: Diffraction Limited Astronomical Imaging through the Atmosphere," PhD thesis, Cambridge Univ., Sept. 2003.
[6] R.K. Tyson, Principles of Adaptive Optics. Academic Press, 1998.
[7] M.A. Vorontsov, "Parallel Image Processing Based on an Evolution Equation with Anisotropic Gain: Integrated Optoelectronic Architectures," J. Optical Soc. Am. A, vol. 16, pp. 1623-1637, 1999.
[8] M.A. Vorontsov and G.W. Carhart, "Anisoplanatic Imaging Through Turbulent Media: Image Recovery by Local Information Fusion from a Set of Short-Exposure Images," J. Optical Soc. Am. A, vol. 18, no. 6, pp. 1312-1324, June 2001.
[9] X. Zhu and P. Milanfar, "Image Reconstruction from Videos Distorted by Atmospheric Turbulence," Proc. SPIE Electronic Imaging Conf. Visual Information Processing and Comm., Jan. 2010.
[10] M. Hirsch, S. Sra, B. Schölkopf, and S. Harmeling, "Efficient Filter Flow for Space-Variant Multiframe Blind Deconvolution," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 607-614, June 2010.
[11] N.M. Law, "Lucky Imaging: Diffraction-Limited Astronomy from the Ground in the Visible," PhD thesis, Cambridge Univ., May 2003.
[12] S. Harmeling, M. Hirsch, S. Sra, and B. Schölkopf, "Online Blind Deconvolution for Astronomical Imaging," Proc. IEEE Int'l Conf. Computational Photography, Apr. 2009.
[13] M.C. Roggemann, C.A. Stoudt, and B.M. Welsh, "Image-Spectrum Signal-to-Noise-Ratio Improvements by Statistical Frame Selection for Adaptive-Optics Imaging through Atmospheric Turbulence," Optical Eng., vol. 33, no. 10, pp. 3254-3264, Oct. 1994.
[14] D.L. Fried, "Probability of Getting a Lucky Short-Exposure Image through Turbulence," Optical Soc. Am., J., vol. 68, pp. 1651-1658, 1978.
[15] M. Aubailly, M.A. Vorontsov, G.W. Carhart, and M.T. Valley, "Image Enhancement by Local Information Fusion with Pre-Processing and Composed Metric," Proc. SPIE, vol. 7090, 2008.
[16] S. John and M.A. Vorontsov, "Multiframe Selective Information Fusion from Robust Error Estimation Theory," IEEE Trans. Image Processing, vol. 14, no. 5, pp. 577-584, May 2005.
[17] M. Aubailly, M.A. Vorontsov, G.W. Carhat, and M.T. Valley, "Automated Video Enhancement from a Stream of Atmospherically-Distorted Images: The Lucky-Region Fusion Approach," Proc. SPIE, vol. 7463, 2009.
[18] N. Joshi and M. Cohen, "Seeing Mt. Rainier: Lucky Imaging for Multi-Image Denoising, Sharpening, and Haze Removal," Proc. IEEE Int'l Conf. Computational Photography, Mar. 2010.
[19] A. Levin, Y. Weiss, F. Durand, and W.T. Freeman, "Understanding and Evaluating Blind Deconvolution Algorithms," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[20] Q. Shan, J. Jia, and A. Agarwala, "High-Quality Motion Deblurring from a Single Image," ACM Trans. Graphics, vol. 27, article 73, 2008.
[21] X. Zhu and P. Milanfar, "Stabilizing and Deblurring Atmospheric Turbulence," Proc. IEEE Int'l Conf. Computational Photography, Apr. 2011.
[22] R. Szeliski and J. Coughlan, "Spline-Based Image Registration," Int'l J. Computer Vision, vol. 22, no. 93, pp. 199-218, 1997.
[23] S. Farsiu, M. Elad, and P. Milanfar, "Constrained, Globally Optimal, Multi-Frame Motion Estimation," Proc. IEEE/SP 13th Workshop Statistical Signal Processing, pp. 1396-1401, July 2005.
[24] M.F. Beg and A. Khan, "Symmetric Data Attachment Terms for Large Deformation Image Registration," IEEE Trans. Medical Imaging, vol. 26, no. 9, pp. 1179-1189, Sept. 2007.
[25] R.F. Dantowitz, S.W. Teare, and M.J. Kozubal, "Ground-Based High-Resolution Imaging of Mercury," The Astronomical J., vol. 119, pp. 2455-2457, May 2000.
[26] J.C. Christou, K.J. Mighell, and R.B. Makidon, "Strehl Ratio and Image Sharpness for Adaptive Optics," Proc. SPIE Advances in Adaptive Optics II, vol. 6272, p. 62721Y, 2006.
[27] L. Davies and U. Gather, "The Identification of Multiple Outliers," J. Am. Statistical Assoc., vol. 88, pp. 782-792, Sept. 1993.
[28] F.R. Hampel, "The Influence Curve and Its Role in Robust Estimation," J. Am. Statistical Assoc., vol. 69, pp. 383-393, 1974.
[29] M.P. Wand and M.C. Jones, Kernel Smoothing, series Monographs on Statistics and Applied Probability. Chapman and Hall, 1995.
[30] H. Takeda, S. Farsiu, and P. Milanfar, "Kernel Regression for Image Processing and Reconstruction," IEEE Trans. Image Processing, vol. 16, no. 2, pp. 349-366, Feb. 2007.
[31] A. Buades, B. Coll, and J.M. Morel, "A Review of Image Denoising Algorithms, with a New One," Multiscale Modeling and Simulation (SIAM Interdisciplinary J.), vol. 4, no. 2, pp. 490-530, 2005.
[32] R. Fergus, B. Singh, A. Hertsmann, S.T. Roweis, and W.T. Freeman, "Removing Camera Shake from a Single Image," Proc. ACM Siggraph, 2006.
[33] A. Levin, R. Fergus, F. Durand, and W.T. Freeman, "Image and Depth from a Conventional Camera with a Coded Aperture," ACM Trans. Graphics, vol. 26, article 70, 2007.
[34] O. Oreifej, G. Shu, T. Pace, and M. Shah, "A Two-Stage Reconstruction Approach for Seeing through Water Clearly," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[35] Y. Tian and S. Narasimhan, "Seeing through Water: Image Restoration Using Model-Based Tracking," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[36] G.B. Giannakis and R.W. Heath, "Blind Identification of Multichannel FIR Blurs and Perfect Image Restoration," IEEE Trans. Image Processing, vol. 9, no. 11, pp. 1877-1896, Nov. 2000.
[37] W. Souidene, K. Abed-Meraim, and A. Beghdadi, "A New Look to Multichannel Blind Image Deconvolution," IEEE Trans. Image Processing, vol. 18, no. 7, pp. 1487-1500, July 2009.
51 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool