This Article 
 Bibliographic References 
 Add to: 
Image Restoration by Matching Gradient Distributions
April 2012 (vol. 34 no. 4)
pp. 683-694
Sing Bing Kang, Microsoft Res., Redmond, WA, USA
N. Joshi, Microsoft Res., Redmond, WA, USA
C. L. Zitnick, Microsoft Res., Redmond, WA, USA
Taeg Sang Cho, WilmerHale, LLP, Boston, MA, USA
R. Szeliski, Microsoft Res., Redmond, WA, USA
W. T. Freeman, Massachusetts Inst. of Technol., Cambridge, MA, USA
The restoration of a blurry or noisy image is commonly performed with a MAP estimator, which maximizes a posterior probability to reconstruct a clean image from a degraded image. A MAP estimator, when used with a sparse gradient image prior, reconstructs piecewise smooth images and typically removes textures that are important for visual realism. We present an alternative deconvolution method called iterative distribution reweighting (IDR) which imposes a global constraint on gradients so that a reconstructed image should have a gradient distribution similar to a reference distribution. In natural images, a reference distribution not only varies from one image to another, but also within an image depending on texture. We estimate a reference distribution directly from an input image for each texture segment. Our algorithm is able to restore rich mid-frequency textures. A large-scale user study supports the conclusion that our algorithm improves the visual realism of reconstructed images compared to those of MAP estimators.

[1] E.P. Bennett and L. McMillan, "Video Enhancement Using Per-Pixel Virtual Exposures," Proc. Siggraph, , pp. 845-852, 2005.
[2] C.A. Bouman and K. Sauer, "A Generalized Gaussian Image Model for Edge-Preserving MAP Estimation," IEEE Trans. Image Processing, vol. 2, no. 3, pp. 296-310, July 1993.
[3] T. Chan and C.-K. Wong, "Total Variation Blind Deconvolution," IEEE Trans. Image Processing, vol. 7, no. 3, pp. 370-375, Mar. 1998.
[4] T.S. Cho, N. Joshi, C.L. Zitnick, S.B. Kang, R. Szeliski, and W.T. Freeman, "A Content-Aware Image Prior," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[5] C.M. Christoudias, B. Georgescu, and P. Meer, "Synergism in Low Level Vision," Proc. IEEE 16th Int'l Conf. Pattern Recognition, 2002.
[6] E. Eisemann and F. Durand, "Flash Photography Enhancement via Intrinsic Relighting," ACM Trans. Graphics, vol. 23, , pp. 673-678, Aug. 2004.
[7] R. Fergus, B. Singh, A. Hertzmann, S. Roweis, and W.T. Freeman, "Removing Camera Shake from a Single Photograph," Proc. ACM Siggraph, 2006.
[8] W.T. Freeman and E.H. Adelson, "The Design and Use of Steerable Filters," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 9, pp. 891-906, Sept. 1991.
[9] W.T. Freeman, E.C. Pasztor, and O.T. Carmichael, "Learning Low-Level Vision," Int'l J. Computer Vision, vol. 40, no. 1, pp. 25-47, 2000.
[10] R.C. Gonzalez and R.E. Woods, Digital Image Processing. Prentice Hall, 2008.
[11] A. Gupta, N. Joshi, C.L. Zitnick, M. Cohen, and B. Curless, "Single Image Deblurring Using Motion Density Functions," Proc. 11th European Conf. Computer Vision, pp. 171-184, 2010.
[12] Y. HaCohen, R. Fattal, and D. Lischinski, "Image Upsampling via Texture Hallucination," Proc. IEEE Int'l Conf. Computational Photography, 2010.
[13] D.J. Heeger and J.R. Bergen, "Pyramid-Based Texture Analysis/Synthesis," Proc. ACM Siggraph, 1995.
[14] N. Joshi and M. Cohen, "Seeing Mt. Rainier: Lucky Imaging for Multi-Image Denoising, Sharpening, and Haze Removal," Proc. IEEE Int'l Conf. Computational Photography, pp. 1-8, 2010.
[15] N. Joshi, S.B. Kang, C.L. Zitnick, and R. Szeliski, "Image Deblurring Using Inertial Measurement Sensors," Proc. ACM Siggraph, , pp. 30:1-30:9, July 2010.
[16] N. Joshi, C.L. Zitnick, R. Szeliski, and D. Kriegman, "Image Deblurring and Denoising Using Color Priors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[17] J. Kopf, C.-W. Fu, D. Cohen-Or, O. Deussen, D. Lischinski, and T.-T. Wong, "Solid Texture Synthesis from 2D Exemplars," ACM Trans. Graphics, vol. 26, no. 3, pp. 2:1-2:9, 2007.
[18] D. Kundur and D. Hatzinakos, "Blind Image Deconvolution Revisited," IEEE Signal Processing Magazine, vol. 13, no. 6, pp. 61-63, Nov. 1996.
[19] J.C. Lagarias, J.A. Reeds, M.H. Wright, and P.E. Wright, "Convergence Properties of the Nelder-Mead Simplex Method in Low Dimensions," SIAM J. Optimization, vol. 9, pp. 112-147, 1998.
[20] A.B. Lee, D. Mumford, and J. Huang, "Occlusion Models for Natural Images: A Statistical Study of a Scale-invariant Dead Leaves Model," Int'l J. Computer Vision, vol. 41, pp. 35-59, 2001.
[21] A. Levin, R. Fergus, F. Durand, and W.T. Freeman, "Image and Depth from a Conventional Camera with a Coded Aperture," Proc. ACM Siggraph, 2007.
[22] Y. Li and E.H. Adelson, "Image Mapping Using Local and Global Statistics," Proc. SPIE Electronic Imaging, vol. 6806, pp. 680614.1-680614.11, 2008.
[23] D. Martin, C. Fowlkes, D. Tal, and J. Malik, "A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics," Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 416-423, July 2001.
[24] G. Matheron, Random Sets and Integral Geometry. John Wiley and Sons, 1975.
[25] M. Nikolova, "Model Distortions in Bayesian MAP Reconstruction," Inverse Problems and Imaging, vol. 1, no. 2, pp. 399-422, 2007.
[26] P. Perona and J. Malik, "Scale-Space and Edge Detection Using Anisotropic Diffusion," IEEE Trans Pattern Analysis and Machine Intelligence, vol. 12, no. 7 pp. 629-639, July 1990.
[27] G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, "Digital Photography with Flash and No-Flash Image Pairs," Proc. ACM Siggraph, , pp. 664-672, 2004.
[28] J. Portilla and E.P. Simoncelli, "A Parametric Texture Model Based on Joint Statistics of Complex Wavelet Coefficients," Int'l J. Computer Vision, vol. 40, no. 1, pp. 49-71, Oct. 2000.
[29] R. Raskar, A. Agrawal, and J. Tumblin, "Coded Exposure Photography: Motion Deblurring Using Fluttered Shutter," Proc. ACM Siggraph, , pp. 795-804, 2006.
[30] J. Rossi, "Digital Techniques for Reducing Television Noise," J. Soc. of Motion Picture and Television Engineers, vol. 87, pp. 134-140, 1978.
[31] S. Roth and M. Black, "Fields of Experts," Int'l J. Computer Vision, vol. 82, pp. 205-229, 2009.
[32] S. Roth and M.J. Black, "Steerable Random Fields," Proc. IEEE 11th Int'l Conf. Computer Vision, 2007.
[33] Y. Saad and M.H. Schultz, "GMRES: A Generalized Minimal Residual Algorithm for Solving Nonsymmetric Linear Systems," SIAM J. Scientific and Statistical Computing, vol. 7, pp. 856-869, 1986.
[34] U. Schmidt, Q. Gao, and S. Roth, "A Generative Perspective on MRFs in Low-Level Vision," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[35] E.P. Simoncelli and E.H. Adelson, "Noise Removal via Bayesian Wavelet Coring," Proc. IEEE Int'l Conf. Image Processing, vol. 1, pp. 379-382, 1996.
[36] C. Tomasi and R. Manduchi, "Bilateral Filtering for Gray and Color Images," Proc. IEEE Int'l Conf. Computer Vision, pp. 839-846, 1998.
[37] Z. Wang, A.C. Bovik, H.R. Sheikh, and E.P. Simoncelli, "Image Quality Assessment: From Error Visibility to Structural Similarity," IEEE Trans. Image Processing, vol. 13, no. 4, pp. 600-612, Apr. 2004.
[38] O. Whyte, J. Sivic, A. Zisserman, and J. Ponce, "Non-Uniform Deblurring for Shaken Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 491-498, 2010.
[39] O.J. Woodford, C. Rother, and V. Kolmogorov, "A Global Perspective on MAP Inference for Low-Level Vision," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[40] L. Yuan, J. Sun, L. Quan, and H.-Y. Shum, "Image Deblurring with Blurred/Noisy Image Pairs," Proc. ACM Siggraph, , 2007.
[41] S. Zhu, Y. Wu, and D. Mumford, "Filters, Random Fields and Maximum Entropy (Frame): Towards a Unified Theory for Texture Modeling," Int'l J. Computer Vision, vol. 27, no. 2, pp. 107-126, 1998.

Index Terms:
maximum likelihood estimation,deconvolution,image restoration,image texture,iterative methods,reference distribution,image restoration,gradient distribution matching,blurry image,noisy image,MAP estimator,maximum a priori estimator,posterior probability,sparse gradient image prior,piecewise smooth image,image texture,visual realism,deconvolution method,iterative distribution reweighting method,Image reconstruction,Image restoration,Noise,Deconvolution,Kernel,Cost function,Gaussian distribution,image denoising.,Nonblind deconvolution,image prior,image deblurring
Sing Bing Kang, N. Joshi, C. L. Zitnick, Taeg Sang Cho, R. Szeliski, W. T. Freeman, "Image Restoration by Matching Gradient Distributions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 683-694, April 2012, doi:10.1109/TPAMI.2011.166
Usage of this product signifies your acceptance of the Terms of Use.