The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2008 vol.30)
pp: 299-314
ABSTRACT
Image denoising algorithms often assume an additive white Gaussian noise (AWGN) process that is independent of the actual RGB values. Such approaches are not fully automatic and cannot effectively remove color noise produced by today’s CCD digital camera. In this paper, we propose a unified framework for two tasks: automatic estimation and removal of color noise from a single image using piecewise smooth image models. We introduce the noise level function (NLF), which is a continuous function describing the noise level as a function of image brightness. We then estimate an upper bound of the real noise level function by fitting a lower envelope to the standard deviations of per-segment image variances. For denoising, the chrominance of color noise is significantly removed by projecting pixel values onto a line fit to the RGB values in each segment. Then, a Gaussian conditional random field (GCRF) is constructed to obtain the underlying clean image from the noisy input. Extensive experiments are conducted to test the proposed algorithm, which is shown to outperform state-of-the-art denoising algorithms.
INDEX TERMS
image denoising, piecewise smooth image model, segmentation-based computer vision algorithms, noise estimation, Gaussian conditional random field, automatic vision system
CITATION
Ce Liu, Richard Szeliski, Sing Bing Kang, C. Lawrence Zitnick, William T. Freeman, "Automatic Estimation and Removal of Noise from a Single Image", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 2, pp. 299-314, February 2008, doi:10.1109/TPAMI.2007.1176
REFERENCES
[1] S.P. Awate and R.T. Whitaker, “Higher-Order Image Statistics for Unsupervised, Information-Theoretic, Adaptive, Image Filtering,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[2] D. Barash, “A Fundamental Relationship between Bilateral Filtering, Adaptive Smoothing, and the Nonlinear Diffusion Equation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 844-847, June 2002.
[3] E.P. Bennett and L. McMillan, Video Enhancement Using Per-Pixel Virtual Exposures, ACM SIGGRAPH, pp. 845-852, 2005.
[4] M.J. Black, G. Sapiro, D.H. Marimont, and D. Heeger, “Robust Anisotropic Diffusion,” IEEE Trans. Image Processing, vol. 7, no. 3, pp. 421-432, 1998.
[5] A. Blake and A. Zisserman, Visual Reconstruction. MIT Press, 1987.
[6] A. Buades, B. Coll, and J.-M. Morel, “A Non-Local Algorithm for Image Denoising,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[7] J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 6, pp.679-698, 1986.
[8] D. Comaniciu and P. Meer, “Mean Shift: A Robust Approach Toward Feature Space Analysis,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, pp. 603-619, 2002.
[9] M.S. Crouse, R.D. Nowak, and R.G. Baraniuk, “Wavelet-Based Statistical Signal Processing Using Hidden Markov Models,” IEEE Trans. Signal Processing, vol. 46, no. 4, pp. 886-902, 1998.
[10] D. Donoho, “De-Noising by Soft-Thresholding,” IEEE Trans. Information Theory, vol. 41, no. 3, pp. 613-627, 1995.
[11] F. Durand and J. Dorsey, “Fast Bilateral Filtering for the Display of High-Dynamic-Range Images,” Proc. ACM SIGGRAPH '02, pp.257-266, 2002.
[12] M. Elad and M. Aharon, “Image Denoising via Learned Dictionaries and Sparse Representation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[13] M. Evans, N. Hastings, and B. Peacock, Statistical Distributions. Wiley-Interscience, 2000.
[14] G. Fan and X.-G. Xia, “Image Denoising Using a Local Contextual Hidden Markov Model in the Wavelet Domain,” IEEE Signal Processing Letters, vol. 8, no. 5, pp. 125-128, 2001.
[15] P.F. Felzenszwalb and D.P. Huttenlocher, “Efficient Graph-Based Image Segmentation,” Int'l J. Computer Vision, vol. 59, pp. 167-181, 2004.
[16] D. Field, “Relations between the Statistics of Natural Images and the Response Properties of Cortial Cells,” J. Optical Soc. Am. A, vol. 4, no. 12, pp. 2379-2394, Dec. 1987.
[17] W. Förstner, Image Preprocessing for Feature Extraction in Digital Intensity, Color and Range Images. Springer Lecture Notes on Earth Sciences, 1998.
[18] D. Gabor, “Theory of Communication,” J. IEE, vol. 93, no. 26, pp.429-457, 1946.
[19] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, pp. 721-741, 1984.
[20] T. Gevers and A.W.M. Smeulders, “Color-Based Object Recognition,” Pattern Recognition, vol. 32, no. 3, pp. 453-464, 1999.
[21] E.B. Gindele and N. Serrano, “Estimating Noise for a Digital Image Utilizing Updated Statistics,” US patent 7,054,501, 2006.
[22] R.T. Gray and D.R. Cok, “Adjusting Film Grain Properties in Digital Images,” US patent 5,641,596, 1997.
[23] M.D. Grossberg and S.K. Nayar, “Modeling the Space of Camera Response Functions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1272-1282, Oct. 2004.
[24] G. Healey, “Segmenting Images Using Normalized Color,” IEEE Trans. Systems, Man, and Cybernetics, vol. 22, no. 1, pp. 64-73, 1992.
[25] G.E. Healey and R. Kondepudy, “Radiometric CCD Camera Calibration and Noise Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 16, no. 3, pp. 267-276, Mar. 1994.
[26] G. Klinker, S. Shafer, and T. Kanade, “A Physical Approach to Color Image Understanding,” Int'l J. Computer Vision, vol. 4, no. 1, pp. 7-38, Jan. 1990.
[27] J. Lafferty, A. McCallum, and F. Pereira, “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data,” Proc. Int'l Conf. Machine Learning, 2001.
[28] C. Laroche and M. Prescott, “Apparatus and Methods for Adaptively Interpolating a Full Color Image Utilizing Chrominance Gradients,” U.S. patent 5,373,322, 1994.
[29] A.B. Lee, D. Mumford, and J. Huang, “Occlusion Models for Natural Images: A Statistical Study of a Scale-Invariant Dead Leaves Model,” Int'l J. Compter Vision, vol. 41, nos. 1/2, pp. 35-59, 2001.
[30] C. Liu, W.T. Freeman, R. Szeliski, and S.B. Kang, “Noise Estimation from a Single Image,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 901-908, June 2006.
[31] S.G. Mallat, “A Theory for Multiresolution Signal Decomposition: The Wavelet Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 7, pp. 674-693, July 1989.
[32] D. Martin, C. Fowlkes, D. Tal, and J. Malik, “A Database of Human Segmented Natural Images and Its Application to Evaluating Segmentation Algorithms and Measuring Ecological Statistics,” Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 416-423, July 2001.
[33] D. Mumford and B. Gidas, “Stochastic Models for Generic Images,” Quarterly J. Applied Math., vol. LIX, no. 1, pp. 85-111, 2001.
[34] S. Paris and F. Durand, “A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach,” Proc. European Conf. Computer Vision, 2006.
[35] P. Perona and J. Malik, “Scale-Space and Edge Detection Using Anisotropic Diffusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, July 1990.
[36] G. Petschnigg, R. Szeliski, M. Agrawala, M. Cohen, H. Hoppe, and K. Toyama, “Digital Photography with Flash and No-Flash Image Pairs,” ACM Trans. Graphics, vol. 23, no. 3, pp. 664-672, 2004.
[37] J. Portilla, “Full Blind Denoising through Noise Covariance Estimation Using Gaussian Scale Mixtures in the Wavelet Domain,” Proc. IEEE Int'l Conf. Image Processing, pp. 1217-1220, 2004.
[38] J. Portilla, V. Strela, M.J. Wainwright, and E.P. Simoncelli, “Image Denoising Using Scale Mixtures of Gaussians in the Wavelet Domain,” IEEE Trans. Image Processing, vol. 12, no. 11, pp. 1338-1351, Nov. 2003.
[39] R. Ramanath, W.E. Snyder, G.L. Bilbro, and W.A. Sander, “Demosaicking Methods for Bayer Color Arrays,” J. Electronic Imaging, vol. 11, no. 3, pp. 306-315, July 2002.
[40] J. Rossi, “Digital Techniques for Reducing Television Noise,” J.Soc. Motion Picture and Television Eng., vol. 87, no. 134-140, 1978.
[41] S. Roth and M.J. Black, “Fields of Experts: A Framework for Learning Image Priors,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[42] W.F. Schreiber, Fundamentals of Electronic Imaging Systems. Springer, 1986.
[43] E.P. Simoncelli, “Statistical Models for Images: Compression, Restoration and Synthesis,” Proc. 31st Asilomar Conf. Signals, Systems and Computers, pp. 673-678, 1997.
[44] E.P. Simoncelli and E.H. Adelson, “Noise Removal via Bayesian Wavelet Coring,” Proc. IEEE Int'l Conf. Image Processing, vol. I, pp.379-382, 1996.
[45] P.D. Snyder, T.F. Pawlicki, and R.S. Gaborski, “Apparatus and Method for Signal Dependent Noise Estimation and Reduction in Digital Images,” US patent 5,923,775, 1999.
[46] A. Stefano, P. White, and W. Collis, “Training Methods for Image Noise Level Estimation on Wavelet Components,” EURASIP J.Applied Signal Processing, vol. 16, pp. 2400-2407, 2004.
[47] G. Strang, Introduction to Applied Math. Wellesley-Cambridge Press, 1986.
[48] M.F. Tappen, E.H. Adelson, and W.T. Freeman, “Estimating Intrinsic Component Images Using Non-Linear Regression,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[49] M.F. Tappen, W.T. Freeman, and E.H. Adelson, “Recovering Intrinsic Images from a Single Image,” Advances in Neural Information Processing System, pp. 1343-1350, 2003.
[50] M.F. Tappen, C. Liu, E.H. Adelson, and W.T. Freeman, “Learning Gaussian Conditional Random Fields for Low-Level Vision,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[51] D. Terzopoulos, “Regularization of Inverse Visual Problems Involving Discontinuities,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 4, pp. 413-424, July 1986.
[52] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” Proc. Sixth Int'l Conf. Computer Vision, pp. 839-846, 1998.
[53] D. Tschumperlé, “Fast Anisotropic Smoothing of Multi-Valued Images Using Curvature-Preserving PDE's,” Int'l J. Computer Vision, vol. 68, no. 1, pp. 65-82, 2006.
[54] Y. Tsin, V. Ramesh, and T. Kanade, “Statistical Calibration of CCD Imaging Process,” Proc. IEEE Int'l Conf. Computer Vision, pp. 480-487, 2001.
[55] S.C. Zhu and D. Mumford, “Prior Learning and Gibbs Reaction-Diffusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 11, pp. 1236-1250, Nov. 1997.
[56] S.C. Zhu, Y. Wu, and D. Mumford, “Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling,” Int'l J. Computer Vision, vol. 27, no. 2, pp. 107-126, 1998.
[57] C.L. Zitnick, N. Jojic, and S.B. Kang, “Consistent Segmentation for Optical Flow Estimation,” Proc. IEEE Int'l Conf. Computer Vision, 2005.
39 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool