This Article 
 Bibliographic References 
 Add to: 
Vector-Valued Image Regularization with PDEs: A Common Framework for Different Applications
April 2005 (vol. 27 no. 4)
pp. 506-517
In this paper, we focus on techniques for vector-valued image regularization, based on variational methods and PDEs. Starting from the study of PDE-based formalisms previously proposed in the literature for the regularization of scalar and vector-valued data, we propose a unifying expression that gathers the majority of these previous frameworks into a single generic anisotropic diffusion equation. On one hand, the resulting expression provides a simple interpretation of the regularization process in terms of local filtering with spatially adaptive Gaussian kernels. On the other hand, it naturally disassembles any regularization scheme into the smoothing process itself and the underlying geometry that drives the smoothing. Thus, we can easily specialize our generic expression into different regularization PDEs that fulfill desired smoothing behaviors, depending on the considered application: image restoration, inpainting, magnification, flow visualization, etc. Specific numerical schemes are also proposed, allowing us to implement our regularization framework with accuracy by taking the local filtering properties of the proposed equations into account. Finally, we illustrate the wide range of applications handled by our selected anisotropic diffusion equations with application results on color images.

[1] J.F. Abramatic and L.M. Silverman, “Non Linear Restoration of Noisy Images,” IEEE Trans. Patern Analysis and Machine Intelligence, vol. 4, no. 2, pp. 141-149, 1982.
[2] L. Alvarez, F. Guichard, P.L. Lions, and J.M. Morel, “Axioms and Fundamental Equations of Image Processing,” Archive for Rational Mechanics and Analysis, vol. 123, no. 3, pp. 199-257, 1993.
[3] H.C. Andrews and B.R. Hunt, Digital Image Restoration. Signal Processing, Englewood Cliffs, N.J.: Prentice Hall, 1977.
[4] G. Aubert and P. Kornprobst, “Mathematical Problems in Image Processing: Partial Differential Equations and the Calculus of Variations,” Applied Math. Sciences, Springer-Verlag, Jan. 2002.
[5] D. Barash, “Bilateral Filtering and Anisotropic Diffusion: Towards a Unified Viewpoint,” technical report, HP Laboratories, Israel, 2000.
[6] J. Becker, T. Preusser, and M. Rumpf, “Pde Methods in Flow Simulation Post Processing,” Computing and Visualization in Science, vol. 3, no. 3, pp. 159-167, 2000.
[7] M. Bertalmio, L.T. Cheng, S. Osher, and G. Sapiro, “Variational Problems and Partial Differential Equations on Implicit Surfaces: The Framework and Examples in Image Processing and Pattern Formation,” UCLA research report, June 2000.
[8] M. Bertalmio, G. Sapiro, V. Caselles, and C. Ballester, “Image Inpainting,” Proc. SIGGRAPH, K. Akeley, ed., pp. 417-424, 2000.
[9] P. Blomgren, “Total Variation Methods for Restoration of Vector Valued Images,” PhD thesis, Dept. of Math., Univ. of California, Los Angeles, June 1998
[10] P. Blomgren and T.F. Chan, “Color TV: Total Variation Methods for Restoration of Vector-Valued Images,” IEEE Trans. Image Processing, special issue on partial differential equations and geometry-driven diffusion in image processing and analysis, vol. 7, no. 3, pp. 304-309, 1998.
[11] D. Buerkle, T. Preusser, and M. Rumpf, “Transport and Diffusion in Timedependent Flow Visualization,” Proc. IEEE Visualization, 2001.
[12] T. Chan, S.H. Kang, and J. Shen, “Euler's Elastica and Curvature Based Inpainting,” SIAM J. Applied Math., 2002.
[13] T. Chan and J. Shen, “Mathematical Models for Local Deterministic Inpaintings,” Technical Report 00-11, Dept. of Math., UCLA, Los Angeles, Mar. 2000.
[14] T.F. Chan and J. Shen, “Non-Texture Inpainting by Curvature-Driven Diffusions (CDD),” J. Visual Comm. and Image Representation, vol. 12, no. 4, pp. 436-449, 2001.
[15] P. Charbonnier, L. Blanc-Féraud, G. Aubert, and M. Barlaud, “Deterministic Edge-Preserving Regularization in Computed Imaging,” IEEE Trans. Image Processing, vol. 6, no. 2, pp. 298-311, 1997.
[16] R. Deriche and O. Faugeras, “Les EDP en Traitement des Images et Vision par Ordinateur,” technical report, INRIA, Nov. 1995.
[17] U. Diewald, T. Preusser, and M. Rumpf, “Anisotropic Diffusion in Vector Field Visualization on Euclidian Domains and Surfaces,” IEEE Trans. Visualization and Computer Graphics, vol. 6, no. 2, pp. 139-149, Apr.-June 2000.
[18] B.R. Frieden, “Restoring with Maximum Likelihood and Maximum Entropy,” J. Optical Soc. Am., vol. 62, pp. 511-518, 1972.
[19] S. Geman and D. Geman, “Stochastic Relaxation, Gibbs Distributions, and the Bayesian Restoration of Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 6, no. 6, pp. 721-741, 1984.
[20] G. Gerig, O. Kubler, R. Kikinis, and F. Jolesz, “Nonlinear Anisotropic Filtering of MRI Data,” IEEE Trans. Medical Imaging, vol. 11, no. 2, pp. 221-231, 1992.
[21] B.R. Hunt, “The Application of Constrained Least Squares Estimation to Image Restoration by Digital Computer,” IEEE Trans. Computers, vol. 22, no. 9, pp. 805-812, Sept. 1973.
[22] R. Kimmel, R. Malladi, and N. Sochen, “Images as Embedded Maps and Minimal Surfaces: Movies, Color, Texture, and Volumetric Medical Images,” Int'l J. Computer Vision, vol. 39, no. 2, pp. 111-129, Sept. 2000.
[23] J.J. Koenderink, “The Structure of Images,” Biological Cybernetics, vol. 50, pp. 363-370, 1984.
[24] P. Kornprobst, “Contributions à la Restauration d'Images et à l'Analyse de Séquences: Approches Variationnelles et Solutions de Viscosité,” PhD thesis, Université de Nice-Sophia Antipolis, 1998.
[25] P. Kornprobst, R. Deriche, and G. Aubert, “Image Restoration via PDE's,” Proc. First Ann. Symp. Enabling Technologies for Law Enforcement and Security— SPIE Conf. 2942: Investigative Image Processing, Nov. 1996.
[26] P. Kornprobst, R. Deriche, and G. Aubert, “Nonlinear Operators in Image Restoration” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 325-331, 1997.
[27] T. Lindeberg, Scale-Space Theory in Computer Vision. Kluwer Academic Publishers, 1994.
[28] L. Lucido, R. Deriche, L. Alvarez, and V. Rigaud, “Sur Quelques Schémas Numériques de Résolution d'Équations aux dÉrivées Partielles pour le Traitement d'Images,” Rapport de Recherche 3192, INRIA, June 1997.
[29] Y. Meyer, “Oscillatory Patterns in Image Processing and Nonlinear Evolution Equations,” Univ. Lecture Series, vol. 22, Am. Math. Soc., Providence, R.I., 2001.
[30] M. Nielsen, L. Florack, and R. Deriche, “Regularization, Scale-Space and Edge Detection Filters,” J. Math. Imaging and Vision, vol. 7, no. 4, pp. 291-308, 1997. Related ECCV:96 article available at:
[31] M. Nikolova, “Image Restoration by Minimizing Objective Functions with Nonsmooth Data-Fidelity Terms,” Proc. IEEE Workshop Variational and Level Set Methods, pp. 11-19, 2001.
[32] T. Papadopoulo and M.I. A. Lourakis, “Estimating the Jacobian of the Singular Value Decomposition: Theory and Applications,” Research Report 3961, INRIA Sophia-Antipolis, June 2000.
[33] A. Pardo and G. Sapiro, “Vector Probability Diffusion,” Proc. Int'l Conf. Image Processing, 2000.
[34] P. Perona and J. Malik, “Scale-Space and Edge Detection Using Anisotropic Diffusion,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 7, pp. 629-639, July 1990.
[35] G. Sapiro, “Vector-Valued Active Contours,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 680-685, 1996.
[36] G. Sapiro, “Color Snakes,” Computer Vision and Image Understanding, vol. 68, no. 2, 1997.
[37] G. Sapiro, Geometric Partial Differential Equations and Image Analysis. Cambridge Univ. Press, 2001.
[38] G. Sapiro and D.L. Ringach, “Anisotropic Diffusion of Multivalued Images with Applications to Color Filtering,” IEEE Trans. Image Processing, vol. 5, no. 11, pp. 1582-1585, 1996.
[39] J. Shah, “Curve Evolution and Segmentation Functionals: Applications to Color Images,” Proc. Int'l Conf. Image Processing, pp. 461-464, 1996.
[40] N. Sochen, R. Kimmel, and A.M. Bruckstein, “Diffusions and Confusions in Signal and Image Processing,” J. Math. Imaging and Vision, vol. 14, no. 3, pp. 195-209, 2001.
[41] B. Tang, G. Sapiro, and V. Caselles, “Direction Diffusion,” Proc. Int'l Conf. Computer Vision, 1998.
[42] B. Tang, G. Sapiro, and V. Caselles, “Diffusion of General Data on Non-Flat Manifolds via Harmonic Maps Theory: The Direction Diffusion Case,” The Int'l J. Computer Vision, vol. 36, no. 2, pp. 149-161, Feb. 2000.
[43] C. Tomasi and R. Manduchi, “Bilateral Filtering for Gray and Color Images,” Proc. IEEE Int'l Conf. Computer Vision, pp. 839-846, Jan. 1998.
[44] D. Tschumperlé, “PDE's Based Regularization of Multivalued Images and Applications,” PhD thesis, Université de Nice-Sophia Antipolis, Dec. 2002.
[45] D. Tschumperlé and R. Deriche, “Constrained and Unconstrained PDE's for Vector Image Restoration,” Proc. 10th Scandinavian Conf. Image Analysis, Ivar Austvoll, ed., pp. 153-160, 2001.
[46] D. Tschumperlé and R. Deriche, “Diffusion Tensor Regularization with Constraints Preservation,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, 2001.
[47] D. Tschumperlé and R. Deriche, “Regularization of Orthonormal Vector Sets Using Coupled PDE's,” Proc. IEEE Workshop Variational and Level Set Methods, pp. 3-10, 2001.
[48] D. Tschumperlé and R. Deriche, “Diffusion PDE's on Vector-Valued Images,” IEEE Signal Processing Magazine, vol. 19, no. 5, pp. 16-25, 2002.
[49] B. Vemuri, Y. Chen, M. Rao, T. McGraw, T. Mareci, and Z. Wang, “Fiber Tract Mapping from Diffusion Tensor MRI,” Proc. First IEEE Workshop Variational and Level Set Methods in Computer Vision, 2001.
[50] J. Weickert, “Anisotropic Diffusion in Image Processing,” PhD thesis, Univ. of Kaiserslautern, Germany, Laboratory of Tech nomath., Jan. 1996.
[51] J. Weickert, Anisotropic Diffusion in Image Processing. Stuttgart: Teubner-Verlag, 1998.
[52] P. Wesseling, Principles of Computational Fluid Dynamics. Springer-Verlag, 2000.
[53] A.P. Witkin, “Scale-Space Filtering,” Proc. Int'l Joint Conf. Artificial Intelligence, pp. 1019-1021, 1983.
[54] A. Yezzi, “Modified Curvature Motion for Image Smoothing and Enhancement,” IEEE Trans. Image Processing, pp 345-352, vol. 7, no. 3, Mar. 1998.
[55] S. Di Zenzo, “A Note on the Gradient of a Multi-Image,” Computer Vision, Graphics, and Image Processing, vol. 33, pp. 116-125, 1986.

Index Terms:
Diffusion PDEs, color image regularization, denoising, inpainting, vector-valued smoothing, anisotropic filtering, flow visualization.
David Tschumperl?, Rachid Deriche, "Vector-Valued Image Regularization with PDEs: A Common Framework for Different Applications," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 506-517, April 2005, doi:10.1109/TPAMI.2005.87
Usage of this product signifies your acceptance of the Terms of Use.