This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Restoring 2D Content from Distorted Documents
November 2007 (vol. 29 no. 11)
pp. 1904-1916
This paper presents a framework to restore the 2D content printed on documents in the presence of geometric distortion and non-uniform illumination. Compared with textbased document imaging approaches that correct distortion to a level necessary to obtain sufficiently readable text or to facilitate optical character recognition (OCR), our work targets nontextual documents where the original printed content is desired. To achieve this goal, our framework acquires a 3D scan of the document’s surface together with a high-resolution image. Conformal mapping is used to rectify geometric distortion by mapping the 3D surface back to a plane while minimizing angular distortion. This conformal "deskewing" assumes no parametric model of the document’s surface and is suitable for arbitrary distortions. Illumination correction is performed by using the 3D shape to distinguish content gradient edges from illumination gradient edges in the high-resolution image. Integration is performed using only the content edges to obtain a reflectance image with significantly less illumination artifacts. This approach makes no assumptions about light sources and their positions. The results from the geometric and photometric correction are combined to produce the final output.

[1] G. Agam and C. Wu, “Structural Rectification of Non-Planar Document Images: Application to Graphics Recognition,” Lecture Notes in Computer Science, p. 2390, 2002.
[2] A. Agrawal, R. Raskar, and R. Chellappa, “What Is the Range of Surface Reconstructions from a Gradient Field,” Proc. European Conf. Computer Vision, 2006.
[3] H.G. Barrow and J.M. Tenenbaum, Recovering Intrinsic Scene Characteristics from Images. Academic Press, 1978.
[4] M. Bell and W.T. Freeman, “Learning Local Evidence for Shading and Reflection,” Proc. Int'l Conf. Computer Vision, pp. 670-677, 2001.
[5] A. Blake, “Boundary Conditions for Lightness Computation in Mondrian World,” Computer Vision, Graphics and Image Processing, vol. 32, pp. 314-327, 1985.
[6] M.S. Brown and W.B. Seales, “Document Restoration Using 3D Shape: A General Deskewing Algorithm,” Proc. Int'l Conf. Computer Vision, July 2001.
[7] M.S. Brown and W.B. Seales, “Image Restoration of Arbitrarily Warped Documents,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1295-1306, Oct. 2004.
[8] M.S. Brown and Y.C. Tsoi, “Geometric and Shading Correction for Images of Printed Materials Using Boundary,” IEEE Trans. Image Processing, vol. 15, no. 6, pp. 1544-1554, 2006.
[9] M.S. Brown and C.J. Pisula, “Conformal Deskewing of Warped Documents,” Computer Vision and Pattern Recognition, 2005.
[10] H. Cao, X. Ding, and C. Liu, “A Cylindrical Surface Model to Rectify the Bound Document Image,” Proc. Int'l Conf. Computer Vision, pp. 228-233, 2003.
[11] H. Cao, X. Ding, and C. Liu, “Rectifying the Bound Document Image Captured by the Camera: A Model Based Approach,” Proc. Int'l Conf. Document Analysis and Recognition, 2003.
[12] K.B. Chua, L. Zhang, Y. Zhang, and C.L. Tan, “A Fast and Stable Approach for Restoration of Warped Document Images,” Proc. Int'l Conf. Document Analysis and Recognition, 2005.
[13] U. Clarenz, N. Litke, and M. Rumpf, “Axioms and Variational Problems in Surface Parameterization,” Computer Aided Geometric Design, vol. 21, no. 8, pp. 727-749, 2004.
[14] R. Courant, “Plateau's Problem,” Dirichelt's Principle, Conformal Mapping and Minimal Surfaces, chapter 3, pp. 95-134, Springer, 1977.
[15] P. Debevec, “Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,” Proc. Int'l Conf. Computer Graphics and Interactive Techniques, pp. 189-198, 1998.
[16] M. Desburn, M. Meyer, and P. Alliez, “Intrinsic Parameterizations of Surface Meshes,” Proc. Ann. Conf. European Assoc. for Computer Graphics, vol. 21, pp. 209-218, Sept. 2002.
[17] G. Finlayson, S. Hordley, and M. Drew, “Removing Shadows from Images,” Proc. European Conf. Computer Vision, pp. 823-836, 2002.
[18] M. Floater and M. Reimers, “Meshless Parameterization and Surface Reconstruction,” Computer Aided Design, vol. 18, no. 2, pp.77-92, Mar. 2001.
[19] A. Fournier, A. Gunawan, and C. Romanzin, “Common Illumination between Real and Computer Generated Scenes,” Proc. Graphics Interface, 1993.
[20] B.V. Funt, M.S. Drew, and M. Brockington, “Recovering Shading from Color Images,” Proc. European Conf. Computer Vision, pp. 123-132, 1992.
[21] N. Gumerov, A. Zandifar, R. Duraiswami, and L.S. Davis, “Structure of Applicable Surfaces from Single Views,” Proc. European Conf. Computer Vision, 2004.
[22] B.K.P. Horn, “Determining Lightness from an Image,” Computer Vision, Graphics and Image Processing, vol. 3, pp. 277-299, 1974.
[23] G.M. Johnson and M.D. Fairchild, “Full-Spectral Color Calculations in Realistic Image Synthesis,” Computer Graphics and Applications, vol. 19, no. 4, pp. 47-53, July/Aug. 1999.
[24] T. Kanungo, “Document Degradation Models and Methodology for Degradation Model Validation,” PhD dissertation, Dept. of Electrical Eng., Univ. of Washington, Mar. 1996.
[25] T. Kanungo, R.M. Haralick, and I. Phillips, “Global and Local Document Degradation Models,” Proc. Int'l Conf. Document Analysis and Recognition, pp. 730-734, 1993.
[26] S.C. Krantz, “The Cauchy-Riemann Equations,” Handbook of Complex Analysis, Birkhauser, 1999.
[27] E.H. Land and J.J. McCann, “Lightness and Retinex Theory,” J.Optical Soc. Am., vol. 61, pp. 1-11, 1971.
[28] B. Lévy, S. Petijean, R. Nicholas, and J. Maillot, “Least Squares Conformal Maps for Automatic Texture Atlas Generation,” Proc. Int'l Conf. Computer Graphics and Interactive Techniques, pp. 362-371, 2002.
[29] J. Liang, D. DeMenthon, and D. Doermann, “Flattening Curved Documents in Images,” Proc. Computer Vision and Pattern Recognition, pp. 338-345, 2005.
[30] J. Liang, D. Doermann, and H. Li, “Camera-Based Analysis of Text and Documents: A Survey,” Int'l J. Document Analysis and Recognition, vol. 7, no. 2-3, pp. 84-104, July 2005.
[31] S. Marschner and D. Greenberg, “Inverse Lighting for Photography1:74217,” Proc. IS&T/SID Fifth Color Imaging Conf., pp. 262-265, 1997.
[32] B.M. Oh, M. Chen, J. Dorsey, and F. Durand, “Image-Based Modeling and Photo Editing,” Proc. Int'l Conf. Computer Graphics and Interactive Techniques, pp. 433-442, 2001.
[33] P. Perez, M. Gangnet, and A. Blake, “Poisson Image Editing,” Proc. Int'l Conf. Computer Graphics and Interactive, vol. 22, no. 3, pp.313-318, 2003.
[34] M. Pilu, “Undoing Paper Curl Distortion Using Applicable Surfaces,” Proc. Computer Vision and Pattern Recognition, Dec. 2001.
[35] M.X. Sun, R.G. Yang, Y. Lin, G. Landon, B. Seales, and M.S. Brown, “Geometric and Photometric Restoration of Distorted Documents,” Proc. Int'l Conf. Computer Vision, 2005.
[36] C.L. Tan, L. Zhang, Z. Zhang, and T. Xia, “Restoring Warped Document Images through 3D Shape Modeling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 2, pp. 195-208, Feb. 2006.
[37] M.F. Tappen, W.T. Freeman, and E.H. Adelson, “Recovering Intrinsic Images from a Single Image,” Advances in Neural Information Processing Systems, 2003.
[38] R. Tsai, “An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision,” Proc. Computer Vision and Pattern Recognition, pp. 364-374, 1986.
[39] Y.C. Tsoi and M.S. Brown, “Geometric and Shading Correction of Imaged Printed Materials: A Unified Approach Using Boundary,” Proc. Computer Vision and Pattern Recognition, June 2004.
[40] A. Ulges, C.H. Lampert, and T.M. Breuel, “Document Capture Using Stereo Vision,” Proc. ACM Symp. Document Eng., pp. 198-200, 2004.
[41] A. Ulges, C.H. Lampert, and T.M. Breuel, “Document Image Dewarping Using Robust Estimation of Curled Text Lines,” Proc. Int'l Conf. Document Analysis and Recognition, 2005.
[42] T. Wada, H. Ukida, and T. Matsuyama, “Shape from Shading with Interreflections under Proximal Light Source,” Proc. Int'l Conf. Computer Vision, pp. 66-71, 1995.
[43] Y. Weiss, “Deriving Intrinsic Images from Image Sequences,” Proc. Int'l Conf. Computer Vision, vol. 2, pp. 68-75, July 2001.
[44] Y. Yu, P. Debevec, J. Malik, and T. Hawkins, “Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs,” Proc. Int'l Conf. Computer Graphics and Interactive Technique, pp. 215-224, 1999.
[45] L. Zhang, Z. Zhang, C.L. Tan, and X. Tao, “3D Geometric and Optical Modeling of Warped Documents Images from Scanners,” Proc. Computer Vision and Pattern Recognition, 2005.
[46] Z. Zhang and C.L. Tan, “Correcting Document Image Warping Based on Regression of Curved Text Lines,” Proc. Int'l Conf. Document Analysis and Recognition, 2003.
[47] Z. Zhang, C.L. Tan, and L. Fan, “Estimation of 3D Shape of Warped Documents Surface for Image Restoration,” Proc. Computer Vision and Pattern Recognition, 2004.

Index Terms:
Document Restoration, Geometric Correction, Shading Correction, Shading Correction, Photometric Correction, Conformal Mapping, Document Processing
Citation:
Michael S. Brown, Mingxuan Sun, Ruigang Yang, Lin Yun, W. Brent Seales, "Restoring 2D Content from Distorted Documents," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1904-1916, Nov. 2007, doi:10.1109/TPAMI.2007.1118
Usage of this product signifies your acceptance of the Terms of Use.