This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Metric Rectification of Curved Document Images
April 2012 (vol. 34 no. 4)
pp. 707-722
ASCII Text
x 
 
Gaofeng Meng, Chunhong Pan, Shiming Xiang, Jiangyong Duan, "Metric Rectification of Curved Document Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 707-722, April, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.151,
author = {Gaofeng Meng and Chunhong Pan and Shiming Xiang and Jiangyong Duan},
title = {Metric Rectification of Curved Document Images},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {34},
number = {4},
issn = {0162-8828},
year = {2012},
pages = {707-722},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.151},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Metric Rectification of Curved Document Images
IS - 4
SN - 0162-8828
SP707
EP722
EPD - 707-722
A1 - Gaofeng Meng,
A1 - Chunhong Pan,
A1 - Shiming Xiang,
A1 - Jiangyong Duan,
PY - 2012
KW - Document image analysis
KW - imaging geometry
KW - geometric correction
KW - shape-from-X
KW - mesh warping.
VL - 34
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Gaofeng Meng, Chinese Academy of Sciences, Beijing
Chunhong Pan, Chinese Academy of Sciences, Beijing
Shiming Xiang, Chinese Academy of Sciences , Beijing
Jiangyong Duan, Chinese Academy of Sciences, Beijing
In this paper, we propose a metric rectification method to restore an image from a single camera-captured document image. The core idea is to construct an isometric image mesh by exploiting the geometry of page surface and camera. Our method uses a general cylindrical surface (GCS) to model the curved page shape. Under a few proper assumptions, the printed horizontal text lines are shown to be line convergent symmetric. This property is then used to constrain the estimation of various model parameters under perspective projection. We also introduce a paraperspective projection to approximate the nonlinear perspective projection. A set of close-form formulas is thus derived for the estimate of GCS directrix and document aspect ratio. Our method provides a straightforward framework for image metric rectification. It is insensitive to camera positions, viewing angles, and the shapes of document pages. To evaluate the proposed method, we implemented comprehensive experiments on both synthetic and real-captured images. The results demonstrate the efficiency of our method. We also carried out a comparative experiment on the public CBDAR2007 data set. The experimental results show that our method outperforms the state-of-the-art methods in terms of OCR accuracy and rectification errors.

[1] Z. Zhang and C.L. Tan, "Correcting Document Image Warping Based on Regression of Curved Text Lines," Proc. Seventh Int'l Conf. Document Analysis and Recognition, vol. 1, pp. 589-593, 2003.
[2] Y.C. Tsoi and M.S. Brown, "Geometric and Shading Correction for Images of Printed Materials: A Unified Approach Using Boundary," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 240-246, 2004.
[3] A. Ulges, C.H. Lampert, and T.M. Breuel, "Document Image Dewarping Using Robust Estimation of Curled Text Lines," Proc. Eighth Int'l Conf. Document Analysis and Recognition, vol. 2, pp. 1001-1005, 2005.
[4] M.S. Brown and Y.C. Tsoi, "Geometric and Shading Correction for Images of Printed Materials Using Boundary," IEEE Trans. Image Processing, vol. 15, no. 6, pp. 1544-1554, June 2006.
[5] M. Pilu, "Undoing Paper Curl Distortion Using Applicable Surfaces," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 67-72, 2001.
[6] M.S. Brown and W.B. Seales, "Image Restoration of Arbitrarily Warped Documents," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1295-1306, Oct. 2004.
[7] M. Sun, R. Yang, L. Yun, G. Landon, B. Seales, and M.S. Brown, "Geometric and Photometric Restoration of Distorted Documents," Proc. 10th IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1117-1123, 2005.
[8] M.S. Brown, M. Sun, R. Yang, L. Yun, and W.B. Seales, "Restoring 2D Content from Distorted Documents," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 11, pp. 1904-1916, Nov. 2007.
[9] F. Courteille, A. Crouzil, J.-D. Durou, and P. Gurdjos, "Shape from Shading for the Digitization of Curved Documents," Machine Vision and Applications, vol. 18, pp. 301-316, 2007.
[10] K. Fujimoto, J. Sun, H. Takebe, M. Suwa, and S. Naoi, "Curved Paper Rectification for Digital Camera Document Images by Shape from Parallel Geodesics Using Continuous Dynamic Programming," Proc. Ninth Int'l Conf. Document Analysis and Recognition, vol. 1, pp. 267-271, 2007.
[11] L. Zhang, A.M. Yip, and C.L. Tan, "A Restoration Framework for Correcting Photometric and Geometric Distortions in Camera-Based Document Images," Proc. 11th IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.
[12] L. Zhang, Y. Zhang, and C.L. Tan, "An Improved Physically-Based Method for Geometric Restoration of Distorted Document Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 728-734, Apr. 2008.
[13] L. Zhang, A.M. Yip, M.S. Brown, and C.L. Tan, "A Unified Framework for Document Restoration Using Inpainting and Shape-from-Shading," Pattern Recognition, vol. 42, no. 11, pp. 2961-2978, 2009.
[14] H. Cao, X. Ding, and C. Liu, "A Cylindrical Surface Model to Rectify the Bound Document Image," Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 228-233, 2003.
[15] J. Liang, D. DeMenthon, and D. Doermann, "Geometric Rectification of Camera-Captured Document Images," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 4, pp. 591-605, Apr. 2008.
[16] C.L. Tan, L. Zhang, Z. Zhang, and T. Xia, "Restoring Warped Document Images through 3D Shape Modeling," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 2, pp. 195-208, Feb. 2006.
[17] J. Liang, D. DeMenthon, and D. Doermann, "Flattening Curved Documents in Images," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 338-345, 2005.
[18] H.I. Koo, J. Kim, and N.I. Cho, "Composition of a Dewarped and Enhanced Document Image from Two View Images," IEEE Trans. Image Processing, vol. 18, no. 7, pp. 1551-1562, July 2009.
[19] Y.C. Tsoi and M.S. Brown, "Multi-View Document Rectification Using Boundary," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2007.
[20] S. Lu and C.L. Tan, "Document Flattening through Grid Modeling and Regularization," Proc. Int'l Conf. Pattern Recognition, pp. 971-974, 2006.
[21] D.C. Schneider, M. Block, and R. Rojas, "Robust Document Warping with Interpolated Vector Fields," Proc. Ninth Int'l Conf. Document Analysis and Recognition, pp. 113-117, 2007.
[22] M.S. Brown and W.B. Seales, "Document Restoration Using 3D Shape: A General Deskewing Algorithm for Arbitrarily Warped Documents," Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 367-374, 2001.
[23] T. Wada, H. Ukida, and T. Matsuyama, "Shape from Shading with Interreflections under a Proximal Light Source: Distortion-Free Copying of an Unfolded Book," Int'l J. Computer Vision, vol. 24, no. 2, pp. 125-135, 1997.
[24] N. Gumerov, A. Zandifar, R. Duraiswami, and L. Davis, "Structure of Applicable Surfaces from Single Views," Proc. Eighth European Conf. Computer Vision, pp. 482-496, 2004.
[25] K. Tapas, M.H. Robert, and P. Ihsin, "Global and Local Document Degradation Models," Proc. Third Int'l Conf. Document Analysis and Recognition, pp. 730-734, 1993.
[26] L. Zhang, Z. Zhang, C.L. Tan, and T. Xia, "3D Geometric and Optical Modeling of Warped Document Images from Scanners," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 337-342, 2005.
[27] Z. Zhang, C.L. Tan, and L. Fan, "Restoration of Curved Document Images through 3D Shape Modeling," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 10-15, 2004.
[28] G. Wolberg, "Image Morphing: A Survey," The Visual Computer, vol. 14, no. 8, pp. 360-372, 1998.
[29] S. Basu, C. Chaudhuri, M. Kundu, M. Nasipuri, and D. Basu, "Text Line Extraction from Multi-Skewed Handwritten Documents," Pattern Recognition, vol. 40, no. 6, pp. 1825-1839, 2007.
[30] S.S. Bukhari, F. Shafait, and T.M. Breuel, "Dewarping of Document Images Using Coupled-Snakes," Proc. Third Int'l Workshop Camera-Based Document Analysis and Recognition, pp. 34-41, July 2009.
[31] S. Lu, B.M. Chen, and C.C. Ko, "Perspective Rectification of Document Images Using Fuzzy Set and Morphological Operations," Image and Vision Computing, vol. 23, pp. 541-553, 2005.
[32] F. Shafait and T. Breuel, "The Effect of Border Noise on the Performance of Projection-Based Page Segmentation Methods," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 4, pp. 846-851, Apr. 2011.
[33] R. Bajcsy, "Computer Description of Textured Surfaces," Proc. Third Int'l Joint Conf. Artificial Intelligence, pp. 572-579, 1973.
[34] T. Abatzoglou and B. O'Donnell, "Minimization by Coordinate Descent," J. Optimization Theory and Applications, vol. 36, pp. 163-174, 1982.
[35] B. Fu, M. Wu, R. Li, W. Li, and Z. Xu, "A Model-Based Book Dewarping Method Using Text Line Detection," Proc. Second Int'l Workshop Camera-Based Document Analysis and Recognition, pp. 63-70, Sept. 2007.
[36] Y.-i. Ohta, K. Maenobu, and T. Sakai, "Obtaining Surface Orientation from Texels under Perspective Projection," Proc. Seventh Int'l Joint Conf. Artificial Intelligence, pp. 746-751, 1981.
[37] S.T. Barnard, "Interpreting Perspective Images," Artificial Intelligence, vol. 21, no. 4, pp. 435-462, 1983.
[38] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[39] F. Shafait and T.M. Breuel, "Document Image Dewarping Contest," Proc. Second Int'l Workshop Camera-Based Document Analysis and Recognition, pp. 181-188, Sept. 2007.
[40] B. Gatos, I. Pratikakis, and K. Ntirogiannis, "Segmentation Based Recovery of Arbitrarily Warped Document Images," Proc. Ninth Int'l Conf. Document Analysis and Recognition, pp. 989-993, Sept. 2007.
[41] A. Masalovitch and L. Mestetskiy, "Usage of Continuous Skeletal Image Representation for Document Images Dewarping," Proc. Second Int'l Workshop Camera-Based Document Analysis and Recognition, pp. 45-52, Sept. 2007.
[42] V. Levenshtein, "Binary Codes Capable of Correcting Deletions, Insertions and Reversals," Soviet Physics Doklady, vol. 10, pp. 707-710, 1966.
[43] D. Liebowitz and A. Zisserman, "Metric Rectification for Perspective Images of Planes," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 482-488, 1998.

Index Terms:
Document image analysis, imaging geometry, geometric correction, shape-from-X, mesh warping.
Citation:
Gaofeng Meng, Chunhong Pan, Shiming Xiang, Jiangyong Duan, "Metric Rectification of Curved Document Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 4, pp. 707-722, April 2012, doi:10.1109/TPAMI.2011.151
Usage of this product signifies your acceptance of the Terms of Use.