This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Globally Consistent Reconstruction of Ripped-Up Documents
January 2008 (vol. 30 no. 1)
pp. 1-13
One of the most crucial steps for automatically reconstructing ripped-up documents is to find a globally consistent solution from the ambiguous candidate matches. However, little work has been done so far to solve this problem in a general computational framework without using application-specific features. In this paper, we propose a global approach for reconstructing ripped-up documents by first finding candidate matches from document fragments using curve matching and then disambiguating these candidates through a relaxation process to reconstruct the original document. The candidate disambiguation problem is formulated in a relaxation scheme, in which the definition of compatibility between neighboring matches is proposed and global consistency is defined as the global criterion. Initially, global match confidences are assigned to each of the candidate matches. After that, the overall local relationships among neighboring matches are evaluated by computing their global consistency. Then these confidences are iteratively updated using the gradient projection method to maximize the criterion. This leads to a globally consistent solution and thus provides a sound document reconstruction. The overall performance of our approach in several practical experiments is illustrated. The results indicate that the reconstruction of ripped-up documents up to fifty pieces is possibly accomplished automatically.

[1] H. Freeman and L. Garder, “Apictorial Jigsaw Puzzles: The Computer Solution of a Problem in Pattern Recognition,” IEEE Trans. Electronic Computers, vol. 13, no. 2, pp. 118-127, 1964.
[2] H. Wolfson, A. Kalvin, E. Schonberg, and Y. Lambdan, “Solving Jigsaw Puzzles by Computer,” Annals of Operations Research, vol. 12, pp. 51-64, 1988.
[3] G.C. Burdea and H.J. Wolfson, “Solving Jigsaw Puzzles by a Robot,” IEEE Trans. Robotics and Automation, vol. 5, no. 6, pp. 752-764, 1989.
[4] D. Goldberg, C. Malon, and M. Bern, “A Global Approach to Automatic Solution of Jigsaw Puzzles,” Computational Geometry, vol. 28, no. 2, pp. 165-174, 2004.
[5] F.H. Yao and G.F. Shao, “A Shape and Image Merging Technique to Solve Jigsaw Puzzles,” Pattern Recognition Letters, vol. 24, no. 12, pp. 1819-1835, 2003.
[6] G. Ucoluk and I.H. Toroslu, “Automatic Reconstruction of Broken 3-d Surface Objects,” Computer Graphics, vol. 23, no. 4, pp. 573-582, 1999.
[7] K. Hori, M. Imai, and T. Ogasawara, “Joint Detection for Potsherds of Broken Earthenware,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 440-445, 1999.
[8] H.C.G. Leit˜ao and J. Stolfi, “A Multiscale Method for the Reassembly of Two-Dimensional Fragmented Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 9, pp.1239-1251, Sept. 2002.
[9] C. Papaodysseus, T. Panagopoulos, M. Exarhos, C. Triantafillou, D. Fragoulis, and C. Doumas, “Contour-Shape Based Reconstruction of Fragmented, 1600 B.C. Wall Paintings,” IEEE Trans. Signal Processing, vol. 50, no. 6, pp. 1277-1288, 2002.
[10] F. Amigoni, S. Gazzani, and S. Podico, “A Method for Reassembling Fragments in Image Reconstruction,” Proc. Int'l Conf. Image Processing, vol. 2, pp. 581-584, 2003.
[11] P. De Smet, De Bock, and E. Corluy, “Computer Vision Techniques for Semi-Automatic Reconstruction of Ripped-Up Documents,” Proc. SPIE Visual Information Processing, vol. 5108, pp. 189-196, 2003.
[12] C. Solana, E. Justino, L.S. Oliveira, and F. Bortolozzi, “Document Reconstruction Based on Feature Matching,” Proc. 18th Brazilian Symp. Computer Graphics and Image Processing, pp. 163-170, 2005.
[13] A. Ukovich, G. Ramponi, H. Doulaverakis, Y. Kompatsiaris, and M.G. Strintzis, “Shredded Document Reconstruction Using MPEG-7 Standard Descriptors,” Proc. IEEE Int'l Symp. Signal Processing and Information Technology, pp. 18-21, 2004.
[14] G. Paaioannou, E. Karabassi, and T. Theoharis, “Reconstruction of Three-Dimensional Objects through Matching of Their Parts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 1, pp. 114-124, Jan. 2002.
[15] M. Kano, S. Yasuhara, S. Kato, and H. Itohi, “Similarity-Based Approach to Earthenware Reconstruction,” Proc. Int'l Conf. Image Analysis and Processing, pp. 478-483, 2001.
[16] W.X. Kong and B.B Kimia, “On Solving 2D and 3D Puzzles Using Curve Matching,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 583-590, 2001.
[17] J. McBride and B.B. Kimia, “Archaeological Fragment Reassembly Using Curve-Matching,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2003.
[18] H. Wolfson, “On Curve Matching,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, no. 5, pp. 483-489, May 1990.
[19] A. Rosenfeld, R.A. Hummel, and W. Zucker, “Scene Labeling by Relaxation Operations,” IEEE Trans. Systems, Man, and Cybernetics, vol. 6, no. 6, pp. 420-433, 1976.
[20] R.A. Hummel and S.W. Zucker, “On the Foundations of Relaxation Labeling Processes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 5, no. 3, pp. 267-286, 1983.
[21] P. Faber, “A General Framework for Relaxation Processes,” Technical Report EDI-INF-RR-0057, Univ. of Edinburgh, 2001.
[22] D.H. Ballard and C.M. Brown, Computer Vision. Prentice Hall, 1982.
[23] M. Levy, “New Theoretical Approach to Relaxation, Application to Edge Detection,” Proc. Ninth Int'l Conf. Pattern Recognition, pp.208-212, 1988.
[24] P. Faber, “A Theoretical Framework for Relaxation Processes in Pattern Recognition: Application to Robust Nonparametric Contour Generalization,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 1021-1027, Aug. 2003.
[25] L. Lam and C.Y. Suen, “Structural Classification and Relaxation Matching of Totally Unconstrained Handwritten Zip-Code Numbers,” Pattern Recognition, vol. 21, no. 1, pp. 19-31, 1988.
[26] O. Faugeras and M. Berth, “Improving Consistency and Reducing Ambiguity in Stochastic Labeling: An Optimization Approach,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 3, no. 4, pp. 412-424, 1981.
[27] C. Hilditch, “Linear Skeletons from Square Cupboards,” Machine Intelligence, vol. 4, pp. 403-420, 1969.
[28] Z. Zhang, R. Deriche, O. Faugeras, and Q.T. Luong, “A Robust Technique for Matching Two Uncalibrated Images through the Recovery of the Unknown Epipolar Geometry,” Artificial Intelligence, vol. 78, pp. 87-119, 1995.
[29] W.J. Chiristmas, J. Kittler, and M. Petrou, “Structural Matching in Computer Vision Using Probabilistic Relaxation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 749-764, Aug. 1995.
[30] J. Matas, Z. Shao, and J. Kittler, “Estimation of Curvature and Tangent Direction by Median Filtered Differencing,” Proc. Int'l Conf. Image Analysis and Processing, pp. 13-15, 1995.
[31] E. Calabi, P.J. Oliver, C. Shakiban, A. Tannenbaum, and S. Haker, “Differential and Numerically Invariant Signature Curves Applied to Object Recognition,” J. Computer Vision, vol. 26, pp. 107-135, 1998.
[32] T. Pajdla and L. Van Gool, “Matching of 3-D Curves Using Semi-Differential Invariants,” Proc. Int'l Conf. Computer Vision, pp. 390-395, 1995.
[33] D.P. Bertsekas, Nonlinear Programming, second ed. Athena Scientific, 1999.
[34] P.L. Rosin and G.A.W. West, “Non-Parametric Segmentation of Curves into Various Representations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 12, pp. 1140-1153, Dec. 1995.
[35] E.M. Arkin, L.P. Chew, D.P. Huttenlocher, K. Kadem, and J.S.B. Mitchell, “An Efficiently Computable Metric for Comparing Polygonal Shapes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 3, pp. 209-216, Mar. 1991.
[36] M. Werman and D. Weinshall, “Similarity and Affine Invariant Distances between 2D Point Sets,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 810-814, Aug. 1995.

Index Terms:
Reconstruction of ripped-up documents, compatibility, global consistency, gradient projection, relaxation
Citation:
Liangjia Zhu, Zongtan Zhou, Dewen Hu, "Globally Consistent Reconstruction of Ripped-Up Documents," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 1, pp. 1-13, Jan. 2008, doi:10.1109/TPAMI.2007.1163
Usage of this product signifies your acceptance of the Terms of Use.