This Article 
 Bibliographic References 
 Add to: 
Goal-Directed Evaluation of Binarization Methods
December 1995 (vol. 17 no. 12)
pp. 1191-1201

Abstract—This paper presents a methodology for evaluation of low-level image analysis methods, using binarization (two-level thresholding) as an example. Binarization of scanned gray scale images is the first step in most document image analysis systems. Selection of an appropriate binarization method for an input image domain is a difficult problem. Typically, a human expert evaluates the binarized images according to his/her visual criteria. However, to conduct an objective evaluation, one needs to investigate how well the subsequent image analysis steps will perform on the binarized image. We call this approach goal-directed evaluation, and it can be used to evaluate other low-level image processing methods as well. Our evaluation of binarization methods is in the context of digit recognition, so we define the performance of the character recognition module as the objective measure. Eleven different locally adaptive binarization methods were evaluated, and Niblack’s method gave the best performance.

[1] Proc. First Int’l Conf. Document Analysis and Recognition,Saint-Malo, France, 1991.
[2] Proc. Second Int’l Conf. Document Analysis and Recognition,Tsukuba Science City, Japan, 1993.
[3] R. Kasturi and L. O’Gorman, guest eds., “Special issue: Document image analysis techniques,” , Machine Vision and Applications, vol. 5, no. 3, pp. 141-248, 1992.
[4] R. Kasturi and L. O’Gorman, guest eds., “Special issue: Document image analysis techniques,” , Machine Vision and Applications, vol. 6, no. 2-3, pp. 67-180, 1993.
[5] L. O’Gorman and R. Kasturi, guest eds., “Special issue on document image analysis systems,” Computer, vol. 25, no. 7, pp. 1-112, July 1992.
[6] R. Plamondon, Guest ed., “, Special issue: Handwriting processing and recognition,” Pattern Recognition, vol. 26, no. 3, pp. 379-460, Mar. 1993.
[7] T. Pavlidis and S. Mori, guest eds., “Special issue on optical character recognition,” Proc. IEEE, vol. 80, no. 7, pp. 1,027-1,215, July 1992.
[8] H. Tominaga, guest ed., “, Special issue on postal processing and character recognition,” Pattern Recognition Letters, vol. 14, no. 4, pp. 257-354, Apr. 1993.
[9] O.D. Trier and T. Taxt, “Evaluation of Binarization Methods for Document Images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 3, pp. 312-314, Mar. 1995.
[10] S.-W. Lee,L. Lam,, and C.Y. Suen,“Performance evaluation of skeletonizing algorithms for documentimage processing,” Proc. First Int’l Conf. Document Analysis and Recognition, pp. 260-271,Saint-Malo, France, 1991.
[11] S.O. Belkasim,M. Shridhar,, and M. Ahmadi,“Pattern recognition with moment invariants: A comparative study and new results,” Pattern Recognition, vol. 24, pp. 1117-1138, 1991.
[12] R. Haralick, “Performance Characterization in Image Analysis: Thinning, a Case in Point,” Pattern Recognition Letters, vol. 13, pp. 5-12, 1992.
[13] M.Y. Jaisimha,R.M. Haralick,, and D. Dori,“Quantitative performance evaluation of thinning algorithms under noisy conditions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 678-683,Seattle, Wash., June 1994.
[14] Ø.D. Trier,A.K. Jain,, and T. Taxt,“Feature extraction methods for character recognition—a survey,” Technical Report, Michigan State Univ., Dec. 1994, to appear inPattern Recognition.
[15] R. Duda, P. Hart, and D. Stork, Pattern Classification. New York: John Wiley&Sons, 2001.
[16] K.-S. Fu,Syntactic Pattern Recognition and Application. Englewood Cliffs, N.J.: Prentice Hall, 1982.
[17] J. Bernsen,“Dynamic thresholding of grey-level images,” Proc. Eighth Int’l Conf. Pattern Recognition, pp. 1,251-1,255,Paris, 1986.
[18] C.K. Chow and T. Kaneko,“Automatic detection of the left ventricle from cineangiograms,” Computers and Biomedical Research, vol. 5, pp. 388-410, 1972.
[19] Y. Nakagawa and A. Rosenfeld,“Some experiments on variable thresholding,” Pattern Recognition, vol. 11, no. 3, pp. 191-204, 1979.
[20] L. Eikvil,T. Taxt,, and K. Moen,“A fast adaptive method for binarization of document images,” Proc. First Int’l Conf. Document Analysis and Recognition, pp. 435-443,Saint-Malo, France, 1991.
[21] K.V. Mardia and T.J. Hainsworth,“A spatial thresholding method for image segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, no. 6, pp. 919-927, 1988.
[22] W. Niblack, An Introduction to Digital Image Processing, pp. 115-116, Englewood Cliffs, N.J.: Prentice Hall, 1986.
[23] T. Taxt,P.J. Flynn,, and A. K. Jain,“Segmentation of document images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 12, pp. 1322-1329, 1989.
[24] S.D. Yanowitz and A.M. Bruckstein, “A New Method for Image Segmentation,” Computer Vision, Graphics, and Image Processing, vol. 46, pp. 82-95, 1989.
[25] J.M. White and G.D. Rohrer,“Image thresholding for optical character recognition and otherapplications requiring character image extraction,” IBM J. Research and Development, vol. 27, no. 4, pp. 400-411, July 1983.
[26] J.R. Parker,“Gray level thresholding in badly illuminated images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 8, pp. 813-819, 1991.
[27] Ø. D. Trier and T. Taxt,“Improvement of’integrated function algorithm’for binarization of document images,” Pattern Recognition Letters, to appear.
[28] A.S. Abutaleb, “Automatic Thresholding of Gray-Level Pictures Using Two-Dimensional Entropy,” Computer Vision, Graphics, and Image Processing, vol. 47, pp. 22-32, 1989.
[29] J.N. Kapur,P.K. Sahoo,, and A.K.C. Wong,“A new method for gray-level picture thresholding using theentropy of the histogram,” Computer Vision, Graphics and Image Processing, vol. 29, pp. 273-285, 1985.
[30] J. Kittler and J. lllingworth, “Minimum Error Thresholding,” Pattern Recognition, vol. 19, pp. 41-47, 1986.
[31] N. Otsu,“A threshold selection method from gray-level histograms,” IEEE Trans. Systems, Man, and Cybernetics, vol. 9, no. 1, pp. 62-66, 1979.
[32] T. Kurita,N. Otsu,, and N. Abdelmalek,“Maximum likelihood thresholding based on population mixture models,” Pattern Recognition, vol. 25, no. 10, pp. 1,231-1,240, 1992.
[33] S. Cho,R. Haralick,, and S. Yi,“Improvement of Kittler andIllingworth’s minimum error thresholding,” Pattern Recognition, vol. 22, no. 5, pp. 609-617, 1989.
[34] D.M. Titterington,A.F.M. Smith,, and U.E. Makov,Statistical Analysis of Finite Mixture Distributions.New York: John Wiley&Sons, 1985.
[35] A.K. Jain and R.C. Dubes, Algorithms for Clustering Data. Englewood Cliffs, N.J.: Prentice Hall, 1988.
[36] W.K. Pratt, Digital Image Processing. Wiley-Interscience, 1991.
[37] A.K. Jain and M.-P. Dubuisson,“Segmentation of X-ray and C-scan images of fiber reinforcedcomposite materials,” Pattern Recognition, vol. 25, no. 3, pp. 257-270, 1992.
[38] Jim M. White,1994, Private communication.
[39] F.P. Kuhl and C.R. Giardina,“Elliptic Fourier features of a closed contour,” Computer Vision, Graphics and Image Processing, vol. 18, pp. 236-258, 1982.
[40] T. Taxt,J.B. Olafsdóttir,, and M. Dæhlen,“Recognition of handwritten symbols,” Pattern Recognition, vol. 23, no. 11, pp. 1,155-1,166, 1990.
[41] N.L. Hjort,“Notes on the theory of statistical pattern recognition,” Report no. 778, NorwegianComputing Center, Oslo, Norway, Dec. 1986.
[42] K.V. Mardia,J.T. Kent,, and J.M. Bibby,Multivariate Analysis.London: Academic Press, 1979.
[43] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[44] S.A. Glantz,Primer of Biostatistics, third edition, pp. 311-314.New York: McGraw-Hill, 1992.
[45] Tim Hesterberg,1994, Private communication.

Index Terms:
Objective evaluation, performance evaluation, binarization, segmentation, document images.
Øivind Due Trier, Anil K. Jain, "Goal-Directed Evaluation of Binarization Methods," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 17, no. 12, pp. 1191-1201, Dec. 1995, doi:10.1109/34.476511
Usage of this product signifies your acceptance of the Terms of Use.