The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.04 - April (2011 vol.33)
pp: 767-779
Salvador España-Boquera , Universitat Politècnica de València, Valencia
Maria Jose Castro-Bleda , Universitat Politècnica de València, Valencia
Jorge Gorbe-Moya , Universitat Politècnica de València, Valencia
Francisco Zamora-Martinez , Universidad CEU-Cardenal Herrera, Alfara del Patriarca and Universitat Politècnica de València, Valencia
ABSTRACT
This paper proposes the use of hybrid Hidden Markov Model (HMM)/Artificial Neural Network (ANN) models for recognizing unconstrained offline handwritten texts. The structural part of the optical models has been modeled with Markov chains, and a Multilayer Perceptron is used to estimate the emission probabilities. This paper also presents new techniques to remove slope and slant from handwritten text and to normalize the size of text images with supervised learning methods. Slope correction and size normalization are achieved by classifying local extrema of text contours with Multilayer Perceptrons. Slant is also removed in a nonuniform way by using Artificial Neural Networks. Experiments have been conducted on offline handwritten text lines from the IAM database, and the recognition rates achieved, in comparison to the ones reported in the literature, are among the best for the same task.
INDEX TERMS
Handwriting recognition, offline handwriting, hybrid HMM/ANN, HMM, neural networks, multilayer perceptron, image normalization.
CITATION
Salvador España-Boquera, Maria Jose Castro-Bleda, Jorge Gorbe-Moya, Francisco Zamora-Martinez, "Improving Offline Handwritten Text Recognition with Hybrid HMM/ANN Models", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.33, no. 4, pp. 767-779, April 2011, doi:10.1109/TPAMI.2010.141
REFERENCES
[1] T. Steinherz, E. Rivlin, and N. Intrator, "Offline Cursive Script Word Recognition—A Survey," Int'l J. Document Analysis and Recognition, vol. 2, no. 2, pp. 90-110, 1999.
[2] R. Plamondon and S.N. Srihari, "On-Line and Off-Line Handwritting Recognition: A Comprehensive Survey," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 1, pp. 63-84, Jan. 2000.
[3] N. Arica and F. Yarman-Vural, "An Overview of Character Recognition Focused on Off-Line Handwriting," IEEE Trans. Systems, Man, and Cybernetics, Part C: Applications and Rev., vol. 31, no. 2, pp. 216-233, May 2001.
[4] A. Vinciarelli, "A Survey on Off-Line Cursive Word Recognition," Pattern Recognition, vol. 35, no. 7, pp. 1433-1446, 2002.
[5] H. Bunke, "Recognition of Cursive Roman Handwriting—Past, Present, and Future," Proc. Seventh Int'l Conf. Document Analysis and Recognition, vol. 1, pp. 448-459, Aug. 2003.
[6] A. Koerich, R. Sabourin, and C. Suen, "Large Vocabulary Off-Line Handwriting Recognition: A Survey," Pattern Analysis and Applications, vol. 6, no. 2, pp. 97-121, 2003.
[7] H. Fujisawa, "Forty Years of Research in Character and Document Recognition—An Industrial Perspective," Pattern Recognition, vol. 41, no. 8, pp. 2435-2446, 2008.
[8] A. El-Yacoubi, M. Gilloux, R. Sabourin, and C.Y. Suen, "An HMM-Based Approach for Off-Line Unconstrained Handwritten Word Modeling and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 8, pp. 752-760, Aug. 1999.
[9] U.-V. Marti and H. Bunke, "Using a Statistical Language Model to Improve the Performance of an HMM-Based Cursive Handwriting Recognition Systems," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 15, no. 1, pp. 65-90, 2001.
[10] A.H. Toselli, A. Juan, J. González, I. Salvador, E. Vidal, F. Casacuberta, D. Keysers, and H. Ney, "Integrated Handwriting Recognition and Interpretation Using Finite-State Models," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 18, no. 4, pp. 519-539, 2004.
[11] A. Vinciarelli, S. Bengio, and H. Bunke, "Offline Recognition of Unconstrained Handwritten Texts Using HMMs and Statistical Language Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 6, pp. 709-720, June 2004.
[12] J. Gorbe-Moya, S. España-Boquera, F. Zamora-Martínez, and M.J. Castro-Bleda, "Handwritten Text Normalization by Using Local Extrema Classification," Proc. Eighth Int'l Workshop Pattern Recognition in Information Systems, pp. 164-172, 2008.
[13] Y. Bengio, "A Connectionist Approach to Speech Recognition," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 7, no. 4, pp. 647-667, 1993.
[14] H. Bourlard and N. Morgan, Connectionist Speech Recognition—A Hybrid Approach. Kluwer Academic, 1994.
[15] M.J. Castro and F. Casacuberta, "Hybrid Connectionist-Structural Acoustical Modeling in the ATROS System," Proc. Sixth European Conf. Speech Comm. and Technology, vol. 3, pp. 1299-1302, 1999.
[16] M.J. Castro and F. Casacuberta, "Committees of MLPs for Acoustic Modeling," Proc. Fifth Iberoam. Symp. Pattern Recognition, pp. 797-807, 2000.
[17] R. Gemellovo, F. Mana, and D. Albesano, "Hybrid HMM/Neural Network Based Speech Recognition in Loquendo ASR," http://www.loquendo.com/en/brochureSpeech_Recognition_ASR. pdf , 2008.
[18] Y. Bengio, Y. LeCun, C. Nohl, and C. Burges, "LeRec: A NN/HMM Hybrid for On-Line Handwriting Recognition," Neural Computation, vol. 7, no. 6, pp. 1289-1303, 1995.
[19] M. Schenkel, I. Guyon, and D. Henderson, "On-Line Cursive Script Recognition Using Time Delay Neural Networks and Hidden Markov Models," Machine Vision and Applications, vol. 8, no. 4, pp. 215-223, 1995.
[20] S. Jaeger, S. Manke, and A. Waibel, "Npen++: An On-Line Handwriting Recognition System," Proc. Seventh Int'l Workshop Frontiers in Handwriting Recognition, pp. 249-260, 2000.
[21] S. Marukatat, T. Artières, B. Dorizzi, and P. Gallinari, "Sentence Recognition through Hybrid Neuro-Markovian Modelling," Proc. Int'l Conf. Document Analysis and Recognition, pp. 731-735, 2001.
[22] É. Caillault and C. Viard-Gaudin, "Mixed Discriminant Training of Hybrid ANN/HMM Systems for Online Handwritten Word Recognition," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 21, no. 1, pp. 117-134, 2007.
[23] A. Graves, S. Fernandez, M. Liwicki, H. Bunke, and J. Schmidhuber, "Unconstrained Online Handwriting Recognition with Recurrent Neural Networks," Advances in Neural Information Processing Systems, vol. 20, pp. 577-584, MIT Press, 2008.
[24] S. Knerr and E. Augustin, "A Neural Network-Hidden Markov Model Hybrid for Cursive Word Recognition," Proc. 14th Int'l Conf. Pattern Recognition, vol. 2, pp. 1518-1520, 1998.
[25] A.W. Senior and A.J. Robinson, "An Off-Line Cursive Handwritten Recognition System," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 3, pp. 309-321, Mar. 1998.
[26] J.H. Kim, K.K. Kim, and C.Y. Suen, "An HMM-MLP Hybrid Model for Cursive Script Recognition," Pattern Analysis and Applications, vol. 3, pp. 314-324, 2000.
[27] C. Burges, O. Matan, Y. LeCun, J. Denker, L. Jackel, C. Stenard, C. Nohl, and J. Ben, "Shortest Path Segmentation: A Method for Training a Neural Network to Recognize Character Strings," Proc. Int'l Joint Conf. Neural Networks, vol. 3, pp. 165-172, 1992.
[28] C. Burges, J. Ben, J. Denker, Y. LeCun, and R. Nohl, "Off-Line Recognition of Handwritten Postal Words Using Neural Networks," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 7, no. 4, pp. 689-704, 1993.
[29] A. Koerich, Y. Leydier, R. Sabourin, and C. Suen, "A Hybrid Large Vocabulary Handwritten Word Recognition System Using Neural Networks with Hidden Markov Models," Proc. Eighth Int'l Workshop Frontiers in Handwriting Recognition, pp. 99-104, 2002.
[30] Y. Tay, M. Khalid, R. Yusof, and C. Viard-Gaudin, "Offline Cursive Handwriting Recognition System Based on Hybrid Markov Model and Neural Networks," Proc. IEEE Int'l Symp. Computational Intelligence in Robotics and Automation, pp. 1190-1195, July 2003.
[31] S. Marinai, M. Gori, and G. Soda, "Artificial Neural Networks for Document Analysis and Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 23-35, Jan. 2005.
[32] U.-V. Marti and H. Bunke, "The IAM-Database: An English Sentence Database for Offline Handwriting Recognition," Int'l J. Document Analysis and Recognition, vol. 5, no. 1, pp. 39-46, 2002.
[33] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, "A Novel Connectionist System for Unconstrained Handwriting Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 855-868, May 2009.
[34] J.L. Hidalgo, S. España, M.J. Castro, and J.A. Pérez, "Enhancement and Cleaning of Handwritten Data by Using Neural Networks," Proc. Second Iberian Conf. Pattern Recognition and Image Analysis, pp. 376-383, 2005.
[35] D.J. Burr, "A Normalizing Transform for Cursive Script Recognition," Proc. Sixth Int'l Conf. Pattern Recognition, pp. 1027-1030, 1982.
[36] R.M. Bozinovic and S.N. Srihari, "Off-Line Cursive Script Word Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, no. 1, pp. 68-83, Jan. 1989.
[37] A. Vinciarelli and J. Luettin, "A New Normalization Technique for Cursive Handwritten Words," Pattern Recognition Letters, vol. 22, no. 9, pp. 1043-1050, 2001.
[38] K.Y. Wong, R.G. Casey, and F.M. Wahl, "Document Analysis System," IBM J. Research and Development, vol. 26, no. 6, pp. 647-655, 1982.
[39] V. Romero, M. Pastor, A.H. Toselli, and E. Vidal, "Improving Handwritten Off-Line Text Slant Correction," Proc. Sixth IASTED Int'l Conf. Visualization, Imaging, and Image Processing, pp. 389-394, 2006.
[40] P. Simard, D. Steinkraus, and M. Agrawala, "Ink Normalization and Beautification," Proc. Eighth Int'l Conf. Document Analysis and Recognition, pp. 1182-1187, 2005.
[41] M. Pastor, A. Toselli, and E. Vidal, "Projection Profile Based Algorithm for Slant Removal," Proc. Int'l Conf. Image Analysis and Recognition, pp. 183-190, 2004.
[42] S. Uchida, E. Taira, and H. Sakoe, "Nonuniform Slant Correction Using Dynamic Programming," Proc. Sixth Int'l Conf. Document Analysis and Recognition, vol. 1, pp. 434-438, 2001.
[43] J. Schenk, J. Lenz, and G. Rigoll, "On-Line Recognition of Handwritten Whiteboard Notes: A Novel Approach for Script Line Identification and Normalization," Proc. 11th Int'l Workshop Frontiers in Handwriting Recognition, pp. 540-543, 2008.
[44] L. Rabiner and B.H. Huang, Fundamentals of Speech Recognition. Prentice-Hall, 1993.
[45] F. Jelinek, Statistical Methods for Speech Recognition. MIT Press, 1997.
[46] C.M. Bishop, Neural Networks for Pattern Recognition. Oxford Univ. Press, 1995.
[47] S. Johansson, E. Atwell, R. Garside, and G. Leech, The Tagged LOB Corpus: User's Manual. Norwegian Computing Centre for the Humanities, 1986.
[48] R. Bertolami and H. Bunke, "Ensemble Methods to Improve the Performance of an English Handwritten Text Line Recognizer," Proc. Conf. Arabic and Chinese Handwriting Recognition, pp. 265-277, 2008.
[49] R. Bertolami and H. Bunke, "Hidden Markov Models-Based Ensemble Methods for Offline Handwritten Text Line Recognition," Pattern Recognition, vol. 41, no. 11, pp. 3452-3460, 2008.
[50] W. Francis and H. Kucera, "Brown Corpus Manual, Manual of Information to Accompany a Standard Corpus of Present-Day Edited American English," technical report, Dept. of Linguistics, Brown Univ., 1979.
[51] L. Bauer, "Manual of Information to Accompany the Wellington Corpus of Written New Zealand English," technical report, Dept. of Linguistics, Victoria Univ., 1993.
[52] A. Stolcke, "SRILM: An Extensible Language Modeling Toolkit," Proc. Int'l Conf. Spoken Language Processing, pp. 901-904, 2002.
[53] J.M. Vilar, "Efficient Computation of Confidence Intervals for Word Error Rates," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 5101-5104, 2008.
[54] S.J. Young, P.C. Woodland, and W.J. Byrne, "HTK: Hidden Markov Model Toolkit V1.5," technical report, Cambridge Univ. Eng. Dept. Speech Group and Entropic Research Laboratories, Inc., 1993.
[55] S. España-Boquera, F. Zamora-Martínez, M.J. Castro-Bleda, and J. Gorbe-Moya, "Efficient BP Algorithms for General Feedforward Neural Networks," Proc. Second Int'l Work-Conf. Interplay between Natural and Artificial Computation, Part I, Bio-Inspired Modeling of Cognitive Tasks, pp. 327-336, 2007.
[56] S. España-Boquera, M.J. Castro-Bleda, F. Zamora-Martínez, and J. Gorbe-Moya, "Efficient Viterbi Algorithms for Lexical Tree Based Models," Proc. Int'l Conf. Advances in Non-Linear Speech Processing, pp. 179-187, 2007.
[57] F. Zamora-Martínez, M.J. Castro-Bleda, S. España-Boquera, and J. Gorbe-Moya, "Improving Isolated Handwritten Word Recognition Using a Specialized Classifier for Short Words," Current Topics in Artificial Intelligence, pp. 61-70, 2010.
[58] Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, "Greedy Layer-Wise Training of Deep Networks," Proc. Neural Information Processing Systems Conf., pp. 153-160, 2006.
[59] Y. Bengio, "Learning Deep Architectures for AI," Foundations and Trends in Machine Learning, vol. 2, no. 1, 2009.
16 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool