The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2013 vol.35)
pp: 2413-2426
Xiang-Dong Zhou , Beijing Key Lab. of Human-Comput. Interaction, Inst. of Software, Beijing, China
Da-Han Wang , Nat. Lab. of Pattern Recognition (NLPR), Inst. of Autom., Beijing, China
Feng Tian , State Key Lab. of Comput. Sci., Inst. of Software, Beijing, China
Cheng-Lin Liu , Nat. Lab. of Pattern Recognition (NLPR), Inst. of Autom., Beijing, China
M. Nakagawa , Dept. of Comput. & Inf. Sci., Tokyo Univ. of Agric. & Technol., Koganei, Japan
ABSTRACT
This paper proposes a method for handwritten Chinese/Japanese text (character string) recognition based on semi-Markov conditional random fields (semi-CRFs). The high-order semi-CRF model is defined on a lattice containing all possible segmentation-recognition hypotheses of a string to elegantly fuse the scores of candidate character recognition and the compatibilities of geometric and linguistic contexts by representing them in the feature functions. Based on given models of character recognition and compatibilities, the fusion parameters are optimized by minimizing the negative log-likelihood loss with a margin term on a training string sample set. A forward-backward lattice pruning algorithm is proposed to reduce the computation in training when trigram language models are used, and beam search techniques are investigated to accelerate the decoding speed. We evaluate the performance of the proposed method on unconstrained online handwritten text lines of three databases. On the test sets of databases CASIA-OLHWDB (Chinese) and TUAT Kondate (Japanese), the character level correct rates are 95.20 and 95.44 percent, and the accurate rates are 94.54 and 94.55 percent, respectively. On the test set (online handwritten texts) of ICDAR 2011 Chinese handwriting recognition competition, the proposed method outperforms the best system in competition.
INDEX TERMS
Lattices, Character recognition, Training, Handwriting recognition, Context, Text recognition, Context modeling,beam search, Character string recognition, semi-Markov conditional random field, lattice pruning
CITATION
Xiang-Dong Zhou, Da-Han Wang, Feng Tian, Cheng-Lin Liu, M. Nakagawa, "Handwritten Chinese/Japanese Text Recognition Using Semi-Markov Conditional Random Fields", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 10, pp. 2413-2426, Oct. 2013, doi:10.1109/TPAMI.2013.49
REFERENCES
[1] C.L. Liu, S. Jaeger, and M. Nakagawa, "Online Recognition of Chinese Characters: The State-of-the-Art," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 2, pp. 198-213, Feb. 2004.
[2] Y. Hotta, H. Takebe, M. Suwa, and S. Naoi, "Accuracy Improvement for Handwritten Japanese Word Recognition by Combination of Character and Word Recognizer," Proc. Eighth Int'l Conf. Document Analysis and Recognition, pp. 685-689, 2005.
[3] Z.B Yao, X.Q Ding, and C.S Liu, "On-Line Handwritten Chinese Word Recognition Based on Lexicon," Proc. 18th Int'l Conf. Pattern Recognition, pp. 320-323, 2006.
[4] T. Long and L.W. Jin, "A Novel Orientation Free Method for Online Unconstrained Cursive Handwritten Chinese Word Recognition," Proc. 19th Int'l Conf. Pattern Recognition, pp. 1-4, 2008.
[5] B.L. Zhu and M. Nakagawa, "Trie-Lexicon-Driven Recognition for On-Line Handwritten Japanese Disease Names Using a Time-Synchronous Method," Proc. 11th Int'l Conf. Document Analysis and Recognition, pp. 1130-1134, 2011.
[6] H.S. Tang, E. Augustin, C.Y. Suen, O. Baret, and M. Cheriet, "Spiral Recognition Methodology and Its Application for Recognition of Chinese Bank Checks," Proc. Ninth Int'l Workshop Frontiers in Handwriting Recognition, pp. 263-268, 2004.
[7] C.L. Liu, M. Koga, and H. Fujisawa, "Lexicon-Driven Segmentation and Recognition of Handwritten Character Strings for Japanese Address Reading," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 11, pp. 1425-1437, Nov. 2002.
[8] C.H. Wang, Y. Hotta, M. Suwa, and S. Naoi, "Handwritten Chinese Address Recognition," Proc. Ninth Int'l Workshop Frontiers in Handwriting Recognition, pp. 539-544, 2004.
[9] Q. Fu, X.Q. Ding, T. Liu, Y. Jiang, and Z. Ren, "A Novel Segmentation and Recognition Algorithm for Chinese Handwritten Address Character Strings," Proc. 18th Int'l Conf. Pattern Recognition, pp. 974-977, 2006.
[10] C.L. Liu, F. Yin, Q.F. Wang, and D.H. Wang, "ICDAR 2011 Chinese Handwriting Recognition Competition," Proc. 11th Int'l Conf. Document Analysis and Recognition, pp. 1464-1469, 2011.
[11] Q.F. Wang, F. Yin, and C.L. Liu, "Handwritten Chinese Text Recognition by Integrating Multiple Contexts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 8, pp. 1469-1481, Aug. 2012.
[12] B. Zhu, X.D. Zhou, C.L. Liu, and M. Nakagawa, "A Robust Model for On-Line Handwritten Japanese Text Recognition," Int'l J. Document Analysis and Recognition, vol. 13, no. 2, pp. 121-131, 2010.
[13] X.D. Zhou, C.L. Liu, and M. Nakagawa, "Online Handwritten Japanese Character String Recognition Using Conditional Random Fields," Proc. 10th Int'l Conf. Document Analysis and Recognition, pp. 521-525, 2009.
[14] M. Cheriet, N. Kharma, C.L. Liu, and C.Y. Suen, Character Recognition Systems: A Guide for Students and Practitioners. John Wiley & Sons, Inc., 2007.
[15] S. Senda and K. Yamada, "A Maximum-Likelihood Approach to Segmentation-Based Recognition of Unconstrained Handwriting Text," Proc. Sixth Int'l Conf. Document Analysis and Recognition, pp. 184-188, 2001.
[16] M. Nakagawa, B. Zhu, and M. Onuma, "A Model of On-Line Handwritten Japanese Text Recognition Free from Line Direction and Writing Format Constraints," IEICE Trans. Information and Systems, vol. E88-D, no. 8, pp. 1815-1822, 2005.
[17] X.D. Zhou, J.L. Yu, C.L. Liu, T. Nagasaki, and K. Marukawa, "Online Handwritten Japanese Character String Recognition Incorporating Geometric Context," Proc. Ninth Int'l Conf. Document Analysis and Recognition, pp. 48-52, 2007.
[18] Q.F. Wang, F. Yin, and C.L. Liu, "Integrating Language Model in Handwritten Chinese Text Recognition," Proc. 10th Int'l Conf. Document Analysis and Recognition, pp. 1036-1040, 2009.
[19] N.X. Li and L.W. Jin, "A Bayesian-Based Method of Unconstrained Handwritten Offline Chinese Text Line Recognition," Int'l J. Document Analysis and Recognition, vol. 16, no. 1, pp. 17-31, Mar. 2013.
[20] S. Sarawagi and W. Cohen, "Semi-Markov Conditional Random Fields for Information Extraction," Neural Information Processing Systems, vol. 17, pp. 1185-1192, 2005.
[21] G. Heigold, T. Deselaers, R. Schlüter, and H. Ney, "Modified MMI/MPE: A Direct Evaluation of the Margin in Speech Recognition," Proc. 25th Int'l Conf. Machine Learning, pp. 384-391, 2008.
[22] G. Heigold, P. Dreuw, S. Hahn, R. Schlüter, and H. Ney, "Margin-Based Discriminative Training for String Recognition," IEEE J. Selected Topics in Signal Processing—Statistical Learning Methods for Speech and Language Processing, vol. 4, no. 6, pp. 917-925, Dec. 2010.
[23] D. Povey, D. Kanevsky, B. Kingsbury, B. Ramabhadran, G. Saon, and K. Visweswariah, "Boosted MMI for Model and Feature-Space Discriminative Training," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 4057-4060, 2008.
[24] L.Y. Tseng and R.C. Chen, "Segmentation Handwritten Chinese Characters Based on Heuristic Merging of Stroke Bounding Boxes and Dynamic Programming," Pattern Recognition Letters, vol. 19, no 10, pp. 963-973, 1998.
[25] Y. Lu, C.L. Tan, P.F. Shi, and K.H. Zhang, "Segmentation of Handwritten Chinese Characters from Destination Addresses of Mail Pieces," Int'l J. Pattern Recognition and Artificial Intelligence, vol. 16, no. 1, pp. 85-96, 2002.
[26] S. Zhao, Z. Chi, P. Shi, and H. Yan, "Two-Stage Segmentation of Unconstrained Handwritten Chinese Characters," Pattern Recognition, vol. 36, no 1, pp. 145-156, 2003.
[27] X. Wei, S. Ma, and Y. Jin, "Segmentation of Connected Chinese Characters Based on Genetic Algorithm," Proc. Eighth Int'l Conf. Document Analysis and Recognition, pp. 645-649, 2005.
[28] Z. Liang and P. Shi, "A Metasynthetic Approach for Segmenting Handwritten Chinese Character Strings," Pattern Recognition Letters, vol. 26, no. 10, pp. 1498-1511, 2005.
[29] C.L. Liu, H. Sako, and H. Fujisawa, "Effects of Classifier Structures and Training Regimes on Integrated Segmentation and Recognition of Handwritten Numeral Strings," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 11, pp. 1395-1407, Nov. 2004.
[30] T.H. Su, T.W. Zhang, D.J. Guan, and H.J. Huang, "Off-Line Recognition of Realistic Chinese Handwriting Using Segmentation-Free Strategy," Pattern Recognition, vol. 42, no. 1, pp. 167-182, 2009.
[31] Z.W. Jiang, X.Q. Ding, C.S. Liu, and Y.W. Wang, "A Novel Short Merged Off-Line Handwritten Chinese Character String Segmentation Algorithm Using Hidden Markov Model," Proc. 11th Int'l Conf. Document Analysis and Recognition, pp. 668-672, 2011.
[32] H. Murase, "Online Recognition of Free-Format Japanese Handwritings," Proc. Ninth Int'l Conf. Pattern Recognition, pp. 1143-1147, 1988.
[33] X. Gao, P.M. Lallican, and C. Viard-Gaudin, "A Two-Stage Online Handwritten Chinese Character Segmentation Algorithm Based on Dynamic Programming," Proc. Eighth Int'l Conf. Document Analysis and Recognition, pp. 735-739, 2005.
[34] S. Tulyakov and V. Govindaraju, "Probabilistic Model for Segmentation Based Word Recognition with Lexicon," Proc. Sixth Int'l Conf. Document Analysis and Recognition, pp. 164-167, 2001.
[35] J. Lafferty, A. McCallum, and F. Pereira, "Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data," Proc. 18th Int'l Conf. Machine Learning, pp. 282-289, 2001.
[36] T.M.T. Do and T. Artières, "Conditional Random Fields for Online Handwriting Recognition," Proc. 10th Int'l Workshop Frontiers in Handwriting Recognition, pp. 197-202, 2006.
[37] S. Feng, R. Manmatha, and A. McCallum, "Exploring the Use of Conditional Random Field Models and HMMs for Historical Handwritten Document Recognition," Proc. Second Int'l Conf. Document Image Analysis for Libraries, pp. 30-37, 2006.
[38] F. Kimura, K. Takashina, S. Tsuruoka, and Y. Miyake, "Modified Quadratic Discriminant Functions and Its Application to Chinese Character Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp. 149-153, Jan. 1987.
[39] A. Graves, M. Liwicki, S. Fernández, R. Bertolami, H. Bunke, and J. Schmidhuber, "A Novel Connectionist System for Unconstrained Handwriting Recognition," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 5, pp. 855-868, May 2009.
[40] S. Shetty, H. Srinivasan, M. Beal, and S.N. Srihari, "Segmentation and Labeling of Documents Using Conditional Random Fields," Proc. SPIE Document Recognition and Retrieval XIV, pp. 6500U-1-11, 2007.
[41] C. Pal, C. Sutton, and A. McCallum, "Sparse Forward-Backward Using Minimum Divergence Beams for Fast Training of Conditional Random Fields," Proc. Int'l Conf. Acoustics, Speech, and Signal Processing, pp. v581-v584, 2006.
[42] T. Cohn, "Efficient Inference in Large Conditional Random Fields," Proc. 17th European Conf. Machine Learning, pp. 606-613, 2006.
[43] M. Jeong, C.Y. Lin, and G.G. Lee, "Efficient Inference of CRFs for Large-Scale Natural Language Data," Proc. Conf. Short Papers ACL-IJCNLP, pp. 281-284, 2009.
[44] D. Okanohara, Y. Miyao, Y. Tsuruoka, and J. Tsujii, "Improving the Scalability of Semi-Markov Conditional Random Fields for Named Entity Recognition," Proc. 21st Int'l Conf. Computational Linguistics and 44th Ann. Meeting of the Assoc. for Computational Linguistics, pp. 465-472, 2006.
[45] N. Ye, W.S. Lee, H.L. Chieu, and D. Wu, "Conditional Random Fields with High-Order Features for Sequence Labeling," Proc. 22nd Ann. Conf. Neural Information Processing Systems, pp. 1393-1400, 2009.
[46] X. Qian, X. Jiang, Q. Zhang, X. Huang, and L. Wu, "Sparse Higher Order Conditional Random Fields for Improved Sequence Labeling," Proc. 26th Ann. Int'l Conf. Machine Learning, pp. 849-856, 2009.
[47] V.C. Nguyen, N. Ye, W.S. Lee, and H.L. Chieu, "Semi-Markov Conditional Random Field with High-Order Features," Proc. 28th Int'l Conf. Machine Learning, 2011.
[48] D. Yu and L. Deng, "Large-Margin Discriminative Training of Hidden Markov Models for Speech Recognition," Proc. First Int'l Conf. Semantic Computing, pp. 429-438, 2007.
[49] T.M.T. Do and T. Artières, "Maximum Margin Training of Gaussian HMMs for Handwriting Recognition," Proc. 10th Int'l Conf. Document Analysis and Recognition, pp. 976-980, 2009.
[50] L.R. Bahl, P.F. Brown, P.V. DeSouza, and R.L. Mercer, "Maximum Mutual Information Estimation of Hidden Markov Model Parameters for Speech Recognition," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 49-52, 1986.
[51] B.H. Juang and S. Katagiri, "Discriminative Learning for Minimum Error Classification," IEEE Trans. Signal Processing, vol. 40, no 12, pp. 3043-3054, Dec. 1992.
[52] D. Povey, "Discriminative Training for Large Vocabulary Speech Recognition," PhD dissertation, Cambridge Univ., 2003.
[53] M.Y. Kim, "Large Margin Cost-Sensitive Learning of Conditional Random Fields," Pattern Recognition, vol. 43, no. 10, pp. 3683-3692, 2010.
[54] A. Sixtus and S. Ortmanns, "High Quality Word Graphs Using Forward-Backward Pruning," Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, vol. 2, pp. 593-596, 1999.
[55] C.L. Liu and X.D. Zhou, "Online Japanese Character Recognition Using Trajectory-Based Normalization and Direction Feature Extraction," Proc. 10th Int'l Workshop Frontiers in Handwriting Recognition, pp. 217-222, 2006.
[56] C.L. Liu, "Classifier Combination Based on Confidence Transformation," Pattern Recognition, vol. 38, no. 1, pp. 11-28, 2005.
[57] C. Sutton and A. McCallum, "An Introduction to Conditional Random Fields for Relational Learning," Introduction to Statistical Relational Learning, L. Getoor and B. Taskar, eds., MIT Press, 2006.
[58] A. Vinciarelli, S. Bengio, and H. Bunke, "Offline Recognition of Unconstrained Handwritten Texts Using HMMs and Statistical Language Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no 6, pp. 709-720, June 2004.
[59] A. Stolcke, "SRILM—An Extensible Language Modeling Toolkit," Proc. Seventh Int'l Conf. Spoken Language Processing, pp. 901-904, 2002.
[60] C.L. Liu, F. Yin, D.H. Wang, and Q.F. Wang, "CASIA Online and Offline Chinese Handwriting Databases," Proc. 11th Int'l Conf. Document Analysis and Recognition, pp. 37-41, 2011.
[61] K. Matsumoto, T. Fukushima, and M. Nakagawa, "Collection and Analysis of On-Line Handwritten Japanese Character Patterns," Proc. Sixth Int'l Conf. Document Analysis and Recognition, pp. 496-500, 2001.
[62] S. Ortmanns, H. Ney, and X. Aubert, "A Word Graph Algorithm for Large Vocabulary Continuous Speech Recognition," Computer Speech Language, vol. 11, pp. 43-72, 1997.
[63] C.L. Liu, R. Mine, and M. Koga, "Building Compact Classifier for Large Character Set Recognition Using Discriminative Feature Extraction," Proc. Eight Int'l Conf. Document Analysis and Recognition, pp. 846-850, 2005.
[64] C.L. Liu, "High Accuracy Handwritten Chinese Character Recognition Using Quadratic Classifiers with Discriminative Feature Extraction," Proc. 18th Int'l Conf. Pattern Recognition, pp. 942-945, 2006.
63 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool