The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - December (2009 vol.31)
pp: 2168-2178
Paulo Rodrigo Cavalin , Génie de la Production Automatisée (GPA), Montréal
Robert Sabourin , École de Technologie Supérieure, Montréal
Albert Hung-Ren Ko , University of Toronto, Toronto
ABSTRACT
Hidden Markov Models (HMMs) have been shown to be useful in handwritten pattern recognition. However, owing to their fundamental structure, they have little resistance to unexpected noise among observation sequences. In other words, unexpected noise in a sequence might “ break” the normal transmission of states for this sequence, making it unrecognizable to trained models. To resolve this problem, we propose a leave-one-out-training strategy, which will make the models more robust. We also propose a leave-one-out-testing method, which will compensate for some of the negative effects of this noise. The latter is actually an example of a system with a single classifier and multiple classifications. Compared with the 98.00 percent accuracy of the benchmark HMMs, the new system achieves a 98.88 percent accuracy rate on handwritten digits.
INDEX TERMS
Hidden Markov Models, ensemble of classifiers, sequence, noise, leave one out, pattern recognition.
CITATION
Paulo Rodrigo Cavalin, Robert Sabourin, Albert Hung-Ren Ko, "Leave-One-Out-Training and Leave-One-Out-Testing Hidden Markov Models for a Handwritten Numeral Recognizer: The Implications of a Single Classifier and Multiple Classifications", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.31, no. 12, pp. 2168-2178, December 2009, doi:10.1109/TPAMI.2008.254
REFERENCES
[1] N. Arica and F.T.Y. Vural, “A Shape Descriptor Based on Circular Hidden Markov Model,” Proc. 15th Int'l Conf. Pattern Recognition, 2000.
[2] S. Bandyopadhyay and U. Maulik, “Non-Parametric Genetic Clustering: Comparison of Validity Indices,” IEEE Trans. Systems, Man, and Cybernetics, Part C, vol. 31, no. 1, pp. 120-125, Feb. 2001.
[3] Y. Bengio, “Markovian Models for Sequential Data,” Neural Computing Surveys, vol. 2, pp. 129-162, 1999.
[4] A. BrittoJr., “A Two-Stage HMM-Based Method for Recognizing Handwritten Numeral Strings,” PhD thesis, Pontifical Catholic Univ. of Paraná, 2001.
[5] A. Britto, R. Sabourin, F. Bortolozzi, and C.Y. Suen, “Recognition of Handwritten Numeral Strings Using a Two-Stage HMM-Based Method,” Int'l J. Document Analysis and Recognition, vol. 5, no. 2, pp. 102-117, 2003.
[6] G. Brown, J. Wyatt, R. Harris, and X. Yao, “Diversity Creation Methods: A Survey and Categorisation,” Int'l J. Information Fusion, vol. 6, no. 1, pp. 5-20, 2005.
[7] T.G. Dietterich, “Machine Learning for Sequential Data: A Review,” Structural, Syntactic, and Statistical Pattern Recognition, pp. 15-30, Springer-Verlag, 2002.
[8] D. Eppstein, “Fast Hierarchical Clustering and Other Applications of Dynamic Closest Pairs,” Proc. Ninth ACM-SIAM Symp. Discrete Algorithms, pp. 619-628, 1998.
[9] S. Gunter and H. Bunke, “Creation of Classifier Ensembles for Handwritten Word Recognition Using Feature Selection Algorithms,” Proc. Eighth Int'l Workshop Frontiers in Handwriting Recognition, pp. 183-188, 2002.
[10] S. Gunter and H. Bunke, “Generating Classifier Ensembles from Multiple Prototypes and Its Application to Handwriting Recognition,” Proc. the Third Int'l Workshop Multiple Classifier Systems, pp. 179-188, 2002.
[11] S. Gunter and H. Bunke, “A New Combination Scheme for HMM-Based Classifiers and Its Application to Handwriting Recognition,” Proc. 16th Int'l Conf. Pattern Recognition, vol. 2, pp. 332-337, 2002.
[12] S. Gunter and H. Bunke, “Ensembles of Classifiers for Handwritten Word Recognition,” Int'l J. Document Analysis and Recognition, vol. 5, no. 4, pp. 224-232, 2003.
[13] S. Gunter and H. Bunke, “New Boosting Algorithms for Classification Problems with Large Number of Classes Applied to a Handwritten Word Recognition Task,” Proc. Fourth Int'l Workshop Multiple Classifier Systems, pp. 326-335, 2003.
[14] S. Gunter and H. Bunke, “Offline Cursive Handwriting Recognition—On the Influence of Training Set and Vocabulary Size in Multiple Classifier Systems,” Proc. 11th Conf. Int'l Graphonomics Soc., 2003.
[15] M. Halkidi, Y. Batistakis, and M. Vazirgiannis, “On Clustering Validation Techniques,” J. Intelligent Information Systems, vol. 17, pp. 2-3, 2001.
[16] M. Halkidi, Y. Batistakis, and M. Vazirgiannis, “Clustering Validity Checking Methods: Part II,” ACM SIGMOD Record, vol. 3, no. 3, pp. 19-27, 2002.
[17] A.H.R. Ko, R. Sabourin, and A. BrittoJr., “A New HMM-Based Ensemble Generation Method for Character Recognition,” Proc. Int'l Workshop Multiple Classifier Systems, pp. 52-61, 2007.
[18] A.H.R. Ko, R. Sabourin, and A. BrittoJr., “Ensemble of HMM Classifiers Based on the Clustering Validity Index for a Handwritten Numeral Recognizer,” Pattern Analysis and Applications J., 2008, doi 10.1007/s10044-007-0094-6.
[19] L.I. Kuncheva, M. Skurichina, and R.P.W. Duin, “An Experimental Study on Diversity for Bagging and Boosting with Linear Classifiers,” Int'l J. Information Fusion, vol. 3, no. 2, pp. 245-258, 2002.
[20] J. Milgram, M. Cheriet, and R. Sabourin, “Estimating Accurate Multi-Class Probabilities with Support Vector Machines,” Proc. Int'l Joint Conf. Neural Networks, pp. 1906-1911, 2005.
[21] L.E.S. Oliveira, R. Sabourin, F. Bortolozzi, and C.Y. Suen, “Automatic Recognition of Handwritten Numerical Strings: A Recognition and Verification Strategy,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 11, pp. 1438-1454, Nov. 2002.
[22] L.E.S. Oliveira and R. Sabourin, “Support Vector Machines for Handwritten Numeral String Recognition,” Proc. Ninth Int'l Workshop Frontiers in Handwriting Recognition, pp. 39-44, Oct. 2004.
[23] M.K. Pakhira, S. Bandyopadhyay, and U. Maulik, “Validity Index for Crisp and Fuzzy Clusters,” Pattern Recognition, vol. 37, no. 3, pp. 487-501, 2004.
[24] L.R. Rabiner, “A Tutorial on Hidden Markov Models and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-286, 1989.
[25] L.R. Rabiner and B.H. Juang, Fundamentals of Speech Recognition. Prentice-Hall, 1993.
[26] P. Radtke, T. Wong, and R. Sabourin, “An Evaluation of Over-Fit Control Strategies for Multi-Objective Evolutionary Optimization,” Proc. IEEE World Congress on Computational Intelligence—Int'l Joint Conf. Neural Networks, 2006.
[27] D. Ruta and B. Gabrys, “Classifier Selection for Majority Voting,” Int'l J. Information Fusion, pp. 63-81, 2005.
[28] P. Smyth, D. Heckerman, and M.I. Jordan, “Probabilistic Independence Networks for Hidden Markov Probability Models,” Neural Computation, vol. 9, pp. 227-269, 1997.
[29] X. Wang, “Durationally Constrained Training of HMM without Explicit State Durational,” Proc. Inst. of Phonetic Sciences, vol. 18, pp. 111-130, 1994.
[30] X.L. Xie and G. Beni, “A Validity Measure for Fuzzy Clustering,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 8, pp. 841-847, Aug. 1991.
12 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool