Issue No. 02 - February (2012 vol. 34)
Markus Nussbaum-Thom , Comput. Sci. Dept., RWTH Aachen Univ., Aachen, Germany
Ralf Schlueter , Comput. Sci. Dept., RWTH Aachen Univ., Aachen, Germany
Hermann Ney , Comput. Sci. Dept., RWTH Aachen Univ., Aachen, Germany
In many tasks in pattern recognition, such as automatic speech recognition (ASR), optical character recognition (OCR), part-of-speech (POS) tagging, and other string recognition tasks, we are faced with a well-known inconsistency: The Bayes decision rule is usually used to minimize string (symbol sequence) error, whereas, in practice, we want to minimize symbol (word, character, tag, etc.) error. When comparing different recognition systems, we do indeed use symbol error rate as an evaluation measure. The topic of this work is to analyze the relation between string (i.e., 0-1) and symbol error (i.e., metric, integer valued) cost functions in the Bayes decision rule, for which fundamental analytic results are derived. Simple conditions are derived for which the Bayes decision rule with integer-valued metric cost function and with 0-1 cost gives the same decisions or leads to classes with limited cost. The corresponding conditions can be tested with complexity linear in the number of classes. The results obtained do not make any assumption w.r.t. the structure of the underlying distributions or the classification problem. Nevertheless, the general analytic results are analyzed via simulations of string recognition problems with Levenshtein (edit) distance cost function. The results support earlier findings that considerable improvements are to be expected when initial error rates are high.
pattern recognition, Bayes methods, string recognition, Bayes decision rule, pattern recognition, automatic speech recognition, ASR, optical character recognition, OCR, part-of-speech, POS, symbol sequence, error rate symbol, distance cost function, Cost function, Speech recognition, Error analysis, Measurement uncertainty, Statistical analysis, Bayesian methods, cost/loss function., Statistical pattern recognition, classifier design and evaluation, Bayes decision rule
Markus Nussbaum-Thom, Ralf Schlueter, Hermann Ney, "Does the Cost Function Matter in Bayes Decision Rule?", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 34, no. , pp. 292-301, February 2012, doi:10.1109/TPAMI.2011.163