Handwriting recognition systems based on hidden Markov models use commonly a vector quantizer to get the required symbol sequence. In order to get better recognition rates semi-continuous hidden Markov models are applied during the last years. Those recognizers need a soft vector quantizer which superimposes a statistical distribution for symbol generation. In general, Gaussian distributions are applied. A disadvantage of this technique is the assumption of a specific distribution. No proof can be given whether this presupposition holds in practice. Therefore, the application of a method which employs no model of a distribution may achieve some improvements. This paper presents the employment of a polynomial classifier as a replacement of a Gaussian classifier in our handwriting recognition system. The replacement improves the recognition rate significantly, as the results show.