Free style Chinese handwriting recognition continues to pose a challenge to researchers due to the variety of writing styles. To recognize handwritten characters in an online mode, Hidden Markov Model (HMM) has been naturally adopted to model the pen trajectory of a character and a de- cent recognition performance is achieved. In this study, we start from a maximum likelihood trained HMM model and focus on minimizing recognition errors at the radical (sub- character) level to optimize the recognition performance. A novel Minimum Radical Error discriminative training cri- terion is proposed, and compared with the discrimination at the character level, our new approach further reduces the character errors by 15.6% relatively (29.0% overall re- duction from the maximum likelihood baseline model) on a Chinese database.