A novel hidden Markov model (HMM) for recognition of handwritten Oriya numerals is proposed. The novelty lies in the fact that the HMM states are not determined a priori, but are determined automatically based on a database of handwritten numeral images. A handwritten numeral is assumed to be a string of several shape primitives. These are in fact the states of the proposed HMM and are found using certain mixture distributions. One HMM is constructed for each numeral. To classify an unknown numeral image, its class conditional probability for each HMM is computed. The classification scheme has been tested on a large handwritten Oriya numeral database developed recently. The classification accuracy is 95.89% and 90.50% for training and test sets respectively.