This paper presents a combined approach for online handwriting symbols recognition. The basic idea of this approach is to employ a set of left-right HMMs as a feature extractor to produce HMM features, and combine them with global features into a new feature vector as input, and then use SVM as a classifier to finally identify unknown symbols. The new feature vector consists of the global features and several pairs of maximum probabilities with their associated different model labels. A recogniser based on this method inherits the practical and dynamical modeling abilities from HMM, and robust discriminating ability from SVM for classification tasks. This technique also reduces the dimensions of feature vectors significantly and solves the speed and size problem when using only SVM. The experimental results show that this combined hybrid approach outperforms several classifiers reported in recent researches, and could achieve recognition rates of 97.48%, 91.99% and 91.74% for digits and upper/lower case characters respectively on the UNIPEN database benchmarks.