This paper analyses the application of multistage classifiers based on the k-NN rule to the automatic classification of handwritten digits. The discriminating capacity of a k-NN classifier increases as the size and dimensionality of the reference pattern set (RPS) increases. This supposes a problem for k -NN classifiers in real applications: the high computational cost required. In order to accelerate the process of calculating the distance to each pattern of the RPS, some authors propose the use of condensing techniques. These methods try to reduce the size of the RPS without losing classification power. Our alternative proposal is based on hierarchical classifiers with rejection techniques and incremental learning that reduce the computational cost of the classifier. We have used 270,000 digits (160,000 digits for training and 110,000 for the test) of the NIST Special Data Bases 19 and 3 (SD19 and SD3) as experimental data sets. The best non-hierarchical classifier achieves a hit rate of 99.50%. The hierarchical classifier achieves the same hit ratio, but with 24.5 times lower computational cost than best non-hierarchical classifier found in our experimentation and 6 times lower than Hart?s Algorithm.