Today, high accuracy of character recognition is attainable using a neural network for problems with a relatively small number of categories. But for large categories, like Chinese characters, it is difficult to reach the neural network convergence because of the "local minima problem" and a large number of calculations. Studies are being done to solve the problem by splitting the neural network into some small modules. The effectiveness of the combination of learning vector quantization (LVQ) and back propagation (BP) has been reported. LVQ is used for rough classification and BP is used for fine recognition. It is difficult to obtain high accuracy for rough classification by LVQ itself. To deal with this problem, we propose hierarchical learning vector quantization (HLVQ). HLVQ divides categories in feature space hierarchically in the learning procedure. The adjacent feature spaces overlap each other near the borders. HLVQ possesses both classification speed and accuracy due to the hierarchical architecture and the overlapping technique. In the experiment using ETL9B, the largest database of handwritten characters in Japan, (includes 3036 categories, 607,200 samples), the effectiveness of HLVQ was verified.