This paper presents a new online clustering algorithm in order to improve plasticity and stability in handwritten character recognition systems. Our clustering algorithm is able to automatically determine the optimal number of clusters in the input data. An incremental learning technique similar to Adaptive Resonance Theory (ART) is used to determine the best cluster for new data. Our technique also allows the previously learned clusters to be merged whenever the newly arrived data points push their centers close together. We also developed new features and similarity measures in order to describe and compare the shapes of handwritten digits to be used in our clustering algorithm. Results of our algorithm on clustering the shapes of the handwritten numerals from the CENPARMI isolated digit database are shown. Our method can incrementally learn new handwriting styles of digits, without forgetting the previous ones, therefore it can improve plasticity and stability.