Support vector machine (SVM) provides accurate classification but suffers from a large amount of computation. In this paper we propose here an incremental procedure for Growing Support Vector Classifiers, which serves to avoid a priori architecture estimation or the application of a pruning mechanism after SVM training. The proposed growing approach also opens up new possibilities for dealing with multi-kernel machines, and automatic selection of hyperparameters. At last, the performance of the proposed algorithm and its extensions is evaluated by an experiment.