We propose an original sequential learning algorithm, SBA, that enables the SVM to efficiently learn from only a small subset of the input data set. The principle is based on sequentially adding convex hull points of the binary classes to a small subset. The SVM is trained on the current training pool and its result is used to find the data which is wrongly classsified and furthest away from the current optimal hyperplane. This point is added to the training pool and the SVM is retrained on it. The iteration stops when no more such points are found. A formal proof of strict convergence is provided and we derive a geometric bound on the training time. It will be explained how SBA can be extended to handle nonlinearly and non-separable class distributions. Experimental trials on some well known data sets verify the speed advantage of our method coupled to any SVM over that of that SVM used and the core vector machine.