This paper investigates the intrinsic ability of Gabor representation and Support Vector Machines (SVM) in capturing discriminatory content for face verification task. The idea is to decompose a face image into different spatial frequencies (scales) and orientations where salient discriminant features may appear. Dimensionality reduction is adopted to create low dimensional feature vectors for more convenient processing. SVM is used to extract relevant information from this low dimensional training data in order to construct a robust client-specific classifier. This method has been tested with publicly available AT&T and BANCA datasets. In the BANCA experiments, it was observed that method consistently yields the lowest error rates in comparison with other methods for all seven test configurations. An equal error rate (EER) of 6.19% on the G configuration of BANCA dataset has been achieved.