This paper proposes a new combination approach between Gaussian Mixture Model (GMM) and Support Vector Machine (SVM) by feature extraction based on adapted GMM for SVM in text-independent speaker verification. Because of excellent scalability, adapted GMM was used to extract a small quantity of typical feature vectors from large numbers of speech data for SVM speaker verification. Using this new combination approach, our speaker verification system performed significantly better than the current state-of-the-art GMM-UBM system on the NIST'04 1side-1side database.