A novel algorithm for voice conversion is proposed in this paper. The mapping function of spectral vectors of the source and target speakers is calculated by the canonical correlation analysis (CCA) estimation based on Gaussian mixture models. Since the spectral envelope feature remains a majority of second order statistical information contained in speech after linear prediction (LPC) analysis, the CCA method is more suitable for spectral conversion than MMSE because CCA explicitly considers the variance of each component of the spectral vectors during conversion procedure. Both subjective and objective evaluations are conducted. The experimental results demonstrate that the proposed scheme can achieve better performance than the previous method which uses MMSE estimation criterion.