Gaussian mixture models (GMMs) have been successfully applied to the classier for speaker modeling in speaker identification. However, there are still problems to solve, such as the clustering methods. Conditional K-Means Algorithm utilizes Euclidean distance taking all data distribution as sphericity, which is not the distribution of the actual data. In this paper we present a new method to make use of covariance information to direct the clustering of GMMs, namely covariance-tied clustering. This method is consisted of two parts: obtaining the covariance matrices using data sharing technique based on binary tree and making use of the covariance matrices to direct clustering. The experiments results prove that this method leads to worthwhile reductions of error rates in speaker identification. Much remains to be done to explore fully the covariance information.