Virtual avatars in many applications are constructed manually or by a single speech-driven model which needs a lot of training data and long training time. It’s an essential problem to build up a user-dependent model more efficiently. In this paper, a new adaptation method, called the partial linear regression (PLR), is proposed and adopted in an audio-driven talking head application. This method allows users to adapt the partial parameters from the available adaptive data while keeping the others unchanged. In our experiments, the PLR algorithm can retrench the hours of time spent on retraining a new user dependent model, and adjust the user-independent model to a more personalized one. The animated results with adapted models were 36% closer to the user-dependent model than using the pre-trained user-independent model.