In this paper, we present an automatic mouth contour and state estimation system. An efficient mouth contour extraction algorithm is proposed under the framework of Active Shape Model (ASM). Considering large mouth shape variations, we propose a textureconstrained shape prediction method for initialization. To improve accuracy and robustness of classical ASM, we use classifiers trained by Real AdaBoost to characterize the local texture model. This model is proved to have much stronger discriminative power than Gaussian model of classical ASM. After extracting the mouth contour, the mouth is classified into one of 4 typical states by Support Vector Machine (SVM) based on the shape parameter. Experiments over a large set show that extracted mouth contours have achieved good accuracy, with an average 89.5% acceptable rate, and the mouth state estimation reaches an average 93% correct rate. This automatic system reaches a speed of about 10 frames per second on a Pentium-IV 1.7GHz PC, which may have potential applications in visual speech recognition etc.