Principle Component Analysis (PCA) and Multiple Discriminant Analysis (MDA) have long been used for the appearance-based hand posture recognition. In this paper, we propose a novel PCA/MDA scheme for hand posture recognition. Unlike other PCA/MDA schemes, the PCA layer acts as a crude classification. Since posture alone cannot provide sufficient discriminating information, each input pattern will be given a likelihood of being in the nodes of PCA layers, instead of a strict division. Based on the Expectation-Maximization (EM) algorithm, we introduce three methods to estimate the parameters for this crude classification during training. The experiments on a 110-sign vocabulary show a significant improvement compared with the global PCA/MDA.