It is difficult to recognize sign language in different viewpoint. The HMM method is hindered by the difficulty of extracting view invariant features. The general template matching methods have a strong constraint such as accurate alignment between the template sign and the test sign. In the paper, we introduce a novel approach for viewpoint invariant sign language recognition. The proposed approach requires no view invariant features, low training and no alignment. Its basic idea is to consider a sign as a series of tiny hand motions and utilize the HOMOGRAPHY of tiny hand motions. Using the word of "homography", we mean that there are the same tiny hand motions as well as their appearance order in different performances of the same sign. The experimental results demonstrate the efficiency of the proposed method.