In this paper, we present an approach to capture markerless human motion and recognize human poses. Different body parts such as the torso and the hands are segmented from the whole body and tracked over time. A 2D model is used for the torso detection and tracking, while a skin color model is utilized for the hands tracking. Moreover, 3D location of these body parts are calculated and further used for pose recognition. By transferring the 2D and 3D coordinates of the torso and both hands into normalized feature space, simple classifiers, such as the nearest mean classifier, are sufficient for recognizing predefined key poses. The experimental results show that the proposed approach can effectively detect and track the torso and both hands in video sequences. Meanwhile, the extracted feature points are used for pose recognition and give good classification results of the multi-class problem. The implementation of the proposed approach is simple, easy to realize, and suitable for real gaming applications.