A system for the detection of human faces and for the classification of hand postures in color images is presented. We first propose to apply a combination of a skin chrominance-based image segmentation with a color vector gradient-based edge detection [1 ] [2 ] to efficiently detect faces and hands. A statistical model for face detection based on invariant moments [3] [4 ] is then used to discriminate between faces and hands in the segmented images. A novel approach to hand posture recognition based on phase-only correlation [5 ] is finally applied to classify a subset of static hand postures of the Japanese Sign Language, each posture representing a given phoneme, and also to discriminate between hand postures and the image scene background. Experiments show that the additional use of the color gradient significantly improves the correct rate of face detection, and that the phase-only correlation filter yields a high rate of discrimination between different static hand postures as well as between hand postures and the scene background.