We propose a multi-camera system that can track multiple human faces and hands as well as focus on the face and hand gestures for recognition. Our current system consists of four cameras. Two fixed cameras are used as a stereo system to estimate face and hand positions. The stereo camera detects faces and hands by a standard skin color method we proposed. The distances of the targets are then estimated. Next, to track multiple targets, we evaluate the positions and sizes of targets between consecutive frames. The other two cameras perform tracking of such targets as faces and hands. The tracking cameras can be controlled in pan tilt angles and zoom ratio by computer. Our system selects the target for recognition from among the unknown targets in sequence. If a target does not have the appropriate size for recognition, the tracking cameras acquire its zoomed image. Since our system has two tracking cameras, it can track two targets at the same time. To recognize faces and hand gestures, we propose four directional features by using linear discriminant analysis. Using our system, we experimented on human position estimation, multiple face tracking, and face and hand gesture recognition. These experiments showed that our system could estimate a human position with the stereo camera and track multiple targets by using targets positions and sizes even if the persons overlapped with each other. In addition, our system could recognize faces and hand gestures by using the four directional features.