A real-time three-dimensional position and orientation tracking system is proposed for use with wearable augmented-reality systems. The system combines infrared markers with a head-mounted stereo camera to detect the user's position, and an orientation sensor to measure the orientation of the user's head. An extended Kalman filter is employed to reduce triangulation and orientation error by integrating the signals acquired by multiple sensors. The system is evaluated through a series of experiments, and a navigation system implemented using the proposed scheme is present. The accuracy of the system is shown to be sufficient to allow annotations and virtual objects to be overlaid on real scenes via a head-mounted display without confusion.