In this paper, we describe an indoor surveillance system that enables the control of a binocular active vision system by a remote operator wearing a head mounted display. Visual feedback to the head mounted display is obtained using the images captured by the active vision system. To maximize the 3D perception of the operator the system is able to adjust the vergence angle of the active vision system in order to maintain fixation on the objects in the center of the image. A wide-angle lens static camera located on the surveillance area is also used to enable the detection and tracking of intruders. In case of an intrusion, a map with the location of the detected targets is displayed on top of the images being shown on the head mounted display. This information allows the remote operator to redirect his attention to where the target is.