We describe a real time visual surveillance system for detecting and tracking people in an outdoor environment with an active camera which has zoom, tilt and pan control. It operates on monocular grayscale video imagery, or on video imagery from an infrared camera. The system learns and models background scene statistically to detect foreground objects, even when the background is not completely stationary (e.g. motion of tree branches) and distinguishes people from other objects (e.g. cars) using shape and periodic motion cues. It can track people with an active camera in a large field of regard when people's movement take them outside of the camera's current field of view, and control zoom, pan, and tilt of the camera to achieve sufficient angular resolution to recognize human actions. Experimental results demonstrate robustness and real-time performance of the algorithm.