Applying advanced video technology to understand (human) activity and intent, including the interaction of multiple people and objects, is becoming increasingly important, especially for intelligent video surveillance. Recently, technical interest in video surveillance has moved from lowlevel processing modules, such as motion detection and motion tracking, to activity awareness and more complex scene understanding. This paper presents an integrated video surveillance system at Honeywell labs, which detects predefined activities with improved robustness. Also, we present the ?new activity? detection (pattern discovery), which can automatically capture new activities, and present the newly detected activities to the operator who checks for their validity and adds them into the activity models. Moreover, we present a torso angle feature, which represents people posture, to detect activities, such as people falling. We used real world data sets to show the effectiveness of our proposed method.