Advanced video surveillance applications require two successive steps: image analysis and content understanding. The first step analyses and extracts the characteristics of the video sequence. It defines the regions or the objects of interest according to their spatial/temporal properties. This analysis results in a segmentation of the video sequence. This is interpreted by the content understanding step according to the specific scenario and surveillance requirements.This paper addresses the image analysis problem for a video surveillance system. We use a statistical model-based change detection technique that defines the areas of interest in the image. Each area is analyzed separately by integrating spatial and temporal descriptors in a multi-feature clustering algorithm. The selective procedure we propose minimizes the computational load and significantly improves the results provided by the change detection technique.We test this method on both indoor and outdoor surveillance sequences. All the results show a correct segmentation of the scene. Moreover each object defined in the segmentation is described in terms of its spatial and temporal properties. These results can represent a valid input for a later content understanding procedure in several surveillance scenarios.