For more efficiently organizing, browsing, and retrieving digital video, it is important to extract video structure information at both scene and shot levels. This paper presents an effective approach to video scene segmentation (shot grouping). In our proposed method, we define a new similarity measurement by considering the visual similarity among shots together with the temporal relation condition and use a slide window model to detect scene boundary. The experimental results show that our method produces reasonable detection results.