We propose a new approach to temporal video segmentation in which shot boundaries are detected by object-level matching of video frames. I-frames are initially segmented using features extracted from the HSV color space. Two consecutive I-frames are compared in terms of the color, size and position of their objects. A shot boundary is detected if there is a strong mismatch between frame objects. We have developed a web-based video retrieval system for demonstrating our work and for performing user queries.