The advances of video technology and video-related applications demand appropriate video semantic models for representing video data and their semantics, and supporting powerful semantic queries on them. In this paper, we propose such a model named SemTTE. The model incorporates features of temporal structure and typed events of video contents. It organizes the whole video into a tree of events, and provides mechanisms for users to define domain-specific constraints. As a result, the contents and semantics of the video can be better represented and queried. For constraints enforcement, an efficient on-line method is proposed.