Video summarization has become an indispensable tool of any practical video content management system in large volume video data. In this paper, we propose a novel approach to automatically generate the video summary for broadcast news videos. Firstly, videos are pre-processed by shot detection, key frame extraction, and story segmentation. Then, a clustering algorithm based on affinity propagation (AP) is originally introduced to group the key frames into clusters. Moreover, a semantic content mining approach based on vector space model (VSM) is adopted to select the most informative video shots for constructing the video summary. This aims to keep the pertinent key frames that distinguish one scene to others and remove the visual-content redundancy from news video. Experimental results show that the proposed method can efficiently generate a set of representative shots and also extract the hierarchical structure of a video sequence.