This paper presents an approach for indexing a large set of videos by considering the dynamic behaviour of local visual features along the sequences. The proposed concept is based on the extraction and the local description of interest points and further on the estimation of their trajectories along the video sequence. Analysing the low-level description obtained allows to highlight trends of behaviour and then to assign a label. Such an indexing approach of the video content has several interesting properties: the lowlevel descriptors provide a rich and compact description, while labels of behaviour provide a generic semantic description of the video content, relevant for video content retrieval. We demonstrate the effectiveness of this approach for Content-Based Copy Detection (CBCD) on large collections of videos (several hundred hours of videos).