We propose a compact descriptor of video contents based on modeling the temporal behavior of image features using coupled Markov chains. The framework allows us to combine multiple features within the same model, including the representation of the dependencies and relationships between them. The Kullback-Leibler divergence stands out as the base of a perceptually significant distance measure for our descriptor. Our experiments show that complex high-level visual contents in different domains can be characterized using very simple low-level features, such as motion and color.