The reliable evaluation of the performance of watermarking algorithms is difficult. An important aspect in this process is the assessment of the visibility of the watermark. In this paper, we address this issue and propose a methodology for evaluating the visual quality of watermarked video. Using a software tool that measures different types of perceptual video artifacts, we determine the most relevant impairments and design the corresponding objective metrics. We demonstrate their performance through subjective experiments on several different watermarking algorithms and video sequences.