This paper describes a novel tracking performance evaluation metric based on the successful detection of events, rather than low-level image processing criteria. A general event metric is defined to measure whether the agents and actions in the scene given by the ground truth were corectly tracked by comparing two event lists using dynamic programming. This metric is suitable to evaluate and compare different tracking approaches where the underlying algorithm may be completely different. Furthermore, we introduce an automatic extraction of those semantically high level events from different types of low level tracking data and human annotated ground truth. A case study with two different trackers on public datasets shows the effectiveness of this evaluation scheme.