CVPR 2011 (2011)
June 20, 2011 to June 25, 2011
V. I. Morariu , Inst. for Adv. Comput. Studies, Univ. of Maryland, College Park, MD, USA
L. S. Davis , Inst. for Adv. Comput. Studies, Univ. of Maryland, College Park, MD, USA
We present a framework for the automatic recognition of complex multi-agent events in settings where structure is imposed by rules that agents must follow while performing activities. Given semantic spatio-temporal descriptions of what generally happens (i.e., rules, event descriptions, physical constraints), and based on video analysis, we determine the events that occurred. Knowledge about spatio-temporal structure is encoded using first-order logic using an approach based on Allen's Interval Logic, and robustness to low-level observation uncertainty is provided by Markov Logic Networks (MLN). Our main contribution is that we integrate interval-based temporal reasoning with probabilistic logical inference, relying on an efficient bottom-up grounding scheme to avoid combinatorial explosion. Applied to one-on-one basketball, our framework detects and tracks players, their hands and feet, and the ball, generates event observations from the resulting trajectories, and performs probabilistic logical inference to determine the most consistent sequence of events. We demonstrate our approach on 1hr (100,000 frames) of outdoor videos.
bottom-up grounding scheme, multiagent event recognition, semantic spatio-temporal descriptions, video analysis, first-order logic, Allen interval logic, Markov logic networks, interval-based temporal reasoning, probabilistic logical inference
V. I. Morariu and L. S. Davis, "Multi-agent event recognition in structured scenarios," CVPR 2011(CVPR), Providence, RI, 2011, pp. 3289-3296.