This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Structured Learning of Human Interactions in TV Shows
Dec. 2012 (vol. 34 no. 12)
pp. 2441-2453
A. Patron-Perez, Dept. of Comput. Sci., George Washington Univ., Washington, DC, USA
M. Marszalek, Google, Inc., Adliswil, Switzerland
I. Reid, Dept. of Eng. Sci., Univ. of Oxford, Oxford, UK
A. Zisserman, Dept. of Eng. Sci., Univ. of Oxford, Oxford, UK
The objective of this work is recognition and spatiotemporal localization of two-person interactions in video. Our approach is person-centric. As a first stage we track all upper bodies and heads in a video using a tracking-by-detection approach that combines detections with KLT tracking and clique partitioning, together with occlusion detection, to yield robust person tracks. We develop local descriptors of activity based on the head orientation (estimated using a set of pose-specific classifiers) and the local spatiotemporal region around them, together with global descriptors that encode the relative positions of people as a function of interaction type. Learning and inference on the model uses a structured output SVM which combines the local and global descriptors in a principled manner. Inference using the model yields information about which pairs of people are interacting, their interaction class, and their head orientation (which is also treated as a variable, enabling mistakes in the classifier to be corrected using global context). We show that inference can be carried out with polynomial complexity in the number of people, and describe an efficient algorithm for this. The method is evaluated on a new dataset comprising 300 video clips acquired from 23 different TV shows and on the benchmark UT--Interaction dataset.
Index Terms:
Magnetic heads,Context awareness,Support vector machines,Video retrieval,Human factors,Tracking,Spatiotemporal phenomena,structured SVM,Human interaction recognition,video retrieval
Citation:
A. Patron-Perez, M. Marszalek, I. Reid, A. Zisserman, "Structured Learning of Human Interactions in TV Shows," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 12, pp. 2441-2453, Dec. 2012, doi:10.1109/TPAMI.2012.24
Usage of this product signifies your acceptance of the Terms of Use.