This Article 
 Bibliographic References 
 Add to: 
Semi-Supervised Video Segmentation Using Tree Structured Graphical Models
Nov. 2013 (vol. 35 no. 11)
pp. 2751-2764
V. Badrinarayanan, Dept. of Eng., Univ. of Cambridge, Cambridge, UK
I. Budvytis, Dept. of Eng., Univ. of Cambridge, Cambridge, UK
R. Cipolla, Dept. of Eng., Univ. of Cambridge, Cambridge, UK
We present a novel patch-based probabilistic graphical model for semi-supervised video segmentation. At the heart of our model is a temporal tree structure that links patches in adjacent frames through the video sequence. This permits exact inference of pixel labels without resorting to traditional short time window-based video processing or instantaneous decision making. The input to our algorithm is labeled key frame(s) of a video sequence and the output is pixel-wise labels along with their confidences. We propose an efficient inference scheme that performs exact inference over the temporal tree, and optionally a per frame label smoothing step using loopy BP, to estimate pixel-wise labels and their posteriors. These posteriors are used to learn pixel unaries by training a Random Decision Forest in a semi-supervised manner. These unaries are used in a second iteration of label inference to improve the segmentation quality. We demonstrate the efficacy of our proposed algorithm using several qualitative and quantitative tests on both foreground/background and multiclass video segmentation problems using publicly available and our own datasets.
Index Terms:
video signal processing,backpropagation,image segmentation,image sequences,inference mechanisms,segmentation quality,semisupervised video segmentation,tree structured graphical model,patch-based probabilistic graphical model,temporal tree structure,video sequence,pixel label inference,short time window-based video processing,pixel-wise label,loopy BP,backpropagation,Image segmentation,Vegetation,Graphical models,Computational modeling,Video sequences,Probabilistic logic,Inference algorithms,structured variational inference,Semi-supervised video segmentation,label propagation,mixture of trees graphical model,tree-structured video models
V. Badrinarayanan, I. Budvytis, R. Cipolla, "Semi-Supervised Video Segmentation Using Tree Structured Graphical Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 11, pp. 2751-2764, Nov. 2013, doi:10.1109/TPAMI.2013.54
Usage of this product signifies your acceptance of the Terms of Use.