The Community for Technology Leaders
Green Image
Issue No. 11 - Nov. (2013 vol. 35)
ISSN: 0162-8828
pp: 2751-2764
V. Badrinarayanan , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
I. Budvytis , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
R. Cipolla , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
ABSTRACT
We present a novel patch-based probabilistic graphical model for semi-supervised video segmentation. At the heart of our model is a temporal tree structure that links patches in adjacent frames through the video sequence. This permits exact inference of pixel labels without resorting to traditional short time window-based video processing or instantaneous decision making. The input to our algorithm is labeled key frame(s) of a video sequence and the output is pixel-wise labels along with their confidences. We propose an efficient inference scheme that performs exact inference over the temporal tree, and optionally a per frame label smoothing step using loopy BP, to estimate pixel-wise labels and their posteriors. These posteriors are used to learn pixel unaries by training a Random Decision Forest in a semi-supervised manner. These unaries are used in a second iteration of label inference to improve the segmentation quality. We demonstrate the efficacy of our proposed algorithm using several qualitative and quantitative tests on both foreground/background and multiclass video segmentation problems using publicly available and our own datasets.
INDEX TERMS
Image segmentation, Vegetation, Graphical models, Computational modeling, Video sequences, Probabilistic logic, Inference algorithms,structured variational inference, Semi-supervised video segmentation, label propagation, mixture of trees graphical model, tree-structured video models
CITATION
V. Badrinarayanan, I. Budvytis, R. Cipolla, "Semi-Supervised Video Segmentation Using Tree Structured Graphical Models", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 35, no. , pp. 2751-2764, Nov. 2013, doi:10.1109/TPAMI.2013.54
321 ms
(Ver 3.3 (11022016))