Issue No. 11 - Nov. (2013 vol. 35)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2013.54
V. Badrinarayanan , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
I. Budvytis , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
R. Cipolla , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
We present a novel patch-based probabilistic graphical model for semi-supervised video segmentation. At the heart of our model is a temporal tree structure that links patches in adjacent frames through the video sequence. This permits exact inference of pixel labels without resorting to traditional short time window-based video processing or instantaneous decision making. The input to our algorithm is labeled key frame(s) of a video sequence and the output is pixel-wise labels along with their confidences. We propose an efficient inference scheme that performs exact inference over the temporal tree, and optionally a per frame label smoothing step using loopy BP, to estimate pixel-wise labels and their posteriors. These posteriors are used to learn pixel unaries by training a Random Decision Forest in a semi-supervised manner. These unaries are used in a second iteration of label inference to improve the segmentation quality. We demonstrate the efficacy of our proposed algorithm using several qualitative and quantitative tests on both foreground/background and multiclass video segmentation problems using publicly available and our own datasets.
Image segmentation, Vegetation, Graphical models, Computational modeling, Video sequences, Probabilistic logic, Inference algorithms,structured variational inference, Semi-supervised video segmentation, label propagation, mixture of trees graphical model, tree-structured video models
V. Badrinarayanan, I. Budvytis, R. Cipolla, "Semi-Supervised Video Segmentation Using Tree Structured Graphical Models", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 35, no. , pp. 2751-2764, Nov. 2013, doi:10.1109/TPAMI.2013.54