Issue No. 11 - Nov. (2013 vol. 35)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPAMI.2013.54
V. Badrinarayanan , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
I. Budvytis , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
R. Cipolla , Dept. of Eng., Univ. of Cambridge, Cambridge, UK
We present a novel patch-based probabilistic graphical model for semi-supervised video segmentation. At the heart of our model is a temporal tree structure that links patches in adjacent frames through the video sequence. This permits exact inference of pixel labels without resorting to traditional short time window-based video processing or instantaneous decision making. The input to our algorithm is labeled key frame(s) of a video sequence and the output is pixel-wise labels along with their confidences. We propose an efficient inference scheme that performs exact inference over the temporal tree, and optionally a per frame label smoothing step using loopy BP, to estimate pixel-wise labels and their posteriors. These posteriors are used to learn pixel unaries by training a Random Decision Forest in a semi-supervised manner. These unaries are used in a second iteration of label inference to improve the segmentation quality. We demonstrate the efficacy of our proposed algorithm using several qualitative and quantitative tests on both foreground/background and multiclass video segmentation problems using publicly available and our own datasets.
Image segmentation, Vegetation, Graphical models, Computational modeling, Video sequences, Probabilistic logic, Inference algorithms
V. Badrinarayanan, I. Budvytis and R. Cipolla, "Semi-Supervised Video Segmentation Using Tree Structured Graphical Models," in IEEE Transactions on Pattern Analysis & Machine Intelligence, vol. 35, no. 11, pp. 2751-2764, 2013.