The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.10 - Oct. (2012 vol.34)
pp: 1942-1951
Alper Ayvaci , University of California, Los Angeles
Stefano Soatto , University of California, Los Angeles
ABSTRACT
We describe an approach for segmenting a moving image into regions that correspond to surfaces in the scene that are partially surrounded by the medium. It integrates both appearance and motion statistics into a cost functional that is seeded with occluded regions and minimized efficiently by solving a linear programming problem. Where a short observation time is insufficient to determine whether the object is detachable, the results of the minimization can be used to seed a more costly optimization based on a longer sequence of video data. The result is an entirely unsupervised scheme to detect and segment an arbitrary and unknown number of objects. We test our scheme to highlight the potential, as well as limitations, of our approach.
INDEX TERMS
Motion segmentation, Optimization, Image segmentation, Object recognition, Linear programming, Mathematical model, model selection., Object detection, video segmentation, occlusion, layers, graph cuts, ordering constraints
CITATION
Alper Ayvaci, Stefano Soatto, "Detachable Object Detection: Segmentation and Depth Ordering from Short-Baseline Video", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 10, pp. 1942-1951, Oct. 2012, doi:10.1109/TPAMI.2011.271
REFERENCES
[1] J.J. Gibson, The Ecological Approach to Visual Perception. LEA, 1984.
[2] A. Ayvaci, M. Raptis, and S. Soatto, "Sparse Occlusion Detection with Optical Flow," Int'l J. Computer Vision, vol. 6, pp. 1-17, 2011.
[3] J. Jackson, A.J. Yezzi, and S. Soatto, "Dynamic Shape and Appearance Modeling via Moving and Deforming Layers," Int'l J. Computer Vision, vol. 79, no. 1, pp. 71-84, Aug. 2008.
[4] J.D. Jackson, A.J. Yezzi, and S. Soatto, "Dynamic Shape and Appearance Modeling via Moving and Deforming Layers," Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2005.
[5] D. Cremers and S. Soatto, "Motion Competition: A Variational Approach to Piecewise Parametric Motion Segmentation," Int'l J. Computer Vision, vol. 62, no. 3, pp. 249-265, May 2005.
[6] X. Bai, J. Wang, D. Simons, and G. Sapiro, "Video SnapCut: Robust Video Object Cutout Using Localized Classifiers," Proc. ACM Siggraph, 2009.
[7] M. Unger, T. Mauthner, T. Pock, and H. Bischof, "Tracking as Segmentation of Spatial-Temporal Volumes by Anisotropic Weighted TV," Proc. Seventh Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2009.
[8] Y. Huang, Q. Liu, and D. Metaxas, "Video Object Segmentation by Hypergraph Cut," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1738-1745. 2009,
[9] J. Wang, Y. Xu, H. Shum, and M. Cohen, "Video Tooning," Proc. ACM Siggraph, 2004.
[10] T. Brox and J. Malik, "Object Segmentation by Long Term Analysis of Point Trajectories," Proc. European Conf. Computer Vision, pp. 282-295. 2010,
[11] W. Brendel and S. Todorovic, "Video Object Segmentation by Tracking Regions," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[12] A. Vazquez-Reina, S. Avidan, H. Pfister, and E. Miller, "Multiple Hypothesis Video Segmentation from Superpixel Flows," Proc. European Conf. Computer Vision, 2010.
[13] G. Brostow and I. Essa, "Motion Based Decompositing of Video," Proc. IEEE Int'l Conf. Computer Vision, 1999.
[14] A. Ogale, C. Ferm, and Y. Aloimonos, "Motion Segmentation Using Occlusions," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 988-992, June 2005.
[15] D. Feldman and D. Weinshall, "Motion Segmentation and Depth Ordering Using an Occlusion Detector," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 7, pp. 1171-1185, July 2008.
[16] M. Irani and S. Peleg, "Motion Analysis for Image Enhancement: Resolution, Occlusion, and Transparency," J. Visual Comm. and Image Representation, vol. 4, pp. 324-324, 1993.
[17] M.P. Kumar, P. Torr, and A. Zisserman, "Learning Layered Motion Segmentations of Video," Int'l J. Computer Vision, vol. 76, no. 3, pp. 301-319, 2008.
[18] A. Jepson, D. Fleet, and M. Black, "A Layered Motion Representation with Occlusion and Compact Spatial Support," Proc. European Conf. Computer Vision, pp. 692-706. 2002,
[19] P. Smith, T. Drummond, and R. Cipolla, "Layered Motion Segmentation and Depth Ordering by Tracking Edges," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 4, pp. 479-494, Apr. 2004.
[20] A. Stein, T. Stepleton, and M. Hebert, "Towards Unsupervised Whole-Object Segmentation: Combining Automated Matting with Boundary Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2008.
[21] N. Apostoloff and A. Fitzgibbon, "Automatic Video Segmentation Using Spatiotemporal T-Junctions," Proc. British Machine Vision Conf., 2006.
[22] A. Stein and M. Hebert, "Occlusion Boundaries from Motion: Low-Level Detection and Mid-Level Reasoning," Int'l J. Computer Vision, vol. 82, no. 3, pp. 325-357, 2009.
[23] M. Sargin, L. Bertelli, B. Manjunath, and K. Rose, "Probabilistic Occlusion Boundary Detection on Spatio-Temporal Lattices," Proc. Int'l Conf. Computer Vision, 2009.
[24] X. He and A. Yuille, "Occlusion Boundary Detection Using Pseudo-Depth," Proc. European Conf. Computer Vision, 2010.
[25] N. Apostoloff and A. Fitzgibbon, "Learning Spatiotemporal T-Junctions for Occlusion Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2005.
[26] J. Morel and P. Salembier, "Monocular Depth by Nonlinear Diffusion," Proc. Indian Conf. Computer Vision, Graphics & Image Processing, 2008.
[27] M. Amer, R. Raich, and S. Todorovic, "Monocular Extraction of 2.1D Sketch," Proc. Int'l Conf. Image Processing, Sept. 2010.
[28] L. Grady, "Random Walks for Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1768-1783, Nov. 2006.
[29] Y. Boykov, O. Veksler, and R. Zabih, "Fast Approximate Energy Minimization via Graph Cuts," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1222-1239, Nov. 2001.
[30] T. Chan and S. Esedoglu, "Aspects of Total Variation Regularized L'Function Approximation," SIAM J. Applied Math., vol. 65, no. 5, pp. 1817-1837, 2005.
[31] S. Boltz, A. Herbulot, E. Debreuve, M. Barlaud, and G. Aubert, "Motion and Appearance Nonparametric Joint Entropy for Video Segmentation," Int'l J. Computer Vision, vol. 80, pp. 242-259, 2007.
[32] A.K. Sinop and L. Grady, "A Seeded Image Segmentation Framework Unifying Graph Cuts and Random Walker Which Yields a New Algorithm," Proc. IEEE Int'l Conf. Computer Vision, 2007.
[33] P. Grunwald and J. Rissanen, The Minimum Description Length Principle. The MIT Press, 2007.
[34] A. Ayvaci and S. Soatto, "Detachable Object Detection with Efficient Model Selection," Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, July 2011.
[35] M. Grant and S. Boyd, "Cvx: Matlab Software for Disciplined Convex Programming, Version 1.21," http://cvxr.comcvx, Oct. 2010.
[36] D. Martin, C. Fowlkes, and J. Malik, "Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530-549, May 2004.
[37] C. Rother, T. Minka, A. Blake, and V. Kolmogorov, "Cosegmentation of Image Pairs by Histogram Matching-Incorporating a Global Constraint into Mrfs," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2006.
[38] A. Yezzi and S. Soatto, "Stereoscopic Segmentation," Int'l J. Computer Vision, vol. 53, no. 1, pp. 31-43, 2003.
[39] P. Arbeláez, M. Maire, C. Fowlkes, and J. Malik, "From Contours to Regions: An Empirical Evaluation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[40] J. Shi and J. Malik, "Normalized Cuts and Image Segmentation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 8, pp. 888-905, Aug. 2000.
[41] L. Zelnik-Manor and P. Perona, "Self-Tuning Spectral Clustering," Advances in Neural Information Processing Systems, vol. 2, pp. 1601-1608, 2004.
[42] B. Fulkerson and S. Soatto, "Really Quick Shift: Image Segmentation on a GPU," Proc. Workshop Computer Vision Using GPUs, held with the European Conf. Computer Vision, Sept. 2010.
[43] C. Wang, M. de La Gorce, and N. Paragios, "Segmentation, Ordering and Multi-Object Tracking Using Graphical Models," Proc. IEEE Int'l Conf. Computer Vision, pp. 747-754, 2009.
23 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool