CSDL Home IEEE Transactions on Pattern Analysis & Machine Intelligence 2008 vol.30 Issue No.09 - September

Subscribe

Issue No.09 - September (2008 vol.30)

pp: 1589-1602

ABSTRACT

We present a novel local spatiotemporal approach to produce motion segmentation and dense temporal trajectories from an image sequence. A common representation of image sequences is a 3D spatiotemporal volume, (x,y,t), and its corresponding mathematical formalism is the fiber bundle. However, directly enforcing the spatiotemporal smoothness constraint is difficult in the fiber bundle representation. Thus, we convert the representation into a new 5D space (x,y,t,vx,vy) with an additional velocity domain, where each moving object produces a separate 3D smooth layer. The smoothness constraint is now enforced by extracting 3D layers using the tensor voting framework in a single step that solves both correspondence and segmentation simultaneously. Motion segmentation is achieved by identifying those layers, and the dense temporal trajectories are obtained by converting the layers back into the fiber bundle representation. We proceed to address three applications (tracking, mosaic, and 3D reconstruction) that are hard to solve from the video stream directly because of the segmentation and dense matching steps, but become straightforward with our framework. The approach does not make restrictive assumptions about the observed scene or camera motion and is therefore generally applicable. We present results on a number of data sets.

INDEX TERMS

Motion analysis, Tensor voting, Optical Flow, Segmentation, Mosaicking

CITATION

Changki Min, Gérard Medioni, "Inferring Segmented Dense Motion Layers Using 5D Tensor Voting",

*IEEE Transactions on Pattern Analysis & Machine Intelligence*, vol.30, no. 9, pp. 1589-1602, September 2008, doi:10.1109/TPAMI.2007.70802REFERENCES

- [1]
Middlebury College Stereo Evaluation Webpage, http://vision. middlebury.edustereo/, 2008.- [2] E. Adelson and Y. Weiss, “A Unified Mixture Framework for Motion Segmentation: Incorporating Spatial Coherence and Estimating the Number of Models,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 321-326, 1996.- [3] T. Amiaz and N. Kiryati, “Dense Discontinuous Optical Flow via Contour-Based Segmentation,”
Proc. IEEE Int'l Conf. Image Processing, vol. 3, pp. 1264-1267, 2005.- [9] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High Accuracy Optical Flow Estimation Based on a Theory for Warping,”
Proc. Eighth European Conf. Computer Vision, vol. 4, pp. 25-36, 2004.- [10] T. Brox, A. Bruhn, and J. Weickert, “Variational Motion Segmentation with Level Sets,”
Proc. Ninth European Conf. Computer Vision, pp. 471-483, May 2006.- [12] J. Costeira and T. Kanade, “A Multibody Factorization Method for Independently Moving Objects,”
Int'l J. Computer Vision, vol. 29, no. 3, pp. 159-179, Sept. 1998.- [16] D. Husemoller,
Fibre Bundles, third ed. Springer, 1993.- [18] A. Klaus, M. Sormann, and K. Karner, “Segment-Based Stereo Matching Using Belief Propagation and a Self-Adapting Dissimilarity Measure,”
Proc. 18th Int'l Conf. Pattern Recognition, 2006.- [20] G. Medioni, M. Lee, and C. Tang,
A Computational Framework for Segmentation and Grouping, first ed. Elsevier, 2000.- [21] E. Mémin and P. Pérez, “Hierarchical Estimation and Segmentation of Dense Motion Fields,”
Int'l J. Computer Vision, vol. 46, no. 2, pp. 129-155, Feb. 2002.- [22] C. Min and G. Medioni, “Motion Segmentation by Spatiotemporal Smoothness Using 5D Tensor Voting,”
Proc. Fifth IEEE Workshop Perceptual Organization in Computer Vision, 2006.- [23] C. Min and G. Medioni, “Tensor Voting Accelerated by Graphics Processing Units (GPU),”
Proc. 18th Int'l Conf. Pattern Recognition, 2006.- [24] C. Min, Q. Yu, and G. Medioni, “Multi-Layer Mosaics in the Presence of Motion and Depth Effects,”
Proc. 18th Int'l Conf. Pattern Recognition, 2006.- [27] H. Shum and R. Szeliski, “Systems and Experiment Paper: Construction of Panoramic Image Mosaics with Global and Local Alignment,”
Int'l J. Computer Vision, vol. 36, no. 2, pp. 101-130, Feb. 2000.- [30] P. Torr and D. Murray, “The Development and Comparison of Robust Methods for Estimating the Fundamental Matrix,”
Int'l J. Computer Vision, vol. 24, no. 3, pp. 271-300, Oct. 1997.- [31] M. Uyttendaele, A. Eden, and R. Szeliski, “Eliminating Ghosting and Exposure Artifacts in Image Mosaics,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 509-516, 2001.- [32] R. Vidal and R. Hartley, “Motion Segmentation with Missing Data Using Power Factorization and GPCA,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 310-316, 2004.- [33] J. Wang and E. Adelson, “Layered Representation for Motion Analysis,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 361-366, 1993.- [36] L. Zelnik-Manor and M. Irani, “Degeneracies, Dependencies and Their Implications in Multi-Body and Multi-Sequence Factorizations,”
Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 287-293, 2003. |