This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Layered 4D Representation and Voting for Grouping from Motion
April 2003 (vol. 25 no. 4)
pp. 492-501
Mircea Nicolescu, IEEE Computer Society

Abstract—We address the problem of perceptual grouping from motion cues by formulating it as a motion layers inference from a sparse and noisy point set in a 4D space. Our approach is based on a layered 4D representation of data, and a voting scheme for token communication, within a tensor voting computational framework. Given two sparse sets of point tokens, the image position and potential velocity of each token are encoded into a 4D tensor. By enforcing the smoothness of motion through a voting process, the correct velocity is selected for each input point as the most salient token. An additional dense voting step allows for the inference of a dense representation in terms of pixel velocities, motion regions, and boundaries. Using a 4D space for this tensor voting approach is essential since it allows for a spatial separation of the points according to both their velocities and image coordinates. Unlike most other methods that optimize certain objective functions, our approach is noniterative and, therefore, does not suffer from local optima or poor convergence problems. We demonstrate our method with synthetic and real images, by analyzing several difficult cases—opaque and transparent motion, rigid and nonrigid motion, curves and surfaces in motion.

[1] S. Ullman, The Interpretation of Visual Motion. MIT Press, 1979.
[2] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43–77, 1994.
[3] B. Horn and B. Schunck, “Determining Optical Flow,” Artificial Intelligence, vol. 17, pp. 185-203, 1981.
[4] B. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. DARPA Image Understanding Workshop, pp. 121-130, 1981.
[5] E.P. Simoncelli, E.H. Adelson, and D.J. Heeger, "Probability Distributions of Optic Flow," IEEE CS Conf. Computer Vision and Pattern Recognition (CVPR'91), pp. 310-315,Maui, Hawaii, June 1991, IEEE CS Press.
[6] H. H. Nagel,W. Enkelmann,“An Investigation of smoothness constraints for the estimation of displacement vector fields from image sequences,” Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 5, pp. 565-593, 1986
[7] P. Anandan, “A Computational Framework and an Algorithm for the Measurement of Visual Motion,” Int'l J. Visual Computing, vol. 2, pp. 283-310, 1989.
[8] P.J. Burt and E.H. Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE Trans. Comm., vol. 31, no. 4, pp. 532-540, 1983.
[9] A. Singh, Optical Flow Computation: A Unified Perspective. IEEE CS Press, 1992.
[10] D. Heeger, “Optical Flow using Spatiotemporal Filters,” Int'l J. Visual Computing, vol. 1, pp. 279-302, 1988.
[11] F. Heitz and P. Bouthemy, Multimodal Estimation of Discontinuous Optical Flow Using Markov Random Fields IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 12, pp. 1217-1232, Dec. 1993.
[12] M. Gelgon and P. Bouthemy, “A Region-Level Graph Labeling Approach to Motion-Based Segmentation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 514-519, 1997.
[13] C. Kervrann and F. Heitz, "A Markov Random Field Model-Based Approach to Unsupervised Texture Segmentation Using Local and Global Statistics," IEEE Trans. Image Processing, vol. 4, no. 6, pp. 856-862, June 1995.
[14] Y. Boykov, O. Veksler, and R. Zabih, Markov Random Fields with Efficient Approximations Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.
[15] S. Ghosal, “A Fast Scalable Algorithm for Discontinuous Optical Flow Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 181-194, Feb. 1996.
[16] R. Deriche, P. Kornprobst, and G. Aubert, “Optical Flow Estimation while Preserving its Discontinuities: A Variational Approach,” Proc. Asian Conf. Computer Vision, pp. 290-295, 1995.
[17] T. Darrell and A.P. Pentland,“Robust estimation of a multilayer motion representation,” Proc. IEEE Workshop on Visual Motion, pp. 173-177, 1991.
[18] S. Ayer and H. Sawhney, "Layered Representation of Motion Video Using Robust Maximum-Likelihood Estimation of Mixture Models and mdl Encoding," Int'l Conf. Computer Vision, pp. 777-784,Cambridge, Mass., June 1995.
[19] G. McLachlan and K. Basford, Mixture Models Inference and Applications to Clustering. Marcel Dekker, Inc., 1988.
[20] S. Hsu, P. Anandan, and S. Peleg, "Accurate Computation of Optical Flow by Using Layered Motion Representation," Proc. ICPR, pp. 743-746,Jerusalem, Oct. 1994.
[21] A.D. Jepson and M. Black, “Mixture Models for Optical Flow Computation,” Proc. Computer Vision and Pattern Recognition, pp. 760-761, June 1993.
[22] Y. Weiss, "Smoothness in Layers: Motion Segmentation Using Nonparametric Mixture Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 520-527, 1997.
[23] M. Irani, B. Rousso, and S. Peleg, “Image Sequence Enhancement Using Multiple Motions Analysis,” Proc. 1992 Conf. Computer Vision and Pattern Recognition, pp. 216-221, 1992.
[24] J.Y.A. Wang and E.H. Adelson, Representing Moving Images with Layers IEEE Trans. Image Processing, vol. 3, no. 5, pp. 625-638, Sept. 1994.
[25] D. Fleet, M. Black, and A. Jepson, “Motion Feature Detection Using Steerable Flow Fields,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 274-281, 1998.
[26] J. Shi and J. Malik, Motion Segmentation and Tracking Using Normalized Cuts Proc. Sixth Int'l Conf. Computer Vision (ICCV '98), pp. 1154-1160, 1998.
[27] Y. Wu, T. Kanade, J. Cohn, and C. Li, “Optical Flow Estimation Using Wavelet Motion Model,” Proc. Int'l Conf. Computer Vision, pp. 992-998, 1998.
[28] J. Little, H. Bulthoff, and T. Poggio, “Parallel Optical Flow Using Local Voting,” Proc. Int'l Conf. Computer Vision, pp. 454-459, 1988.
[29] G. Medioni, M. Lee, and C. Tang, A Computational Framework for Segmentation and Grouping. Elsevier Science B.V., 2000.
[30] L. Gaucher and G. Medioni, Accurate Motion Flow Estimation with Discontinuities Proc. Seventh Int'l Conf. Computer Vision (ICCV '99), pp. 695-702, Sept. 1999.
[31] G. Guy and G. Medioni, “Inference of Surfaces, 3D Curves, and Junctions from Sparse, Noisy, 3D Data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 11, pp. 1265-1277, Nov. 1997.
[32] S. Arya, D.M. Mount, N.S. Netanyahu, R. Silverman, and A.Y. Wu, “An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions,” J. ACM, vol. 45, no. 6, pp. 891-923, Nov. 1998.
[33] J.R. Bergen, P.J. Burt, R. Hingorani, and S. Peleg, "A Three-Frame Algorithm for Estimating Two-Component Image Motion," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, pp. 886-895, Sept. 1992.
[34] R. Szeliski, S. Avidan, and P. Anandan, “Layer Extraction from Multiple Images Containing Reflections and Transparency,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 246-253, 2000.

Index Terms:
Motion analysis, perceptual grouping, tensor voting.
Citation:
Mircea Nicolescu, Gérard Medioni, "Layered 4D Representation and Voting for Grouping from Motion," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 4, pp. 492-501, April 2003, doi:10.1109/TPAMI.2003.1190574
Usage of this product signifies your acceptance of the Terms of Use.