This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Voting-Based Computational Framework for Visual Motion Analysis and Interpretation
May 2005 (vol. 27 no. 5)
pp. 739-752
Most approaches for motion analysis and interpretation rely on restrictive parametric models and involve iterative methods which depend heavily on initial conditions and are subject to instability. Further difficulties are encountered in image regions where motion is not smooth—typically around motion boundaries. This work addresses the problem of visual motion analysis and interpretation by formulating it as an inference of motion layers from a noisy and possibly sparse point set in a 4D space. The core of the method is based on a layered 4D representation of data and a voting scheme for affinity propagation. The inherent problem caused by the ambiguity of 2D to 3D interpretation is usually handled by adding additional constraints, such as rigidity. However, enforcing such a global constraint has been problematic in the combined presence of noise and multiple independent motions. By decoupling the processes of matching, outlier rejection, segmentation, and interpretation, we extract accurate motion layers based on the smoothness of image motion, then locally enforce rigidity for each layer in order to infer its 3D structure and motion. The proposed framework is noniterative and consistently handles both smooth moving regions and motion discontinuities without using any prior knowledge of the motion model.

[1] S. Ullman, The Interpretation of Visual Motion. MIT Press, 1979.
[2] E. Hildreth, The Measurement of Visual Motion. MIT Press, 1983.
[3] G. Medioni, M.-S. Lee, and C.-K. Tang, A Computational Framework for Segmentation and Grouping. Elsevier Science, 2000.
[4] H.C. Longuet-Higgins, “A Computer Algorithm for Reconstructing a Scene from Two Projections,” Nature, vol. 293, pp. 133-135, 1981.
[5] A. Adam, E. Rivlin, and L. Shimshoni, “Ror: Rejection of Outliers by Rotations,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 1, pp. 78-84, Jan. 2001.
[6] Z. Zhang, “Determining the Epipolar Geometry and Its Uncertainty: A Review,” Int'l J. Computer Vision, vol. 27, no. 2, pp. 161-195, 1998.
[7] R.I. Hartley, “In Defense of the 8-Point Algorithm,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 6, pp. 580-593, June 1997.
[8] M. Nicolescu and G. Medioni, “Layered 4D Representation and Voting for Grouping from Motion,” IEEE Trans. Pattern Analysis and Machine Intelligence, special issue on perceptual organization in computer vision, vol. 25, no. 4, pp. 492-501, Apr. 2003.
[9] M. Nicolescu and G. Medioni, “Motion Segmentation with Accurate Boundaries— A Tensor Voting Approach,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 382-389, 2003.
[10] J. Barron, D. Fleet, and S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-77, Feb. 1994.
[11] B. Horn and B. Schunck, “Determining Optical Flow,” Artificial Intelligence, vol. 17, pp. 185-204, 1981.
[12] B. Lucas and T. Kanade, “An Iterative Image Registration Technique with an Application to Stereo Vision,” Proc. DARPA Image Understanding Workshop, pp. 121-130, 1981.
[13] E. Simoncelli, E. Adelson, and D. Heeger, “Probability Distributions of Optical Flow,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 310-315, 1991.
[14] H.-H. Nagel and W. Enkelmann, “An Investigation of Smoothness Constraints for the Estimation of Displacement Vector Fields from Image Sequences,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, pp. 565-593, 1986.
[15] P. Anandan, “A Computational Framework and an Algorithm for the Measurement of Visual Motion,” Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[16] P. Burt and E. Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE Trans. Comm., vol. 31, pp. 532-540, 1983.
[17] A. Singh, Optical Flow Computation: A Unified Perspective. IEEE Press, 1992.
[18] D. Heeger, “Optical Flow Using Spatiotemporal Filters,” Int'l J. Computer Vision, vol. 1, pp. 279-302, 1988.
[19] F. Heitz and P. Bouthemy, “Multimodel Estimation of Discontinuous Optical Flow Using Markov Random Fields,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 12, pp. 1217-1232, Dec. 1993.
[20] M. Gelgon and P. Bouthemy, “A Region-Level Graph Labeling Approach to Motion-Based Segmentation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 514-519, June 1997.
[21] C. Kervrann and F. Heitz, “A Markov Random Field Model-Based Approach to Unsupervised Texture Segmentation Using Local and Global Spatial Statistics,” IEEE Trans. Image Processing, vol. 4, no. 6, pp. 856-862, 1995.
[22] Y. Boykov, O. Veksler, and R. Zabih, “Markov Random Fields with Efficient Approximations,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 648-655, 1998.
[23] S. Ghosal, “A Fast Scalable Algorithm for Discontinuous Optical Flow Estimation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 2, pp. 181-194, Feb. 1996.
[24] R. Deriche, P. Kornprobst, and G. Aubert, “Optical Flow Estimation while Preserving Its Discontinuities: A Variational Approach,” Proc. Asian Conf. Computer Vision, pp. 290-295, 1995.
[25] T. Darrell and A. Pentland, “Robust Estimation of a Multilayered Motion Representation,” Proc. IEEE Workshop Visual Motion, pp. 173-178, 1991.
[26] S. Ayer and H. Sawhney, “Layered Representation of Motion Video Using Robust Maximum Likelihood Estimation of Mixture Models and MDL Encoding,” Proc. IEEE Int'l Conf. Computer Vision, pp. 777-784, 1995.
[27] G. McLachlan and K. Basford, Mixture Models Inference and Applications to Clustering. Marcel Dekker, Inc., 1988.
[28] S. Hsu, P. Anandan, and S. Peleg, “Accurate Computation of Optical Flow by Using Layered Motion Representation,” Proc. Int'l Conf. Pattern Recognition, 1994.
[29] A. Jepson and M. Black, “Mixture Models for Optical Flow Computation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 760-761, 1993.
[30] Y. Weiss, “Smoothness in Layers: Motion Estimation Using Nonparametric Mixture Estimation,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 520-526, 1997.
[31] M. Irani and S. Peleg, “Image Sequence Enhancement Using Multiple Motions Analysis,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 216-221, 1992.
[32] J. Wang and E. Adelson, “Representing Moving Images with Layers,” IEEE Trans. Image Processing, special issue on image sequence compression, vol. 3, no. 5, pp. 625-638, Sept. 1994.
[33] M. Black and P. Anandan, “The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields,” Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[34] D. Fleet, M. Black, and A. Jepson, “Motion Feature Detection Using Steerable Flow Fields,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 274-281, June 1998.
[35] J. Shi and J. Malik, “Motion Segmentation and Tracking Using Normalized Cuts,” Proc. IEEE Int'l Conf. Computer Vision, pp. 1154-1160, Jan. 1998.
[36] Y.-T. Wu, T. Kanade, J. Cohn, and C.-C. Li, “Optical Flow Estimation Using Wavelet Motion Model,” Proc. IEEE Int'l Conf. Computer Vision, pp. 992-998, Jan. 1998.
[37] J. Little, H. Bulthoff, and T. Poggio, “Parallel Optical Flow Using Local Voting,” Proc. IEEE Int'l Conf. Computer Vision, pp. 454-459, 1988.
[38] M.S. Lee, G. Medioni, and P. Mordohai, “Inference of Segmented Overlapping Surfaces from Binocular Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 6, pp. 824-837, June 2002.
[39] W.S. Tong, C.K. Tang, and G. Medioni, “Simultaneous Two-View Epipolar Geometry Estimation and Motion Segmentation by 4D Tensor Voting,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 9, pp. 1167-1184, Sept. 2004.
[40] G. Guy and G. Medioni, “Inference of Surfaces, 3-D Curves, and Junctions from Sparse, Noisy 3-D Data,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 11, pp. 1265-1277, Nov. 1997.
[41] S. Arya, D. Mount, N. Netanyahu, R Silverman, and A. Wu, “An Optimal Algorithm for Approximate Nearest Neighbor Searching in Fixed Dimensions,” J. ACM, vol. 45, no. 6, pp. 891-923, 1998.
[42] D. McReynolds and D. Lowe, “Rigidity Checking of 3D Point Correspondences Under Perspective Projection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 12, pp. 1174-1185, Dec. 1996.
[43] P.H.S. Torr and D.W. Murray, “A Review of Robust Methods to Estimate the Fundamental Matrix,” Proc. Int'l J. Computer Vision, 1997.
[44] P. Smith, T. Drummond, and R. Cipolla, “Layered Motion Segmentation and Depth Ordering by Tracking Edges,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 4, pp. 479-494, Apr. 2004.
[45] P. Smith, “Edge-Based Motion Segmentation,” PhD dissertation, Univ. of Cambridge, 2001.
[46] D. Scharstein and R. Szeliski, “High-Accuracy Stereo Depth Maps Using Structured Light,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 195-202, 2003.

Index Terms:
Motion, image models, scene analysis.
Citation:
Mircea Nicolescu, G?rard Medioni, "A Voting-Based Computational Framework for Visual Motion Analysis and Interpretation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 5, pp. 739-752, May 2005, doi:10.1109/TPAMI.2005.91
Usage of this product signifies your acceptance of the Terms of Use.