This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Optical Flow 3D Segmentation and Interpretation: A Variational Method with Active Curve Evolution and Level Sets
November 2006 (vol. 28 no. 11)
pp. 1818-1829
This study investigates a variational, active curve evolution method for dense three-dimentional (3D) segmentation and interpretation of optical flow in an image sequence of a scene containing moving rigid objects viewed by a possibly moving camera. This method jointly performs 3D motion segmentation, 3D interpretation (recovery of 3D structure and motion), and optical flow estimation. The objective functional contains two data terms for each segmentation region, one based on the motion-only equation which relates the essential parameters of 3D rigid body motion to optical flow, and the other on the Horn and Schunck optical flow constraint. It also contains two regularization terms for each region, one for optical flow, the other for the region boundary. The necessary conditions for a minimum of the functional result in concurrent 3D-motion segmentation, by active curve evolution via level sets, and linear estimation of each region essential parameters and optical flow. Subsequently, the screw of 3D motion and regularized relative depth are recovered analytically for each region from the estimated essential parameters and optical flow. Examples are provided which verify the method and its implementation.

[1] J. Aggarwal and N. Nandhakumar, “On the Computation of Motion from a Sequence of Images: A Review,” Proc. IEEE, vol. 76, pp. 917-935, 1988.
[2] T. Huang and A. Netravali, “Motion and Structure from Feature Correspondences: A Review,” Proc. IEEE, vol. 82, pp. 252-268, 1994.
[3] O. Faugeras, Three Dimensional Computer Vision: A Geometric Viewpoint. MIT Press, 1993.
[4] A. Mitiche, Computational Analysis of Visual Motion. Plenum, 1994.
[5] A. Zisserman and R. Hartley, Multiple View Geometry in Computer Vision. Cambridge Univ. Press, 2000.
[6] J. Morel and S. Solimini, Variational Methods in Image Processing. Birkhauser, 1994.
[7] G. Aubert and P. Kornprobst, Mathematical Problems in Image Processing. Springer, 2002.
[8] S. Osher, N. Paragios, Geometric Level Set Methods in Imaging, Vision, and Graphics. Springer, 2003.
[9] G. Adiv, “Determining Three-Dimensional Motion and Structure from Optical Flow Generated by Several Moving Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, no. 4, pp.384-401, 1985.
[10] B. Horn and E. Weldon, “Direct Methods for Recovering Motion,” Int'l J. Computer Vision, vol. 2, no. 2, pp. 51-76, 1988.
[11] S. Srinivasan, “Extracting Structure from Optical Flow Using the Fast Error Search Technique,” Int'l J. Computer Vision, vol. 37, no. 3, pp. 203-230, 2000.
[12] Y. Hung and H. Ho, “A Kalman Filter Approach to Direct Depth Estimation Incorporating Surface Structure,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 21, no. 6, pp. 570-575, June 1999.
[13] B. Shahraray and M. Brown, “Robust Depth Estimation from Optical Flow,” Proc. Int'l Conf. Computer Vision, pp. 641-650, 1988.
[14] M. Taalebinezhaad, “Direct Recovery of Motion and Shape in the General Case by Fixation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 8, pp. 847-853, Aug. 1992.
[15] E. De Micheli and F. Giachero, “Motion and Structure from One Dimensional Optical Flow,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 962-965, 1994.
[16] N. Gupta and N. Kanal, “3-D Motion Estimation from Motion Field,” Artificial Intelligence, no. 78, pp. 45-86, 1995.
[17] Y. Xiong and S. Shafer, “Dense Structure from a Dense Optical Flow,” Proc. Int'l Conf. Computer Vision and Image Understanding, pp. 222-245, 1998.
[18] T. Brodsky, C. Fermuller, and Y. Aloimonos, “Structure from Motion: Beyond the Epipolar Constraint,” Int'l J. Computer Vision, vol. 37, no. 3, pp. 231-258, 2000.
[19] H. Liu, R. Chellappa, and A. Rosenfeld, “A Hierarchical Approach for Obtaining Structure from Two-Frame Optical Flow,” Proc. Workshop Motion and Video Computing, 2002.
[20] D. Heeger and A. Jepson, “Subspace Methods for Recovering Rigid Motion I: Algorithm and Implementation,” Int'l J. Computer Vision, vol. 7, no. 2, pp. 95-117, 1992.
[21] A. Bruss and B. Horn, “Passive Navigation,” Computer Graphics and Image Processing, pp. 3-20, 1983.
[22] W. MacLean, A. Jepson, and R. Frecher, “Recovery of Egomotion and Segmentation of Independent Object Motion Using the EM Algorithm,” Proc. British Machine Vision Conf., pp. 13-16, 1994.
[23] S. Fejes and L. Davis, “What Can Projections of Flow Fields Tell Us about Visual Motion,” Proc. Int'l Conf. Computer Vision, pp. 979-986, 1998.
[24] J. Weber and J. Malik, “Rigid Body Segmentation and Shape Description from Dense Optical Flow under Weak Perspective,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, pp. 139-143, Feb. 1997.
[25] H. Sekkati and A. Mitiche, “Dense 3D Interpretation of Image Sequences: A Variational Approach Using Anisotropic Diffusion,” Proc. Int'l Conf. Image Analysis and Processing, pp. 424-429, 2003.
[26] A. Mitiche and S. Hadjres, “MDL Estimation of a Dense Map of Relative Depth and 3D Motion from a Temporal Sequence of Images,” Pattern Analysis and Applications, no. 6, pp. 78-87, 2003.
[27] H. Sekkati and A. Mitiche, “Concurrent 3D-Motion Segmentation and 3D Interpretation of Temporal Sequences of Monocular Images,” IEEE Trans. Image Processing, vol. 15, pp. 641-653, 2006.
[28] Y. Leclerc, “Constructing Simple Stable Descriptions for Image Partitioning,” Int'l J. Computer Vision, no. 3, pp. 73-102, 1996.
[29] J. Aloimonos and C. Brown, “Direct Processing of Curvilinear Sensor Motion from a Sequence of Perspective Images,” Proc. IEEE Workshop Computer Vision: Representation and Analysis, pp. 72-77, 1984.
[30] S. Negahdaripour and B. Horn, “Direct Passive Navigation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 9, no. 1, pp.168-176, 1987.
[31] H. Longuet-Higgins and K. Prazdny, “The Interpretation of a Moving Retinal Image,” Royal Soc. London, Series B, vol. 208, pp.385-397, 1981.
[32] X. Zhuang and R. Haralick, “Rigid Body Motion and the Optical Flow Image,” Proc. First Int'l Conf. Artificial Intelligence Applications, pp.366-375, 1984.
[33] B. Horn and B. Schunk, “Determining Optical Flow,” Artifcial Intelligence, no. 17, pp. 185-203, 1981.
[34] A. Mitiche and A. Mansouri, “On Convergence of the Horn and Schunck Optical Flow Estimation Method,” IEEE Trans. Image Processing, vol. 13, no. 11, pp. 1473-1490, 2004.
[35] D. Mumford and J. Shah, “Optimal Approximation by Piecewise Smooth Functions and Associated Variational Problems,” Comm. Pure and Applied Math., no. 42, pp. 577-685, 1989.
[36] G. Aubert, R. Deriche, and P. Kornprobst, “Computing Optical Flow via Variational Techniques,” SIAM J. Applied Math., vol. 60, no. 1, pp. 156-182, 1999.
[37] S. Zhu and A. Yuille, “Region Competition: Unifying Snakes, Region Growing, and Bayes/MDL for Multiband Image Segmentation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 9, pp. 884-900, Sept. 1996.
[38] G. Aubert, M. Barlaud, O. Faugeras, and S. Jehan-Besson, “Image Segmentation Using Active Contours: Calculus of Variations or Shape Gradients?” SIAM J. Applied Math., vol. 63, no. 6, pp. 2128-2154, 2003.
[39] G. Forsythe, M. Malcolm, and C. Moler, Computer Methods for Mathematical Computations. Prentice-Hall, 1977.
[40] J. Sethian, Level Set Methos and Fast Marching Methods. Cambridge Univ. Press, 1999.
[41] M. Rousson and R. Deriche, “A Variational Framework for Active and Adaptive Segmentation,” Proc. IEEE Workshop Motion and Video Computing, pp. 56-61, 2002.
[42] T. Chan and L. Vese, “An Active Contour Model without Edges,” Proc. Int'l Conf. Scale-Space Theories in Computer Vision, pp. 141-151, 1999.
[43] C. Vazquez, A. Mitiche, and I. Ben Ayed, “Image Segmentation as Regularized Clustering: A Fully Global Curve Evolution Method,” Proc. Int'l Conf. Image Processing, pp. 3467-3470, 2004.
[44] A. Mansouri, A. Mitiche, and C. Vazquez, “Multiregion Competition: A Level Set Extention of Region Competition to Multiple Region Image Partioning,” Computer Vision and Image Understanding, vol. 101, pp. 137-150, 2006.
[45] C. Debrunner and N. Ahuja, “Segmentation and Factorization-Based Motion and Structure Estimation for Long Image Sequences,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 2, pp. 206-211, Feb. 1998.

Index Terms:
Optical flow, 3D segmentation, 3D interpretation, level sets, image sequence analysis.
Citation:
Amar Mitiche, Hicham Sekkati, "Optical Flow 3D Segmentation and Interpretation: A Variational Method with Active Curve Evolution and Level Sets," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 11, pp. 1818-1829, Nov. 2006, doi:10.1109/TPAMI.2006.232
Usage of this product signifies your acceptance of the Terms of Use.