This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Motion Segmentation Using Occlusions
June 2005 (vol. 27 no. 6)
pp. 988-992
We examine the key role of occlusions in finding independently moving objects instantaneously in a video obtained by a moving camera with a restricted field of view. In this problem, the image motion is caused by the combined effect of camera motion (egomotion), structure (depth), and the independent motion of scene entities. For a camera with a restricted field of view undergoing a small motion between frames, there exists, in general, a set of 3D camera motions compatible with the observed flow field even if only a small amount of noise is present, leading to ambiguous 3D motion estimates. If separable sets of solutions exist, motion-based clustering can detect one category of moving objects. Even if a single inseparable set of solutions is found, we show that occlusion information can be used to find ordinal depth, which is critical in identifying a new class of moving objects. In order to find ordinal depth, occlusions must not only be known, but they must also be filled (grouped) with optical flow from neighboring regions. We present a novel algorithm for filling occlusions and deducing ordinal depth under general circumstances. Finally, we describe another category of moving objects which is detected using cardinal comparisons between structure from motion and structure estimates from another source (e.g., stereo).

[1] M. Bober and J. Kittler, “Robust Motion Analysis,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 947-952, 1994.
[2] P.J. Burt, J.R. Bergen, R. Hingorani, R. Kolczynski, W.A. Lee, A. Leung, J. Lubin, and H. Shvaytser, “Object Tracking with a Moving Camera,” Proc. IEEE Workshop Visual Motion, pp. 2-12, 1989.
[3] J.-M. Odobez and P. Bouthemy, “MRF-Based Motion Segmentation Exploiting a 2D Motion Model and Robust Estimation,” Proc. Int'l Conf. Image Processing, vol. 3, pp. 628-631, 1995.
[4] Y. Weiss, “Smoothness in Layers: Motion Segmentation Using Nonparametric Mixture Estimation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 520-526, 1997.
[5] G. Adiv, “Determining 3D Motion and Structure from Optical Flow Generated by Several Moving Objects,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 7, pp. 384-401, 1985.
[6] Z. Zhang, O.D. Faugeras, and N. Ayache, “Analysis of a Sequence of Stereo Scenes Containing Multiple Moving Objects Using Rigidity Constraints,” Proc. Second Int'l Conf. Computer Vision, pp. 177-186, 1988.
[7] W.B. Thompson and T.-C. Pong, “Detecting Moving Objects,” Int'l J. Computer Vision, vol. 4, pp. 39-57, 1990.
[8] R.C. Nelson, “Qualitative Detection of Motion by a Moving Observer,” Int'l J. Computer Vision, vol. 7, pp. 33-46, 1991.
[9] D. Sinclair, “Motion Segmentation and Local Structure,” Proc. Fourth Int'l Conf. Computer Vision, pp. 366-373, 1993.
[10] P.H.S. Torr and D.W. Murray, “Stochastic Motion Clustering,” Proc. Third European Conf. Computer Vision, pp. 328-337, 1994.
[11] J. Costeira and T. Kanade, “A Multi-Body Factorization Method for Motion Analysis,” Proc. Int'l Conf. Computer Vision, pp. 1071-1076, 1995.
[12] J. Weber and J. Malik, “Rigid Body Segmentation and Shape Description from Sense Optical Flow under Weak Perspective,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 2, Feb. 1997.
[13] B. Triggs, P. McLauchlan, R. Hartley, and A. Fitzgibbon, “Bundle Adjustment— A Modern Synthesis,” Proc. Vision Algorithms: Theory and Practice, B. Triggs, A. Zisserman, and R. Szeliski, eds., 2000.
[14] S. Ayer, P. Schroeter, and J. Bigün, “Segmentation of Moving Objects by Robust Motion Parameter Estimation over Multiple Frames,” Proc. Third European Conf. Computer Vision, pp. 316-327, 1994.
[15] C.S. Wiles and M. Brady, “Closing the Loop on Multiple Motions,” Proc. Fifth Int'l Conf. Computer Vision, pp. 308-313, 1995.
[16] Q.F. Zheng and R. Chellappa, “Motion Detection in Image Sequences Acquired from a Moving Platform,” Proc. IEEE Int'l Conf. Acoustics, Speech, and Signal Processing, pp. 201-204, 1993.
[17] P.H.S. Torr, “Geometric Motion Segmentation and Model Selection,” Philosophical Trans. Royal Soc. A, J. Lasenby, A. Zisserman, R. Cipolla, and H. Longuet-Higgins, eds., pp. 1321-1340, 1998.
[18] M. Irani and P. Anandan, “A Unified Approach to Moving Object Detection in 2D and 3D Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, pp. 577-589, 1998.
[19] H.S. Sawhney, Y. Guo, and R. Kumar, “Independent Motion Detection in 3D Scenes,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, pp. 1191-1199, 2000.
[20] G. Adiv, “Inherent Ambiguities in Recovering 3D Motion and Structure from a Noisy Flow Field,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 11, pp. 477-489, 1989.
[21] K. Daniilidis and M.E. Spetsakis, “Understanding Noise Sensitivity in Structure from Motion,” Visual Navigation: From Biological Systems to Unmanned Ground Vehicles, chapter 4, Y. Aloimonos, ed., Lawrence Erlbaum Associates, 1997.
[22] S.J. Maybank, “A Theoretical Study of Optical Flow,” PhD dissertation, Univ. London, 1987.
[23] D.J. Heeger and A.D. Jepson, “Subspace Methods for Recovering Rigid Motion I: Algorithm and Implementation,” Int'l J. Computer Vision, vol. 7, pp. 95-117, 1992.
[24] C. Fermüller and Y. Aloimonos, “Observability of 3D Motion,” Int'l J. Computer Vision, vol. 37, pp. 43-63, 2000.
[25] T. Brodský, C. Fermüller, and Y. Aloimonos, “Structure from Motion: Beyond the Epipolar Constraint,” Int'l J. Computer Vision, vol. 37, pp. 231-258, 2000.
[26] T. Darrell and D. Fleet, “Second-Order Method for Occlusion Relationships in Motion Layers,” Technical Report 314, MIT Media Lab, 1995.
[27] L. Bergen and F. Meyer, “Motion Segmentation and Depth Ordering Based on Morphological Segmentation,” Proc. European Conf. Computer Vision, pp. 531-547, 1998.
[28] D. Tweed and A. Calway, “Integrated Segmentation and Depth Ordering of Motion Layers in Image Sequences,” Proc. British Machine Vision Conf., 2000.
[29] B.S. Reddy and B.N. Chatterji, “An FFT-Based Technique for Translation, Rotation and Scale-Invariant Image Registration,” IEEE Trans. Image Processing, vol. 5, no. 8, pp. 1266-1271, Aug. 1996.
[30] D.J. Fleet, “Disparity from Local Weighted Phase-Correlation,” Proc. Int'l Conf. Systems, Man, and Cybernetics, pp. 48-56, Oct. 1994.
[31] A.S. Ogale, “The Compositional Character of Visual Correspondence,” PhD dissertation, Univ. of Maryland, College Park, Aug. 2004.
[32] V. Kolmogorov and R. Zabih, “Computing Visual Correspondence with Occlusions Using Graph Cuts,” Proc. Int'l Conf. Computer Vision, vol. 2, pp. 508-515, 2001.
[33] C. Silva and J. Santos-Victor, “Motion from Occlusions,” Robotics and Autonomous Systems, vol. 35, nos. 3-4, pp. 153-162, June 2001.

Index Terms:
Motion, occlusions, segmentation, ordinal depth, video analysis.
Citation:
Abhijit S. Ogale, Cornelia Fermüller, Yiannis Aloimonos, "Motion Segmentation Using Occlusions," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 6, pp. 988-992, June 2005, doi:10.1109/TPAMI.2005.123
Usage of this product signifies your acceptance of the Terms of Use.