The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.09 - Sept. (2012 vol.34)
pp: 1744-1757
Li Xu , Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Shatin, China
Jiaya Jia , Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Shatin, China
Y. Matsushita , Microsoft Res. Asia, Beijing, China
ABSTRACT
A common problem of optical flow estimation in the multiscale variational framework is that fine motion structures cannot always be correctly estimated, especially for regions with significant and abrupt displacement variation. A novel extended coarse-to-fine (EC2F) refinement framework is introduced in this paper to address this issue, which reduces the reliance of flow estimates on their initial values propagated from the coarse level and enables recovering many motion details in each scale. The contribution of this paper also includes adaptation of the objective function to handle outliers and development of a new optimization procedure. The effectiveness of our algorithm is demonstrated by Middlebury optical flow benchmarkmarking and by experiments on challenging examples that involve large-displacement motion.
INDEX TERMS
optimisation, image motion analysis, image sequences, image motion, optical flow estimation, multiscale variational framework, motion structure, displacement variation, extended coarse-to-fine refinement framework, EC2F refinement framework, motion detail, objective function, outlier, optimization procedure, Middlebury optical flow benchmarkmarking, large-displacement motion, Estimation, Optimization, Optical imaging, Vectors, Adaptive optics, Image color analysis, Robustness, features., Optical flow, image motion, video motion, variational methods, optimization
CITATION
Li Xu, Jiaya Jia, Y. Matsushita, "Motion Detail Preserving Optical Flow Estimation", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 9, pp. 1744-1757, Sept. 2012, doi:10.1109/TPAMI.2011.236
REFERENCES
[1] L. Alvarez, J. Esclarin, M. Lefebure, and J. Sanchez, "A PDE Model for Computing the Optical Flow," Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, pp. 1349-1356, 1999.
[2] P. Anandan, "A Computational Framework and an Algorithm for the Measurement of Visual Motion," Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[3] S. Baker, D. Scharstein, J. Lewis, S. Roth, M.J. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[4] S. Baker, D. Scharstein, J. Lewis, S. Roth, M.J. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Technical Report MSR-TR-2009-179, Microsoft Research, http://vision.middlebury.eduflow/, 2009.
[5] C. Barnes, E. Shechtman, A. Finkelstein, and D.B. Goldman, "Patchmatch: A Randomized Correspondence Algorithm for Structural Image Editing," ACM Trans. Graphics, vol. 28, no. 3, 2009.
[6] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, "Hierarchical Model-Based Motion Estimation," Proc. Second European Conf. Computer Vision, vol. 588, pp. 237-252, 1992.
[7] M.J. Black and P. Anandan, "The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields," Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[8] M.Z. Brown, D. Burschka, and G.D. Hager, "Advances in Computational Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003.
[9] T. Brox, C. Bregler, and J. Malik, "Large Displacement Optical Flow," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[10] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High Accuracy Optical Flow Estimation Based on a Theory for Warping," Proc. European Conf. Computer Vision, vol. 4, pp. 25-36, 2004.
[11] T. Brox and J. Malik, "Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 500-513, Mar. 2011.
[12] A. Bruhn and J. Weickert, "Towards Ultimate Motion Estimation: Combining Highest Accuracy with Real-Time Performance," Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 749-755, 2005.
[13] A. Bruhn, J. Weickert, and C. Schnörr, "Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods," Int'l J. Computer Vision, vol. 61, no. 3, pp. 211-231, 2005.
[14] D. Geiger and F. Girosi, Parallel and Deterministic Algorithms for MRFs Surface Reconstruction and Integration, A.I. Memo 1114. MIT, 1989.
[15] D. Goldfarb and W. Yin, "Parametric Maximum Flow Algorithms for Fast Total Variation Minimization," Technical Report 07-09, Rice Univ., 2007.
[16] E.T. Hale, W. Yin, and Y. Zhang, "Fixed-Point Continuation for l1-Minimization: Methodology and Convergence," SIAM J. Optimization, vol. 19, no. 3, pp. 1107-1130, 2008.
[17] H.W. Haussecker and D.J. Fleet, "Computing Optical Flow with Physical Models of Brightness Variation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 661-673, June 2001.
[18] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, nos. 1-3, pp. 185-203, 1981.
[19] C. Lei and Y.-H. Yang, "Optical Flow Estimation on Coarse-to-Fine Region-Trees Using Discrete Optimization," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[20] V. Lempitsky, S. Roth, and C. Rother, "Fusionflow: Discrete-Continuous Optimization for Optical Flow Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[21] C. Liu, J. Yuen, A. Torralba, J. Sivic, and W.T. Freeman, "Sift Flow: Dense Correspondence across Different Scenes," Proc. European Conf. Computer Vision, vol. 3, pp. 28-42, 2008.
[22] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[23] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981.
[24] É. Mémin and P. Pérez, "Hierarchical Estimation and Segmentation of Dense Motion Fields," Int'l J. Computer Vision, vol. 46, no. 2, pp. 129-155, 2002.
[25] S.M. Seitz and S. Baker, "Filter Flow," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[26] S. Roth and M.J. Black, "On the Spatial Statistics of Optical Flow," Proc. 10th IEEE Conf. Computer Vision, pp. 42-49, 2005.
[27] C. Rother, V. Kolmogorov, V.S. Lempitsky, and M. Szummer, "Optimizing Binary MRFs via Extended Roof Duality," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[28] P. Sand and S.J. Teller, "Particle Video: Long-Range Motion Estimation Using Point Trajectories," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2195-2202, 2006.
[29] D. Scharstein and C. Pal, "Learning Conditional Random Fields for Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[30] L. Sigal, A.O. Balan, and M.J. Black, "Humaneva: Synchronized Video and Motion Capture Data Set and Baseline Algorithm for Evaluation of Articulated Human Motion," Int'l J. Computer Vision, vol. 87, nos. 1/2, pp. 4-27, 2010.
[31] M. Sizintsev and R.P. Wildes, "Efficient Stereo with Accurate 3-D Boundaries," Proc. British Machine Vision Conf., pp. 237-246, 1996.
[32] F. Steinbrücker and T. Pock, "Large Displacement Optical Flow Computation Without Warping," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[33] D. Sun, S. Roth, and M.J. Black, "Secrets of Optical Flow Estimation and Their Principles," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[34] D. Sun, S. Roth, J.P. Lewis, and M.J. Black, "Learning Optical Flow," Proc. 10th European Conf. Computer Vision, vol. 3, pp. 83-97, 2008.
[35] Y. Wang, J. Yang, W. Yin, and Y. Zhang, "A New Alternating Minimization Algorithm for Total Variation Image Reconstruction," SIAM J. Imaging Sciences, vol. 1, no. 3, pp. 248-272, 2008.
[36] A. Wedel, D. Cremers, T. Pock, and H. Bischof, "Structure- and Motion-Adaptive Regularization for High Accuracy Optic Flow," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[37] A. Wedel, T. Pock, C. Zach, H. Bischof, and D. Cremers, "An Improved Algorithm for TV-l1 Optical Flow Computation," Proc. Dagstuhl Visual Motion Analysis Workshop, 2008.
[38] M. Werlberger, T. Pock, and H. Bischof, "Motion Estimation with Non-Local Total Variation Regularization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[39] M. Werlberger, W. Trobin, T. Pock, A. Wedel, D. Cremers, and H. Bischof, "Anisotropic Huber-l1 Optical Flow," Proc. British Machine Vision Conf., 2009.
[40] J. Xiao, H. Cheng, H.S. Sawhney, C. Rao, and M.A. Isnardi, "Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection," Proc. Ninth European Conf. Computer Vision, vol. 1, pp. 211-224, 2006.
[41] L. Xu, J. Chen, and J. Jia, "A Segmentation Based Variational Model for Accurate Optical Flow Estimation," Proc. 10th European Conf. Computer Vision, vol. 1, pp. 671-684, 2008.
[42] L. Xu, J. Jia, and Y. Matsushita, "Motion Detail Preserving Optical Flow Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[43] C. Zach, T. Pock, and H. Bischof, "A Duality Based Approach for Realtime TV-l1 Optical Flow," Proc. 29th DAGM Symp. Pattern Recognition, pp. 214-223, 2007.
[44] H. Zimmer, A. Bruhn, and J. Weickert, "Optic Flow in Harmony," Int'l J. Computer Vision, vol. 93, no. 3, pp. 368-388, 2011.
[45] H. Zimmer, A. Bruhn, J. Weickert, B.R.L. Valgaerts, and A. Salgado, and H.-P. Seidel, "Complementary Optic Flow," Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2009.
37 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool