This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Motion Detail Preserving Optical Flow Estimation
Sept. 2012 (vol. 34 no. 9)
pp. 1744-1757
ASCII Text
x 
 
Li Xu, Jiaya Jia, Y. Matsushita, "Motion Detail Preserving Optical Flow Estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1744-1757, Sept., 2012.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2011.236,
author = { Li Xu and Jiaya Jia and Y. Matsushita},
title = {Motion Detail Preserving Optical Flow Estimation},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {34},
number = {9},
issn = {0162-8828},
year = {2012},
pages = {1744-1757},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2011.236},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Motion Detail Preserving Optical Flow Estimation
IS - 9
SN - 0162-8828
SP1744
EP1757
EPD - 1744-1757
A1 - Li Xu,
A1 - Jiaya Jia,
A1 - Y. Matsushita,
PY - 2012
KW - optimisation
KW - image motion analysis
KW - image sequences
KW - image motion
KW - optical flow estimation
KW - multiscale variational framework
KW - motion structure
KW - displacement variation
KW - extended coarse-to-fine refinement framework
KW - EC2F refinement framework
KW - motion detail
KW - objective function
KW - outlier
KW - optimization procedure
KW - Middlebury optical flow benchmarkmarking
KW - large-displacement motion
KW - Estimation
KW - Optimization
KW - Optical imaging
KW - Vectors
KW - Adaptive optics
KW - Image color analysis
KW - Robustness
KW - features.
KW - Optical flow
KW - image motion
KW - video motion
KW - variational methods
KW - optimization
VL - 34
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Li Xu, Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Shatin, China
Jiaya Jia, Dept. of Comput. Sci. & Eng., Chinese Univ. of Hong Kong, Shatin, China
Y. Matsushita, Microsoft Res. Asia, Beijing, China
A common problem of optical flow estimation in the multiscale variational framework is that fine motion structures cannot always be correctly estimated, especially for regions with significant and abrupt displacement variation. A novel extended coarse-to-fine (EC2F) refinement framework is introduced in this paper to address this issue, which reduces the reliance of flow estimates on their initial values propagated from the coarse level and enables recovering many motion details in each scale. The contribution of this paper also includes adaptation of the objective function to handle outliers and development of a new optimization procedure. The effectiveness of our algorithm is demonstrated by Middlebury optical flow benchmarkmarking and by experiments on challenging examples that involve large-displacement motion.

[1] L. Alvarez, J. Esclarin, M. Lefebure, and J. Sanchez, "A PDE Model for Computing the Optical Flow," Proc. XVI Congreso de Ecuaciones Diferenciales y Aplicaciones, pp. 1349-1356, 1999.
[2] P. Anandan, "A Computational Framework and an Algorithm for the Measurement of Visual Motion," Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[3] S. Baker, D. Scharstein, J. Lewis, S. Roth, M.J. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Proc. 11th IEEE Int'l Conf. Computer Vision, 2007.
[4] S. Baker, D. Scharstein, J. Lewis, S. Roth, M.J. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Technical Report MSR-TR-2009-179, Microsoft Research, http://vision.middlebury.eduflow/, 2009.
[5] C. Barnes, E. Shechtman, A. Finkelstein, and D.B. Goldman, "Patchmatch: A Randomized Correspondence Algorithm for Structural Image Editing," ACM Trans. Graphics, vol. 28, no. 3, 2009.
[6] J.R. Bergen, P. Anandan, K.J. Hanna, and R. Hingorani, "Hierarchical Model-Based Motion Estimation," Proc. Second European Conf. Computer Vision, vol. 588, pp. 237-252, 1992.
[7] M.J. Black and P. Anandan, "The Robust Estimation of Multiple Motions: Parametric and Piecewise-Smooth Flow Fields," Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, 1996.
[8] M.Z. Brown, D. Burschka, and G.D. Hager, "Advances in Computational Stereo," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 8, pp. 993-1008, Aug. 2003.
[9] T. Brox, C. Bregler, and J. Malik, "Large Displacement Optical Flow," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[10] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, "High Accuracy Optical Flow Estimation Based on a Theory for Warping," Proc. European Conf. Computer Vision, vol. 4, pp. 25-36, 2004.
[11] T. Brox and J. Malik, "Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 500-513, Mar. 2011.
[12] A. Bruhn and J. Weickert, "Towards Ultimate Motion Estimation: Combining Highest Accuracy with Real-Time Performance," Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 749-755, 2005.
[13] A. Bruhn, J. Weickert, and C. Schnörr, "Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods," Int'l J. Computer Vision, vol. 61, no. 3, pp. 211-231, 2005.
[14] D. Geiger and F. Girosi, Parallel and Deterministic Algorithms for MRFs Surface Reconstruction and Integration, A.I. Memo 1114. MIT, 1989.
[15] D. Goldfarb and W. Yin, "Parametric Maximum Flow Algorithms for Fast Total Variation Minimization," Technical Report 07-09, Rice Univ., 2007.
[16] E.T. Hale, W. Yin, and Y. Zhang, "Fixed-Point Continuation for l1-Minimization: Methodology and Convergence," SIAM J. Optimization, vol. 19, no. 3, pp. 1107-1130, 2008.
[17] H.W. Haussecker and D.J. Fleet, "Computing Optical Flow with Physical Models of Brightness Variation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 661-673, June 2001.
[18] B.K.P. Horn and B.G. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, nos. 1-3, pp. 185-203, 1981.
[19] C. Lei and Y.-H. Yang, "Optical Flow Estimation on Coarse-to-Fine Region-Trees Using Discrete Optimization," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[20] V. Lempitsky, S. Roth, and C. Rother, "Fusionflow: Discrete-Continuous Optimization for Optical Flow Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[21] C. Liu, J. Yuen, A. Torralba, J. Sivic, and W.T. Freeman, "Sift Flow: Dense Correspondence across Different Scenes," Proc. European Conf. Computer Vision, vol. 3, pp. 28-42, 2008.
[22] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[23] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. Int'l Joint Conf. Artificial Intelligence, pp. 674-679, 1981.
[24] É. Mémin and P. Pérez, "Hierarchical Estimation and Segmentation of Dense Motion Fields," Int'l J. Computer Vision, vol. 46, no. 2, pp. 129-155, 2002.
[25] S.M. Seitz and S. Baker, "Filter Flow," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[26] S. Roth and M.J. Black, "On the Spatial Statistics of Optical Flow," Proc. 10th IEEE Conf. Computer Vision, pp. 42-49, 2005.
[27] C. Rother, V. Kolmogorov, V.S. Lempitsky, and M. Szummer, "Optimizing Binary MRFs via Extended Roof Duality," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[28] P. Sand and S.J. Teller, "Particle Video: Long-Range Motion Estimation Using Point Trajectories," Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 2195-2202, 2006.
[29] D. Scharstein and C. Pal, "Learning Conditional Random Fields for Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2007.
[30] L. Sigal, A.O. Balan, and M.J. Black, "Humaneva: Synchronized Video and Motion Capture Data Set and Baseline Algorithm for Evaluation of Articulated Human Motion," Int'l J. Computer Vision, vol. 87, nos. 1/2, pp. 4-27, 2010.
[31] M. Sizintsev and R.P. Wildes, "Efficient Stereo with Accurate 3-D Boundaries," Proc. British Machine Vision Conf., pp. 237-246, 1996.
[32] F. Steinbrücker and T. Pock, "Large Displacement Optical Flow Computation Without Warping," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[33] D. Sun, S. Roth, and M.J. Black, "Secrets of Optical Flow Estimation and Their Principles," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[34] D. Sun, S. Roth, J.P. Lewis, and M.J. Black, "Learning Optical Flow," Proc. 10th European Conf. Computer Vision, vol. 3, pp. 83-97, 2008.
[35] Y. Wang, J. Yang, W. Yin, and Y. Zhang, "A New Alternating Minimization Algorithm for Total Variation Image Reconstruction," SIAM J. Imaging Sciences, vol. 1, no. 3, pp. 248-272, 2008.
[36] A. Wedel, D. Cremers, T. Pock, and H. Bischof, "Structure- and Motion-Adaptive Regularization for High Accuracy Optic Flow," Proc. 12th IEEE Int'l Conf. Computer Vision, 2009.
[37] A. Wedel, T. Pock, C. Zach, H. Bischof, and D. Cremers, "An Improved Algorithm for TV-l1 Optical Flow Computation," Proc. Dagstuhl Visual Motion Analysis Workshop, 2008.
[38] M. Werlberger, T. Pock, and H. Bischof, "Motion Estimation with Non-Local Total Variation Regularization," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[39] M. Werlberger, W. Trobin, T. Pock, A. Wedel, D. Cremers, and H. Bischof, "Anisotropic Huber-l1 Optical Flow," Proc. British Machine Vision Conf., 2009.
[40] J. Xiao, H. Cheng, H.S. Sawhney, C. Rao, and M.A. Isnardi, "Bilateral Filtering-Based Optical Flow Estimation with Occlusion Detection," Proc. Ninth European Conf. Computer Vision, vol. 1, pp. 211-224, 2006.
[41] L. Xu, J. Chen, and J. Jia, "A Segmentation Based Variational Model for Accurate Optical Flow Estimation," Proc. 10th European Conf. Computer Vision, vol. 1, pp. 671-684, 2008.
[42] L. Xu, J. Jia, and Y. Matsushita, "Motion Detail Preserving Optical Flow Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[43] C. Zach, T. Pock, and H. Bischof, "A Duality Based Approach for Realtime TV-l1 Optical Flow," Proc. 29th DAGM Symp. Pattern Recognition, pp. 214-223, 2007.
[44] H. Zimmer, A. Bruhn, and J. Weickert, "Optic Flow in Harmony," Int'l J. Computer Vision, vol. 93, no. 3, pp. 368-388, 2011.
[45] H. Zimmer, A. Bruhn, J. Weickert, B.R.L. Valgaerts, and A. Salgado, and H.-P. Seidel, "Complementary Optic Flow," Proc. Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2009.

Index Terms:
optimisation,image motion analysis,image sequences,image motion,optical flow estimation,multiscale variational framework,motion structure,displacement variation,extended coarse-to-fine refinement framework,EC2F refinement framework,motion detail,objective function,outlier,optimization procedure,Middlebury optical flow benchmarkmarking,large-displacement motion,Estimation,Optimization,Optical imaging,Vectors,Adaptive optics,Image color analysis,Robustness,features.,Optical flow,image motion,video motion,variational methods,optimization
Citation:
Li Xu, Jiaya Jia, Y. Matsushita, "Motion Detail Preserving Optical Flow Estimation," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 9, pp. 1744-1757, Sept. 2012, doi:10.1109/TPAMI.2011.236
Usage of this product signifies your acceptance of the Terms of Use.