The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.05 - May (2013 vol.35)
pp: 1107-1120
Oisin Mac Aodha , Dept. of Comput. Sci., Univ. Coll. London, London, UK
A. Humayun , Sch. of Interactive Comput., Georgia Inst. of Technol., Atlanta, GA, USA
M. Pollefeys , Dept. of Comput. Sci., ETH Zurich, Zurich, Switzerland
G. J. Brostow , Dept. of Comput. Sci., Univ. Coll. London, London, UK
ABSTRACT
We present a supervised learning-based method to estimate a per-pixel confidence for optical flow vectors. Regions of low texture and pixels close to occlusion boundaries are known to be difficult for optical flow algorithms. Using a spatiotemporal feature vector, we estimate if a flow algorithm is likely to fail in a given region. Our method is not restricted to any specific class of flow algorithm and does not make any scene specific assumptions. By automatically learning this confidence, we can combine the output of several computed flow fields from different algorithms to select the best performing algorithm per pixel. Our optical flow confidence measure allows one to achieve better overall results by discarding the most troublesome pixels. We illustrate the effectiveness of our method on four different optical flow algorithms over a variety of real and synthetic sequences. For algorithm selection, we achieve the top overall results on a large test set, and at times even surpass the results of the best algorithm among the candidates.
INDEX TERMS
Optical imaging, Adaptive optics, Optical variables measurement, Vectors, Prediction algorithms, Supervised learning, Accuracy, algorithm selection, Optical flow, confidence measure, Random Forest, synthetic data
CITATION
Oisin Mac Aodha, A. Humayun, M. Pollefeys, G. J. Brostow, "Learning a Confidence Measure for Optical Flow", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.35, no. 5, pp. 1107-1120, May 2013, doi:10.1109/TPAMI.2012.171
REFERENCES
[1] S. Baker, D. Scharstein, J.P. Lewis, S. Roth, M.J. Black, and R. Szeliski, "A Database and Evaluation Methodology for Optical Flow," Int'l J. Computer Vision, vol. 92, pp. 1-31, 2011.
[2] O. Mac Aodha, G.J. Brostow, and M. Pollefeys, "Segmenting Video into Classes of Algorithm-Suitability," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[3] D. Sun, S. Roth, and M. Black, "Secrets of Optical Flow Estimation and Their Principles," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[4] J. Kybic and C. Nieuwenhuis, "Bootstrap Optical Flow Confidence and Uncertainty Measure," Computer Vision and Image Understanding, vol. 115, no. 10, pp. 1449-1462, 2011.
[5] C. Kondermann, R. Mester, and C. Garbe, "A Statistical Confidence Measure for Optical Flows," Proc. European Conf. Computer Vision, 2008.
[6] S. Gould, R. Fulton, and D. Koller, "Decomposing a Scene into Geometric and Semantically Consistent Regions," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[7] A. Farhadi, I. Endres, D. Hoiem, and D. Forsyth, "Describing Objects by Their Attributes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[8] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, "Performance of Optical Flow Techniques," Int'l J. Computer Vision, vol. 12, pp. 43-77, 1994.
[9] E. Simoncelli, E. Adelson, and D. Heeger, "Probability Distributions of Optical Flow," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1991.
[10] P. Anandan, "A Computational Framework and an Algorithm for the Measurement of Visual Motion," Int'l J. Computer Vision, vol. 2, pp. 283-310, 1989.
[11] S. Uras, F. Girosi, A. Verri, and V. Torre, "A Computational Approach to Motion Perception," Biological Cybernetics, vol. 60, pp. 79-87, 1988.
[12] B. Jähne, H. Haussecker, and P. Geissler, Handbook of Computer Vision and Applications: Signal Processing and Pattern Recognition, vol. 2. Academic Press, 1999.
[13] A. Wedel, D. Cremers, T. Pock, and H. Bischof, "Structure- and Motion-Adaptive Regularization for High Accuracy Optic Flow," Proc. IEEE Int'l Conf. Computer Vision, 2009.
[14] H. Zimmer, A. Bruhn, J. Weickert, L. Valgaerts, A. Salgado, B. Rosenhahn, and H.-P. Seidel, "Complementary Optic Flow," Proc. Seventh Int'l Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition, 2009.
[15] A. Bruhn and J. Weickert, "A Confidence Measure for Variational Optic Flow Methods," Geometric Properties for Incomplete Data, pp. 283-298, Springer, 2006.
[16] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, "Performance of Optical Flow Techniques," Technical Report TR299, Dept. of Computer Science, Univ. of Western Ontario, 1992.
[17] M. Gong and Y.-H. Yang, "Estimate Large Motions Using the Reliability-Based Motion Estimation Algorithm," Int'l J. Computer Vision, vol. 68, pp. 319-330, 2006.
[18] C. Kondermann, D. Kondermann, B. Jähne, and C. Garbe, "An Adaptive Confidence Measure for Optical Flows Based on Linear Subspace Projections," Proc. DAGM Conf. Pattern Recognition, pp. 132-141, 2007.
[19] A. Humayun, O. Mac Aodha, and G.J. Brostow, "Learning to Find Occlusion Regions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[20] S. Gehrig and T. Scharwachter, "A Real-Time Multi-Cue Framework for Determining Optical Flow Confidence," Proc. IEEE Int'l Conf. Computer Vision Workshops, 2011.
[21] A. Bainbridge-Smith and R. Lane, "Measuring Confidence in Optical Flow Estimation," Electronics Letters, vol. 32, no. 10, pp. 882-884, 1996.
[22] M. Reynolds, J. Dobos, L. Peel, T. Weyrich, and G. Brostow, "Capturing Time-of-Flight Data with Confidence," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[23] B. Li, R. Xiao, Z. Li, R. Cai, B.-L. Lu, and L. Zhang, "Rank-Sift: Learning to Rank Local Interest Points," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[24] V.C. Raykar, S. Yu, L.H. Zhao, A. Jerebko, C. Florin, G.H. Valadez, L. Bogoni, and L. Moy, "Supervised Learning from Multiple Experts: Whom to Trust When Everyone Lies a Bit," Proc. Int'l Conf. Machine Learning, 2009.
[25] X. Yong, D. Feng, Z. Rongchun, and M. Petrou, "Learning-Based Algorithm Selection for Image Segmentation," Pattern Recognition Letters, vol. 26, pp. 1059-1068, 2005.
[26] B. Stenger, T. Woodley, and R. Cipolla, "Learning to Track with Multiple Observers," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2009.
[27] N. Alt, S. Hinterstoisser, and N. Navab, "Rapid Selection of Reliable Templates for Visual Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2010.
[28] B. Peng and O. Veksler, "Parameter Selection for Graph Cut Based Image Segmentation," Proc. British Machine Vision Conf., 2008.
[29] M. Muja and D.G. Lowe, "Fast Approximate Nearest Neighbors with Automatic Algorithm Configuration," Proc. Int'l Conf. Computer Vision Theory and Applications, 2009.
[30] D.G. Lowe, "Distinctive Image Features from Scale-Invariant Keypoints," Int'l J. Computer Vision, vol. 60, pp. 91-110, 2004.
[31] V. Lempitsky, S. Roth, and C. Rother, "Fusionflow: Discrete-Continuous Optimization for Optical Flow Estimation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[32] B.D. Lucas and T. Kanade, "An Iterative Image Registration Technique with an Application to Stereo Vision," Proc. Int'l Joint Conf. Artificial Intelligence, 1981.
[33] B. Horn and B.G. Schunck, "Determining Optical Flow," Artificial Intelligence, vol. 17, pp. 185-204, 1981.
[34] A. Bruhn, J. Weickert, and C. Schnörr, "Lucas/Kanade Meets Horn/Schunck: Combining Local and Global Optic Flow Methods," Int'l J. Computer Vision, vol. 61, pp. 211-231, 2005.
[35] S.C. Zhu, Y.N. Wu, and D. Mumford, "Filters, Random Fields and Maximum Entropy (FRAME): Towards a Unified Theory for Texture Modeling," Int'l J. Computer Vision, vol. 27, pp. 107-126, 1998.
[36] S. Roth and M.J. Black, "Fields of Experts," Int'l J. Computer Vision, vol. 82, pp. 205-229, 2009.
[37] O. Woodford, I.D. Reid, P.H.S. Torr, and A.W. Fitzgibbon, "Fields of Experts for Image-Based Rendering," Proc. British Machine Vision Conf., 2006.
[38] S. Roth and M.J. Black, "On the Spatial Statistics of Optical Flow," Int'l J. Computer Vision, vol. 74, pp. 33-50, 2007.
[39] D.Q. Sun, S. Roth, J.P. Lewis, and M.J. Black, "Learning Optical Flow," Proc. European Conf. Computer Vision, 2008.
[40] A. Levin, D. Lischinski, and Y. Weiss, "Colorization Using Optimization," Proc. ACM Siggraph, 2004.
[41] L. Breiman, "Random Forests," Machine Learning, vol. 45, pp. 5-32, 2001.
[42] A. Criminisi, J. Shotton, and E. Konukoglu, "Decision Forests: A Unified Framework for Classification, Regression, Density Estimation, Manifold Learning and Semi-Supervised Learning," Foundations and Trends in Computer Graphics and Vision, vol. 7, pp. 81-227, 2012.
[43] D. Martin, C. Fowlkes, and J. Malik, "Learning to Detect Natural Image Boundaries Using Local Brightness, Color, and Texture Cues," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 26, no. 5, pp. 530-549, May 2004.
[44] C. Liu, W.T. Freeman, E.H. Adelson, and Y. Weiss, "Human-Assisted Motion Annotation," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2008.
[45] J. Shotton, A. Fitzgibbon, M. Cook, T. Sharp, M. Finocchio, R. Moore, A. Kipman, and A. Blake, "Real-Time Human Pose Recognition in Parts from Single Depth Images," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[46] O. Mac Aodha, N.D.F. Campbell, A. Nair, and G.J. Brostow, "Patch Based Synthesis for Single Depth Image Super-Resolution," Proc. European Conf. Computer Vision, 2012.
[47] J. Čech, J. Sanchez-Riera, and R. Horaud, "Scene Flow Estimation by Growing Correspondence Seeds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2011.
[48] B. McCane, K. Novins, D. Crannitch, and B. Galvin, "On Benchmarking Optical Flow," Computer Vision and Image Understanding, vol. 84, pp. 126-143, 2001.
[49] B. Kaneva, A. Torralba, and W. Freeman, "Evaluation of Image Features Using a Photorealistic Virtual World," Proc. IEEE Int'l Conf. Computer Vision, 2011.
[50] S. Meister and D. Kondermann, "Real Versus Realistically Rendered Scenes for Optical Flow Evaluation," Proc. 14th ITG Conf. Electronic Media Technology, 2011.
[51] C. Zach, T. Pock, and H. Bischof, "A Duality Based Approach for Realtime TV-l1 Optical Flow," Proc. DAGM Conf. Pattern Recognition, 2007.
[52] M. Werlberger, W. Trobin, T. Pock, A. Wedel, D. Cremers, and H. Bischof, "Anisotropic Huber-L1 Optical Flow," Proc. British Machine Vision Conf., 2009.
[53] T. Brox and J. Malik, "Large Displacement Optical Flow: Descriptor Matching in Variational Motion Estimation," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 33, no. 3, pp. 500-513, Mar. 2011.
[54] M. Otte and H. Nagel, "Optical Flow Estimation: Advances and Comparisons," Proc. European Conf. Computer Vision, 1994.
[55] X. Hu and P. Mordohai, "A Quantitative Evaluation of Confidence Measures for Stereo Vision," IEEE Trans. Pattern Analysis and Machine Analysis, vol. 34, no. 11, pp. 2121-2133, Nov. 2012.
7 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool