The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.03 - March (2010 vol.32)
pp: 402-415
Thomas Brox , University of California, Berkeley, Berkeley
Bodo Rosenhahn , Leibniz-Universität Hannover, Hannover
Juergen Gall , Max-Planck-Institut für Informatik, Saarbrüecken
Daniel Cremers , University of Bonn, Bonn
ABSTRACT
In this paper, we propose the combined use of complementary concepts for 3D tracking: region fitting on one side and dense optical flow as well as tracked SIFT features on the other. Both concepts are chosen such that they can compensate for the shortcomings of each other. While tracking by the object region can prevent the accumulation of errors, optical flow and SIFT can handle larger transformations. Whereas segmentation works best in case of homogeneous objects, optical flow computation and SIFT tracking rely on sufficiently structured objects. We show that a sensible combination yields a general tracking system that can be applied in a large variety of scenarios without the need to manually adjust weighting parameters.
INDEX TERMS
Tracking, segmentation, motion.
CITATION
Thomas Brox, Bodo Rosenhahn, Juergen Gall, Daniel Cremers, "Combined Region and Motion-Based 3D Tracking of Rigid and Articulated Objects", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.32, no. 3, pp. 402-415, March 2010, doi:10.1109/TPAMI.2009.32
REFERENCES
[1] J.L. Barron, D.J. Fleet, and S.S. Beauchemin, “Performance of Optical Flow Techniques,” Int'l J. Computer Vision, vol. 12, no. 1, pp. 43-77, Feb. 1994.
[2] M.J. Black and P. Anandan, “The Robust Estimation of Multiple Motions: Parametric and Piecewise Smooth Flow Fields,” Computer Vision and Image Understanding, vol. 63, no. 1, pp. 75-104, Jan. 1996.
[3] C. Bregler and J. Malik, “Tracking People with Twists and Exponential Maps,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 8-15, 1998.
[4] C. Bregler, J. Malik, and K. Pullen, “Twist Based Acquisition and Tracking of Animal and Human Kinematics,” Int'l J. Computer Vision, vol. 56, no. 3, pp. 179-194, 2004.
[5] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High Accuracy Optical Flow Estimation Based on a Theory for Warping,” Proc. Eighth European Conf. Computer Vision, T. Pajdla and J. Matas, eds., pp. 25-36, May 2004.
[6] T. Brox, B. Rosenhahn, D. Cremers, and H.-P. Seidel, “High Accuracy Optical Flow Serves 3-D Pose Tracking: Exploiting Contour and Flow Based Constraints,” Proc. European Conf. Computer Vision, A. Leonardis, H. Bischof, and A. Pinz, eds., pp.98-111, 2006.
[7] T. Brox, B. Rosenhahn, D. Cremers, and H.-P. Seidel, “Nonparametric Density Estimation with Adaptive Anisotropic Kernels for Human Motion Tracking,” Proc. Second Int'l Workshop Human Motion, pp. 152-165, 2007.
[8] T. Brox and J. Weickert, “A TV Flow Based Local Scale Estimate and Its Application to Texture Discrimination,” J. Visual Comm. and Image Representation, vol. 17, no. 5, pp. 1053-1073, Oct. 2006.
[9] A. Bruhn and J. Weickert, “Towards Ultimate Motion Estimation: Combining Highest Accuracy with Real-Time Performance,” Proc. 10th IEEE Int'l Conf. Computer Vision, pp. 749-755, Oct. 2005.
[10] A. Bruhn and J. Weickert, “Confidence Measures for Variational Optic Flow Methods,” Geometric Properties from Incomplete Data, R.Klette, R. Kozera, L. Noakes, and J. Weickert, eds., pp. 283-297, Springer, 2006.
[11] T. Chan and L. Vese, “Active Contours without Edges,” IEEE Trans. Image Processing, vol. 10, no. 2, pp. 266-277, Feb. 2001.
[12] D. DeCarlo and D. Metaxas, “Optical Flow Constraints on Deformable Models with Applications to Face Tracking,” Int'l J. Computer Vision, vol. 38, no. 2, pp. 99-127, July 2000.
[13] P.F. Felzenszwalb and D.P. Huttenlocher, “Distance Transforms of Sampled Functions,” Technical Report TR2004-1963, Computer Science Dept., Cornell Univ., Sept. 2004.
[14] D. Gabor, “Theory of Communication,” J. Institution of Electrical Engineers, vol. 93, pp. 429-457, 1946.
[15] J. Gall, B. Rosenhahn, and H.-P. Seidel, “Robust Pose Estimation with 3D Textured Models,” Proc. IEEE Pacific-Rim Symp. Image and Video Technology, pp. 84-95, 2006.
[16] D. Gavrila and L. Davis, “3D Model Based Tracking of Humans in Action: A Multiview Approach,” Proc. Advanced Research Projects Agency Image Understanding Workshop, pp. 73-80, 1996.
[17] W.E.L. Grimson, Object Recognition by Computer. MIT Press, 1990.
[18] D. Lowe, “Solving for the Parameters of Object Models from Image Descriptions,” Proc. Advanced Research Projects Agency Image Understanding Workshop, pp. 121-127, 1980.
[19] D. Lowe, “Three-Dimensional Object Recognition from Single Two-Dimensional Images,” Artificial Intelligence, vol. 31, no. 3, pp.355-395, 1987.
[20] D. Lowe, “Distinctive Image Features from Scale-Invariant Keypoints,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 91-110, 2004.
[21] E. Marchand, P. Bouthemy, and F. Chaumette, “A 2D-3D Model-Based Approach to Real-Time Visual Tracking,” Image and Vision Computing, vol. 19, no. 13, pp. 941-955, Nov. 2001.
[22] I. Matthews and S. Baker, “Active Appearance Models Revisited,” Int'l J. Computer Vision, vol. 60, no. 2, pp. 135-164, 2004.
[23] E. Mémin and P. Pérez, “Robust Discontinuity-Preserving Model for Estimating Optical Flow,” Proc. 13th Int'l Conf. Pattern Recognition, pp. 920-924, Aug. 1996.
[24] K. Mikolajczyk and C. Schmid, “A Performance Evaluation of Local Descriptors,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 10, pp. 1615-1630, Oct. 2005.
[25] T.B. Moeslund, A. Hilton, and V. Krüger, “A Survey of Advances in Vision-Based Human Motion Capture and Analysis,” Computer Vision and Image Understanding, vol. 104, no. 2, pp. 90-126, 2006.
[26] G. Mori and J. Malik, “Recovering 3D Human Body Configurations Using Shape Contexts,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 28, no. 7, pp. 1052-1062, July 2006.
[27] D. Mumford and J. Shah, “Optimal Approximations by Piecewise Smooth Functions and Associated Variational Problems,” Comm. Pure and Applied Math., vol. 42, pp. 577-685, 1989.
[28] R.M. Murray, Z. Li, and S.S. Sastry, Mathematical Introduction to Robotic Manipulation. CRC Press, 1994.
[29] M. Özuysal, V. Lepetit, F. Fleuret, and P. Fua, “Feature Harvesting for Tracking-by-Detection,” Proc. European Conf. Computer Vision, pp. 592-605, 2006.
[30] N. Paragios and R. Deriche, “Geodesic Active Regions: A New Paradigm to Deal with Frame Partition Problems in Computer Vision,” J. Visual Comm. and Image Representation, vol. 13, nos. 1/2, pp. 249-268, 2002.
[31] D. Ramanan, D. Forsyth, and A. Zisserman, “Tracking People by Learning Their Appearance,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 1, pp. 65-81, Jan. 2007.
[32] L.G. Roberts, “Machine Perception of Three-Dimensional Solids,” Optical and Electro-Optical Information Processing, J. Tippet, ed., pp.159-197, MIT Press, 1966.
[33] B. Rosenhahn, T. Brox, D. Cremers, and H.-P. Seidel, “A Comparison of Shape Matching Methods for Contour Based Pose Estimation,” Proc. Int'l Workshop Combinatorial Image Analysis, R.Reulke, U. Eckhardt, B. Flach, U. Knauer, and K. Polthier, eds., pp. 263-276, June 2006.
[34] B. Rosenhahn, T. Brox, and J. Weickert, “Three-Dimensional Shape Knowledge for Joint Image Segmentation and Pose Tracking,” Int'l J. Computer Vision, vol. 73, no. 3, pp. 243-262, July 2007.
[35] F. Shevlin, “Analysis of Orientation Problems Using Plücker Lines,” Proc. Int'l Conf. Pattern Recognition, vol. 1, pp. 685-689, 1998.
[36] J. Shi and C. Tomasi, “Good Features to Track,” Proc. Int'l Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[37] H. Sidenbladh and M.J. Black, “Learning the Statistics of People in Images and Video,” Int'l J. Computer Vision, vol. 54, pp. 183-209, 2003.
[38] H. Sidenbladh, M.J. Black, and D.J. Fleet, “Stochastic Tracking of 3D Human Figures Using 2D Image Motion,” Proc. European Conf. Computer Vision, D. Vernon, ed., pp. 702-718, 2000.
[39] H. Sidenbladh, M.J. Black, and L. Sigal, “Implicit Probabilistic Models of Human Motion for Synthesis and Tracking,” Proc. European Conf. Computer Vision, A. Heyden, G. Sparr, M. Nielsen, and P. Johansen, eds., pp. 784-800, 2002.
[40] L. Sigal, B. Sidharth, S. Roth, M. Black, and M. Isard, “Tracking Loose-Limbed People,” Proc. 2004 IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 421-428, June 2004.
[41] C. Sminchisescu and A. Jepson, “Generative Modeling for Continuous Non-Linearly Embedded Visual Inference,” Proc. Int'l Conf. Machine Learning, 2004.
[42] Geometric Computing with Clifford Algebra: Theoretical Foundations and Applications in Computer Vision and Robotics, G. Sommer, ed. Springer, 2001.
[43] M. Spengler and B. Schiele, “Towards Robust Multi-Cue Integration for Visual Tracking,” Proc. Ninth Int'l Workshop Computer Vision Systems, pp. 93-106, 2001.
[44] R. Urtasun, D.J. Fleet, and P. Fua, “3D People Tracking with Gaussian Process Dynamical Models,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 238-245, 2006.
[45] L. Vacchetti, V. Lepetit, and P. Fua, “Combining Edge and Texture Information for Real-Time Accurate 3D Camera Tracking,” Proc. Third Int'l Symp. Mixed and Augmented Reality, pp. 48-57, 2004.
[46] C. Zach, T. Pock, and H. Bischof, “A Duality Based Approach for Realtime TV-L1 Optical Flow,” Proc. DAGM Symp. Pattern Recognition, pp. 214-223, 2007.
[47] X.S. Zhou, D. Comaniciu, and A. Gupta, “An Information Fusion Framework for Robust Shape Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 1, pp. 115-129, Jan. 2005.
490 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool