The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2012 vol.34)
pp: 2420-2440
Weiming Hu , Nat. Lab. of Pattern Recognition, Inst. of Autom., Beijing, China
Xi Li , Nat. Lab. of Pattern Recognition, Inst. of Autom., Beijing, China
Wenhan Luo , Nat. Lab. of Pattern Recognition, Inst. of Autom., Beijing, China
Xiaoqin Zhang , Nat. Lab. of Pattern Recognition, Inst. of Autom., Beijing, China
S. Maybank , Dept. of Comput. Sci. & Inf. Syst., Birkbeck Coll., London, UK
Zhongfei Zhang , Dept. of Comput. Sci., Binghamton Univ., Binghamton, NY, USA
ABSTRACT
Object appearance modeling is crucial for tracking objects, especially in videos captured by nonstationary cameras and for reasoning about occlusions between multiple moving objects. Based on the log-euclidean Riemannian metric on symmetric positive definite matrices, we propose an incremental log-euclidean Riemannian subspace learning algorithm in which covariance matrices of image features are mapped into a vector space with the log-euclidean Riemannian metric. Based on the subspace learning algorithm, we develop a log-euclidean block-division appearance model which captures both the global and local spatial layout information about object appearances. Single object tracking and multi-object tracking with occlusion reasoning are then achieved by particle filtering-based Bayesian state inference. During tracking, incremental updating of the log-euclidean block-division appearance model captures changes in object appearance. For multi-object tracking, the appearance models of the objects can be updated even in the presence of occlusions. Experimental results demonstrate that the proposed tracking algorithm obtains more accurate results than six state-of-the-art tracking algorithms.
INDEX TERMS
video signal processing, Bayes methods, cameras, covariance matrices, feature extraction, inference mechanisms, learning (artificial intelligence), object tracking, particle filtering (numerical methods), particle filtering-based Bayesian state inference, single object tracking, multiple object tracking, incremental log-Euclidean Riemannian subspace learning algorithm, block-division appearance model, object appearance modeling, videos, nonstationary cameras, occlusion reasoning, symmetric positive definite matrices, covariance matrices, image features, vector space, log-Euclidean Riemannian metric, log-Euclidean block-division appearance model, global spatial layout information, local spatial layout information, Tracking, Cameras, Solid modeling, Covariance matrix, Algorithm design and analysis, Inference algorithms, Visual analytics, block-division appearance model, Visual object tracking, occlusion reasoning, log-euclidean Riemannian subspace, incremental learning
CITATION
Weiming Hu, Xi Li, Wenhan Luo, Xiaoqin Zhang, S. Maybank, Zhongfei Zhang, "Single and Multiple Object Tracking Using Log-Euclidean Riemannian Subspace and Block-Division Appearance Model", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.34, no. 12, pp. 2420-2440, Dec. 2012, doi:10.1109/TPAMI.2012.42
REFERENCES
[1] X. Li, W.M. Hu, Z.F. Zhang, X.Q. Zhang, M.L. Zhu, and J. Cheng, "Visual Tracking via Incremental Log-Euclidean Riemannian Subspace Learning," Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[2] M.J. Black and A.D. Jepson, "EigenTracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation," Int'l J. Computer Vision, vol. 26, no. 1, pp. 63-84, Jan. 1998.
[3] M. Isard and A. Blake, "Contour Tracking by Stochastic Propagation of Conditional Density," Proc. European Conf. Computer Vision, vol. 2, pp. 343-356, 1996.
[4] M.J. Black, D.J. Fleet, and Y. Yacoob, "A Framework for Modeling Appearance Change in Image Sequence," Proc. IEEE Int'l Conf. Computer Vision, pp. 660-667, Jan. 1998.
[5] A.D. Jepson, D.J. Fleet, and T.F. El-Maraghi, "Robust Online Appearance Models for Visual Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 415-422, 2001.
[6] S.K. Zhou, R. Chellappa, and B. Moghaddam, "Visual Tracking and Recognition Using Appearance-Adaptive Models in Particle Filters," IEEE Trans. Image Processing, vol. 13, no. 11, pp. 1491-1506, Nov. 2004.
[7] T. Yu and Y. Wu, "Differential Tracking Based on Spatial-Appearance Model (SAM)," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 720-727, June 2006.
[8] J. Li, S.K. Zhou, and R. Chellappa, "Appearance Modeling under Geometric Context," Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1252-1259, 2005.
[9] K. Lee and D. Kriegman, "Online Learning of Probabilistic Appearance Manifolds for Video-Based Recognition and Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 852-859, 2005.
[10] H. Lim, V. Morariu, O.I. Camps, and M. Sznaier, "Dynamic Appearance Modeling for Human Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 751-757, 2006.
[11] J. Ho, K. Lee, M. Yang, and D. Kriegman, "Visual Tracking Using Learned Linear Subspaces," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 782-789, 2004.
[12] Y. Li, "On Incremental and Robust Subspace Learning," Pattern Recognition, vol. 37, no. 7, pp. 1509-1518, 2004.
[13] D. Skocaj and A. Leonardis, "Weighted and Robust Incremental Method for Subspace Learning," Proc. Ninth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1494-1501, Oct. 2003.
[14] D.A. Ross, J. Lim, R.-S. Lin, and M.-H. Yang, "Incremental Learning for Robust Visual Tracking," Int'l J. Computer Vision, vol. 77, no. 2, pp. 125-141, May 2008.
[15] Y. Wu, T. Yu, and G. Hua, "Tracking Appearances with Occlusions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 789-795, June 2003.
[16] A. Yilmaz, "Object Tracking by Asymmetric Kernel Mean Shift with Automatic Scale and Orientation Selection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-6, June 2007.
[17] G. Silveira and E. Malis, "Real-Time Visual Tracking under Arbitrary Illumination Changes," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-6, June 2007.
[18] M. Grabner, H. Grabner, and H. Bischof, "Learning Features for Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2007.
[19] X. Zhou, W.M. Hu, Y. Chen, and W. Hu, "Markov Random Field Modeled Level Sets Method for Object Tracking with Moving Cameras," Proc. Asian Conf. Computer Vision, pp. 832-842, 2007.
[20] S. Ilic and P. Fua, "Non-Linear Beam Model for Tracking Large Deformations," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, June 2007.
[21] S. Tran and L. Davis, "Robust Object Tracking with Regional Affine Invariant Features," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.
[22] Q. Zhao, S. Brennan, and H. Tao, "Differential EMD Tracking," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, Oct. 2007.
[23] H. Wang, D. Suter, K. Schindler, and C. Shen, "Adaptive Object Tracking Based on an Effective Appearance Filter," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 9, pp. 1661-1667, Sept. 2007.
[24] F. Porikli, O. Tuzel, and P. Meer, "Covariance Tracking Using Model Update Based on Lie Algebra," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 728-735, 2006.
[25] O. Tuzel, F. Porikli, and P. Meer, "Human Detection via Classification on Riemannian Manifolds," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2007.
[26] P.T. Fletcher and S. Joshi, "Principal Geodesic Analysis on Symmetric Spaces: Statistics of Diffusion Tensors," Proc. Computer Vision and Math. Methods in Medical and Biomedical Image Analysis, pp. 87-98, 2004.
[27] T. Lin and H. Zha, "Riemannian Manifold Learning," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 5, pp. 796-809, May 2008.
[28] V. Arsigny, P. Fillard, X. Pennec, and N. Ayache, "Geometric Means in a Novel Vector Space Structure on Symmetric Positive-Definite Matrices," SIAM J. Matrix Analysis and Applications, vol. 29, no. 1, pp. 328-347, Feb. 2007.
[29] O. Tuzel, F. Porikli, and P. Meer, "Region Covariance: A Fast Descriptor for Detection and Classification," Proc. European Conf. Computer Vision, vol. 2, pp. 589-600, 2006.
[30] X. Pennec, P. Fillard, and N. Ayache, "A Riemannian Framework for Tensor Computing," Int'l J. Computer Vision, vol. 66, no. 1, pp. 41-66, Jan. 2006.
[31] W. Rossmann, Lie Groups: An Introduction through Linear Group. Oxford Univ. Press, 2002.
[32] A. Levy and M. Lindenbaum, "Sequential Karhunen-Loeve Basis Extraction and Its Application to Images," IEEE Trans. Image Processing, vol. 9, no. 8, pp. 1371-1374, Aug. 2000.
[33] A. Mittal and L.S. Davis, "M2Tracker: A Multi-View Approach to Segmenting and Tracking People in a Cluttered Scene," Int'l J. Computer Vision, vol. 51, no. 3, pp. 189-203, Feb./Mar. 2003.
[34] C. Stauffer and W.E.L. Grimson, "Adaptive Background Mixture Models for Real-Time Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 246-252, 1999.
[35] S. Yan, S. Shan, X. Chen, W. Gao, and J. Chen, "Matrix-Structural Learning (MSL) of Cascaded Classifier from Enormous Training Set," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-7, June 2007.
[36] B. Babenko, M.-H. Yang, and S. Belongie, "Visual Tracking with Online Multiple Instance Learning," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 983-990, June 2009.
[37] G. Hager and P. Belhumeur, "Real-Time Tracking of Image Regions with Changes in Geometry and Illumination," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 403-410, June 1996.
[38] B. Wu and R. Nevatia, "Detection and Tracking of Multiple, Partially Occluded Humans by Bayesian Combination of Edgelet Based Part Detectors," Int'l J. Computer Vision, vol. 75, no. 2, pp. 247-266, Nov. 2007.
[39] B. Wu and R. Nevatia, "Tracking of Multiple, Partially Occluded Humans Based on Static Body Part Detection," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 951-958, June 2006.
[40] H. Wang and D. Suter, "Tracking and Segmenting People with Occlusions by a Sample Consensus-Based Method," Proc. IEEE Int'l Conf. Image Processing, vol. 2, pp. 410-413, Sept. 2005.
[41] S. Khan and M. Shah, "Tracking People in Presence of Occlusion," Proc. Asian Conf. Computer Vision, pp. 1132-1137, Jan. 2000.
[42] T. Zhao and R. Nevatia, "Tracking Multiple Humans in Crowded Environment," Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 406-413, June-July 2004.
[43] N. Joshi, S. Avidan, W. Matusik, and D. Kriegman, "Synthetic Aperture Tracking: Tracking through Occlusions," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, Oct. 2007.
[44] M. Yang, Z. Fan, J. Fan, and Y. Wu, "Tracking Nonstationary Visual Appearances by Data-Driven Adaptation," IEEE Trans. Image Processing, vol. 18, no. 7, pp. 1633-1644, July 2009.
[45] J. Kwon and K.M. Lee, "Tracking of a Non-Rigid Object via Patch-Based Dynamic Appearance Modeling and Adaptive Basin Hopping Monte Carlo Sampling," Proc. IEEE Conf. Computer Vision and Pattern Recognition Workshops, pp. 1208-1215, June 2009.
[46] K. Ishiguro, T. Yamada, and N. Ueda, "Simultaneous Clustering and Tracking Unknown Number of Objects," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[47] X. Song, J. Cui, H. Zha, and H. Zhao, "Vision-Based Multiple Interacting Targets Tracking via On-Line Supervised Learning," Proc. 10th European Conf. Computer Vision, vol. 3, pp. 642-655, 2008.
[48] F. Fleuret, J. Berclaz, R. Lengagne, and P. Fua, "Multicamera People Tracking with a Probabilistic Occupancy Map," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 30, no. 2, pp. 267-282, Feb. 2008.
[49] S. Khan and M. Shah, "Tracking Multiple Occluding People by Localizing on Multiple Scene Planes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 3, pp. 505-519, Mar. 2009.
[50] X. Mei and H.B. Ling, "Robust Visual Tracking Using L1 Minimization," Proc. IEEE Int'l Conf. Computer Vision, pp. 1436-1443, 2009.
[51] D. Liang, Q. Huang, H. Yao, S. Jiang, R. Ji, and W. Gao, "Novel Observation Model for Probabilistic Object Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1387-1394, June 2010.
[52] W. He, T. Yamashita, H. Lu, and S. Lao, "Surf Tracking," Proc. IEEE Int'l Conf. Computer Vision, pp. 1586-1592, 2009.
[53] D.-N. Ta, W.-C. Chen, N. Gelfand, and K. Pulli, "SURFTrac: Efficient Tracking and Continuous Object Recognition Using Local Feature Descriptors," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 2937-2944, June 2009.
[54] W. Qu, D. Schonfeld, and M. Mohamed, "Real-Time Distributed Multi-Object Tracking Using Multiple Interactive Trackers and a Magnetic-Inertia Potential Model," IEEE Trans. Multimedia, vol. 9, no. 3, pp. 511-519, Apr. 2007.
[55] M. Yang, T. Yu, and Y. Wu, "Game-Theoretic Multiple Target Tracking," Proc. IEEE Int'l Conf. Computer Vision, pp. 1-8, 2007.
[56] Y. Jin and F. Mokhtarian, "Variational Particle Filter for Multi-Object Tracking," Proc. Int'l Conf. Computer Vision, pp. 1-8, 2007.
[57] L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[58] A. Ess, B. Leibe, K. Schindler, and L.V. Gool, "A Mobile Vision System for Robust Multi-Person Tracking," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, June 2008.
[59] C. Huang, B. Wu, and R. Nevatia, "Robust Object Tracking by Hierarchical Association of Detection Responses," Proc. 10th European Conf. Computer Vision, vol. 2, pp. 788-801, 2008.
[60] D. Mitzel, E. Horbert, A. Ess, and B. Leibe, "Multi-Person Tracking with Sparse Detection and Continuous Segmentation," Proc. European Conf. Computer Vision, pp. 397-410, Sept. 2010.
[61] J. Kwon, K.M. Lee, and F.C. Park, "Visual Tracking via Geometric Particle Filtering on the Affine Group with Optimal Importance Functions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 991-998, June 2009.
[62] F. Porikli and O. Tuzel, "Learning on Lie Groups for Invariant Detection via Tracking," Proc. Int'l Workshop Object Recognition, invited, 2008.
[63] E.B. Sudderth, M.I. Mandel, W.T. Freeman, and A.S. Willsky, "Distributed Occlusion Reasoning for Tracking with Nonparametric Belief Propagation," Proc. Ann. Conf. Neural Information Processing Systems, pp. 1369-1376, 2004.
[64] L. Zhang, Y. Li, and R. Nevatia, "Global Data Association for Multi-Object Tracking Using Network Flows," Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 1-8, 2008.
[65] C. Wang, M.L. Gorce, and N. Paragios, "Segmentation, Ordering, and Multi-Object Tracking Using Graphical Models," Proc. IEEE Int'l Conf. Computer Vision, pp. 747-754, 2009.
[66] E. Herbst, S. Seitz, and S. Baker, "Occlusion Reasoning for Temporal Interpolation Using Optical Flow," technical report, Microsoft Research, Aug. 2009.
[67] V. Gay-Bellile, A. Bartoli, and P. Sayd, "Direct Estimation of Nonrigid Registrations with Image-Based Self-Occlusion Reasoning," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 1, pp. 87-104, Jan. 2010.
20 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool