This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Kernel-Based Object Tracking
May 2003 (vol. 25 no. 5)
pp. 564-575

Abstract—A new approach toward target representation and localization, the central component in visual tracking of nonrigid objects, is proposed. The feature histogram-based target representations are regularized by spatial masking with an isotropic kernel. The masking induces spatially-smooth similarity functions suitable for gradient-based optimization, hence, the target localization problem can be formulated using the basin of attraction of the local maxima. We employ a metric derived from the Bhattacharyya coefficient as similarity measure, and use the mean shift procedure to perform the optimization. In the presented tracking examples, the new method successfully coped with camera motion, partial occlusions, clutter, and target scale variations. Integration with motion filters and data association techniques is also discussed. We describe only a few of the potential applications: exploitation of background information, Kalman tracking using motion models, and face tracking.

[1] J.K. Aggarwal and Q. Cai, “Human Motion Analysis: A Review,” Computer Vision and Image Understanding, vol. 73, no. 3, pp. 428-440, 1999.
[2] F. Aherne, N. Thacker, and P. Rockett, “The Bhattacharyya Metric as an Absolute Similarity Measure for Frequency Coded Data,” Kybernetika, vol. 34, no. 4, pp. 363-368, 1998.
[3] S. Arulampalam, S. Maskell, N. Gordon, and T. Clapp, “A Tutorial on Particle Filters for On-Line Non-Linear/Non-Gaussian Bayesian Tracking,” IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 174-189, 2002.
[4] S. Avidan, Support Vector Tracking Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 184-191, 2001.
[5] Y. Bar-Shalom and T. Fortmann, Tracking and Data Association. Academic Press. 1988.
[6] B. Bascle and R. Deriche, “Region Tracking Through Image Sequences,“ Proc. Fifth IEEE Int'l Conf. Computer Vision (ICCV), pp. 302-307, June 1995.
[7] S.T. Birchfield, Elliptical Head Tracking Using Intensity Gradients and Color Histograms Proc. Conf. Computer Vision and Pattern Recognition, pp. 232-237, 1998.
[8] M. Black and D. Fleet, “Probabilistic Detection and Tracking of Motion Boundaries,” Int'l J. Computer Vision, vol. 38, no. 3, pp. 231-245, 2000.
[9] Y. Boykov and D. Huttenlocher, “Adaptive Bayesian Recognition in Tracking Rigid Objects,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 697-704, 2000.
[10] G.R. Bradski, “Computer Vision Face Tracking as a Component of a Perceptual User Interface,” Proc. IEEE Workshop Applications of Computer Vision, pp. 214-219, Oct. 1998.
[11] A.D. Bue, D. Comaniciu, V. Ramesh, and C. Regazzoni, “Smart Cameras with Real-Time Video Object Generation,” Proc. IEEE Int'l Conf. Image Processing, vol. III, pp. 429-432, 2002.
[12] G. Caenen, V. Ferrari, A. Zalesny, L. VanGool, “Analyzing the Layout of Composite Textures,” Proc. Texture 2002 Workshop, pp. 15-19, 2002.
[13] T.J. Cham and J. Rehg, “A Multiple Hypothesis Approach to Figure Tracking,” Proc. Conf. Computer Vision and Pattern Recognition, vol. II, pp. 239–245, June 1999.
[14] H.T. Chen and T.L. Liu, Trust-Region Methods for Real-Time Tracking Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 717-722, 2001.
[15] Y. Chen, Y. Rui, and T. Huang, “JPDAF-Based HMM for Real-Time Contour Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 543-550, 2001.
[16] R.T. Collins, A.J. Fujiyoshi, and T. Kanade, Algorithms for Cooperative Multisensor Surveillance Proc. IEEE, vol. 89, no. 10, pp. 1456-1477, Oct. 2001.
[17] D. Comaniciu and P. Meer, Mean Shift: A Robust Approach towards Feature Space A Analysis IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 5, pp. 603-619, May 2002.
[18] D. Comaniciu, V. Ramesh, and P. Meer, Real-Time Tracking of Non-Rigid Objects Using Mean Shift Proc. Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[19] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley&Sons, 1991.
[20] I.J. Cox and S.L. Hingorani, "An Efficient Implementation of Reid's Multiple Hypothesis Tracking Algorithm and Its Evaluation for the Purpose of Visual Tracking," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 2 , Feb. 1996, pp. 138-150.
[21] D. DeCarlo and D. Metaxas, Optical Flow Constraints on Deformable Models with Applications to Face Tracking Int'l J. Computer Vision, vol. 38, no. 2, pp. 99-127, July 2000.
[22] A. Djouadi, O. Snorrason, and F. Garber, “The Quality of Training-Sample Estimates of the Bhattacharyya Coefficient,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 12, pp. 92-97, 1990.
[23] A. Doucet, S. Godsill, and C. Andrieu, “On Sequential Monte Carlo Sampling Methods for Bayesian Filtering,” Statistics and Computing, vol. 10, no. 3, pp. 197-208, 2000.
[24] A. Elgammal, D. Harwood, and L. Davis, “Non-Parametric Model for Background Subtraction,” Proc. European Conf. Computer Vision, vol. II, pp. 751-767, June 2000.
[25] F. Ennesser and G. Medioni, “Finding Waldo, or Focus of Attention Using Local Color Information,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 8, pp. 805-809, Aug. 1995.
[26] V. Ferrari, T. Tuytelaars, and L.V. Gool, “Real-Time Affine Region Tracking and Coplanar Grouping,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 226-233, 2001.
[27] P. Fieguth and D. Terzopoulos, Color-Based Tracking of Heads and Other Objects at Video Frame Rates Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 1997.
[28] K. Fukunaga, Introduction to Statistical Pattern Recognition, second edition. Academic Press, 1990.
[29] J. Garcia, J. Valdivia, and X. Vidal, “Information Theoretic Measure for Visual Target Distinctness,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 4, pp. 362-383, Apr. 2001.
[30] D.M. Gavrila, “The Visual Analysis of Human Movement: A Survey,” Computer Vision and Image Understanding, vol. 73, no. 1, Jan. 1999.
[31] N. Gordon, D. Salmond, and A. Smith, “Novel Approach to Nonlinear/Non-Gaussian Bayesian State Estimation,” IEE Proc. F, vol. 140, no. 2, pp. 107-113, 1993.
[32] M. Greiffenhagen, D. Comaniciu, H. Niemann, and V. Ramesh, “Design, Analysis and Engineering of Video Monitoring Systems: An Approach and a Case Study,” Proc. IEEE, vol. 89, no. 10, pp. 1498-1517, 2001.
[33] G. Hager and P. Belhumeur, “Real-Time Tracking of Image Regions with Changes in Geometry and Illumination,” Proc. Computer Vision and Pattern Recognition, 1996.
[34] U. Handmann, T. Kalinke, C. Tzomakas, M. Werner, and W. von Seelen, “Computer Vision for Driver Assistance Systems,” Proc. SPIE, vol. 3364, pp. 136-147, 1998.
[35] I. Haritaoglu and M. Flickner, “Detection and Tracking of Shopping Groups in Stores,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 431-438, 2001.
[36] I. Haritaoglu, D. Harwood, and L.S. Davis, “W4 - a Real Time System for Detection and Tracking People and their Parts,” Proc. Third Face and Gesture Recognition Conf., pp. 222-227, 1998.
[37] J. Huang, S.R. Kumar, M. Mitra, W.-J. Zhu, and R. Zabih, “Spatial Color Indexing and Applications,” Int'l J. Computer Vision, vol. 35, no. 3, pp. 245-268, 1999.
[38] C. Hue, J. Cadre, and P. Perez, “Sequential Monte Carlo Filtering for Multiple Target Tracking and Data Fusion,” IEEE Trans. Signal Processing, vol. 50, no. 2, pp. 309-325, 2002.
[39] S.S. Intille, J.W. Davis, and A.F. Bobick, “Real Time Closed World Tracking,” IEEE Proc. Computer Vision and Pattern Recognition, pp. 697-703, 1997.
[40] M. Isard and A. Blake, “Condensation-Conditional Density Propagation for Visual Tracking,” Int'l J. Computer Vision, vol. 29, pp. 5-28, 1998.
[41] A.D. Jepson, D.J. Fleet, and T. El-Maraghi, Robust, On-Line Appearance Models for Vision Tracking Proc. IEEE Conf. Computer Vision and Pattern Recognition, Dec. 2001.
[42] S. Julier and J. Uhlmann, “A New Extension of the Kalman Filter to Nonlinear Systems,” Proc. SPIE, vol. 3068, pp. 182-193, 1997.
[43] T. Kailath, The Divergence and Bhattacharyya Distance Measures in Signal Selection IEEE Trans. Comm. Technology, vol. 15, no. 1, pp. 52-60, Feb. 1967
[44] V. Kettnaker and R. Zabih, Bayesian Multi-Camera Sureveillance Proc. Computer Vision and Pattern Recognition, pp. 253-259, June 1999.
[45] G. Kitagawa, “Non-Gaussian State-Space Modeling of Nonstationary Time Series,” J. Am. Statistical Assoc., vol. 82, pp. 1032-1063, 1987.
[46] S. Konishi, A.L. Yuille, J. Coughlan, and S.C. Zhu, Fundamental Bounds on Edge Detection: An Information Theoretic Evaluation of Different Edge Cues Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 573-579, 1999.
[47] J. Krumm, S. Harris, B. Meyers, B. Brumitt, M. Hale, and S. Shafer, “Multi-Camera Multi-Person Tracking for Easy Living,” Proc. IEEE Int'l Workshop Visual Surveillance, pp. 3-10, 2000.
[48] B. Li and R. Chellappa, “Simultaneous Tracking and Verification via Sequential Posterior Estimation,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 110-117, 2000.
[49] J. Lin, “Divergence Measures Based on the Shannon Entropy,” IEEE Trans. Information Theory, vol. 37, pp. 145-151, 1991.
[50] A. Lipton, H. Fujiyoshi, and R. Patil, “Moving Target Classification and Tracking from Real-Time Video,” Proc. IEEE Workshop Applications of Computer Vision, pp. 8-14, 1998.
[51] J. MacCormick and A. Blake, “A Probabilistic Exclusion Principle for Tracking Multiple Objects,” Int'l J. Computer Vision, vol. 39, no. 1, pp. 57-71, 2000.
[52] R. Mahler, “Engineering Statistics for Multi-Object Tracking,” Proc. IEEE Workshop Multi-Object Tracking, pp. 53-60, 2001.
[53] S. McKenna, Y. Raja, and S. Gong, “Tracking Colour Objects Using Adaptive Mixture Models,” Image and Vision Computing J., vol. 17, pp. 223-229, 1999.
[54] R. Merwe, A. Doucet, N. Freitas, and E. Wan, “The Unscented Particle Filter,” Technical Report CUED/F-INFENG/TR 380, Eng. Dept., Cambridge Univ., 2000.
[55] K. Nickels and S. Hutchinson, “Estimating Uncertainty in SSD-Based Feature Tracking,” Image and Vision Computing, vol. 20, pp. 47-58, 2002.
[56] C. Olson, “Image Registration by Aligning Entropies,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 331-336, 2001.
[57] P. Perez, C. Hue, J. Vermaak, and M. Gangnet, “Color-Based Probabilistic Tracking,” Proc. European Conf. Computer Vision, vol. I, pp. 661-675, 2002.
[58] W.H. Press, S.A. Teukolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C, second ed. Cambridge Univ. Press, 1992.
[59] J. Puzicha, Y. Rubner, C. Tomasi, and J. Buhmann, Empirical Evaluation of Dissimilarity Measures for Color and Texture Proc. Int'l Conf. Computer Vision, 1999.
[60] L.R. Rabiner, “Tutorial on Hidden Markov Model and Selected Applications in Speech Recognition,” Proc. IEEE, vol. 77, no. 2, pp. 257-285, 1989.
[61] C. Rao, A. Yilmaz, and M. Shah, “View-Invariant Representation and Recognition of Actions,” Int'l J. Computer Vision, vol. 50, no. 2, pp. 203-226, Nov. 2002.
[62] C. Rasmussen and G.D. Hager, Probabilistic Data Association Methods for Tracking Complex Visual Objects IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 560-576, June 2001.
[63] D. Reid, "An Algorithm for Tracking Multiple Targets," IEEE Trans. Automatic Control, vol. 24, no. 6, pp. 423-432, Dec. 1979.
[64] A. Roche, G. Malandain, and N. Ayache, “Unifying Maximum Likelihood Approaches in Medical Image Registration,” Technical Report 3741, INRIA, 1999.
[65] R. Rosales and S. Sclaroff, “3D Trajectory Recovery for Tracking Multiple Objects and Trajectory Guided Recognition of Actions,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 117-123, 1999.
[66] Y. Rui and Y. Chen, Better Proposal Distributions: Object Tracking Using Unscented Particle Filter Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 2000.
[67] S. Sclaroff and J. Isidoro, “Active Blobs,” Proc. Sixth Int'l Conf. Computer Vision, pp. 1146-1153, 1998.
[68] D.W. Scott, Multivariate Density Estimation. Wiley, 1992.
[69] C. Sminchisescu and B. Triggs, Covariance Scaled Sampling for Monocular 3D Body Tracking Proc. Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 447-454, 2001.
[70] J. Sullivan and J. Rittscher, “Guiding Random Particles by Deterministic Search,” Proc. Eighth Int'l Conf. Computer Vision, vol. I, pp. 323-330, 2001.
[71] M.J. Swain and B.H. Ballard, “Color Indexing,” Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[72] P. Viola and W.M. WellsIII, “Alignment by Maximization of Mutual Information,” Int'l J. Computer Vision, vol. 24, no. 2, pp. 137-154, 1997.
[73] S. Wachter and H. Nagel, “Tracking Persons in Monocular Image Sequences,” Computer Vision and Image Understanding, vol. 74, no. 3, pp. 174-192, 1999.
[74] R. Wildes, R. Kumar, H. Sawhney, S. Samasekera, S. Hsu, H. Tao, Y. Guo, K. Hanna, A. Pope, D. Hirvonen, M. Hansen, and P. Burt, “Aerial Video Surveillance and Exploitation,” Proc. IEEE, vol. 89, no. 10, pp. 1518-1539, 2001.
[75] C. Wren, A. Azarbayejani, T. Darrell, and A.P. Pentland, Pfinder: Real-Time Tracking of the Human Body IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 780-785, July 1997.
[76] Y. Wu and T.S. Huang, A Co-Inference Approach to Robust Visual Tracking Proc. Eighth IEEE Int'l Conf. Computer Vision, vol. 2, pp. 26-33, 2001.
[77] F. Xu and K. Fujimura, “Pedestrian Detection and Tracking with Night Vision,” Proc. IEEE Intelligent Vehicle Symp., 2002.
[78] A. Yilmaz, K. Shafique, N. Lobo, X. Li, T. Olson, and M. Shah, “Target Tracking in FLIR Imagery Using Mean Shift and Global Motion Compensation,” IEEE Workshop Computer Vision Beyond Visible Spectrum, 2001.
[79] “Real-Time Tracking of Non-Rigid Objects Using Mean Shift,” US patent pending, 2000.

Index Terms:
Nonrigid object tracking, target localization and representation, spatially-smooth similarity function, Bhattacharyya coefficient, face tracking.
Citation:
Dorin Comaniciu, Visvanathan Ramesh, Peter Meer, "Kernel-Based Object Tracking," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 25, no. 5, pp. 564-575, May 2003, doi:10.1109/TPAMI.2003.1195991
Usage of this product signifies your acceptance of the Terms of Use.