The Community for Technology Leaders
RSS Icon
Issue No.03 - March (2008 vol.30)
pp: 463-476
Behavior analysis of social insects has garnered impetus in recent years and has led to some advances in fields like control systems, flight navigation etc. Manual labeling of insect motions required for analyzing the behaviors of insects requires significant investment of time and effort. In this paper, we propose certain general principles that help in simultaneous automatic tracking and behavior analysis with applications in tracking bees and recognizing specific behaviors exhibited by them. The state space for tracking is defined using position, orientation and the current behavior of the insect being tracked. The position and orientation are parametrized using a shape model while the behavior is explicitly modeled using a three-tier hierarchical motion model. The first tier (dynamics) models the local motions exhibited and the models built in this tier act as a vocabulary for behavior modeling. The second tier is a Markov motion model built on top of the local motion vocabulary which serves as the behavior model. The third tier of the hierarchy models the switching between behaviors and this is also modeled as a Markov model. We address issues in learning the three-tier behavioral model, in discriminating between models, detecting and in modeling abnormal behaviors. Another important aspect of this work is that it leads to joint tracking and behavior analysis instead of the traditional track and then recognize approach. We apply these principles for tracking bees in a hive while they are executing the waggle dance and the round dance.
Tracking, Behavior Analysis, Activity Analysis, Waggle Dance, Bee Dance
Ashok Veeraraghavan, Rama Chellappa, Mandyam Srinivasan, "Shape-and-Behavior Encoded Tracking of Bee Dances", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 3, pp. 463-476, March 2008, doi:10.1109/TPAMI.2007.70707
[1] V. Frisch, The Dance Language and Orientation of Bees. Harvard Univ. Press, 1993.
[2] M. Srinivasan, S. Zhang, M. Lehrer, and T. Collett, “Honeybee Navigation en Route to the Goal: Visual Flight Control and Odometry,” J. Experimental Biology, vol. 199, pp. 237-244, 1996.
[3] T. Neumann and H. Bulthoff, “Insect-Inspired Visual Control of Translatory Flight,” Proc. Sixth European Conf. Artificial Life, pp.627-636, 2001.
[4] F. Mura and N. Franceschini, “Visual Control of Altitude and Speed in a Flight Agent,” Proc. Third Int'l Conf. Simulation of Adaptive Behavior: From Animal to Animats, pp. 91-99, 1994.
[5] G. Hager and P. Belhumeur, “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, pp. 1025-1039, 1998.
[6] D. Comaniciu, V. Ramesh, and P. Meer, “Real-Time Tracking of Non-Rigid Objects Using Mean-Shift,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 142-149, 2000.
[7] T. Broida, S. Chandra, and R. Chellappa, “Recursive Techniques for the Estimation of 3D Translation and Rotation Parameters from Noisy Image Sequences,” IEEE Trans. Aerospace and Electronic Systems, vol. 26, pp. 639-656, 1990.
[8] A. Doucet, N. Freitas, and N. Gordon, Sequential Monte Carlo Methods in Practice. Springer-Verlag, 2001.
[9] M. Isard and A. Blake, “Contour Tracking by Stochastic Propagation of Conditional Density,” Proc. Fourth European Conf. Computer Vision, pp. 343-356, 1996.
[10] J. Liu and R. Chen, “Sequential Monte Carlo for Dynamical Systems,” J. Am. Statistical Assoc., vol. 93, pp. 1031-1041, 1998.
[11] S. Zhou, R. Chellappa, and B. Moghaddam, “Visual Tracking and Recognition Using Appearance-Adaptive Models in Particle Filters,” IEEE Trans. Image Processing, vol. 11, pp. 1434-1456, 2004.
[12] Z. Khan, T. Balch, and F. Dellaert, “A Rao-Blackwellized Particle Filter for Eigen Tracking,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2004.
[13] H. Lee and Z. Chen, “Determination of 3D Human Body Posture from a Single View,” Computer Vision, Graphics, Image Processing, vol. 30, pp. 148-168, 1985.
[14] C. Sminchisescu and B. Triggs, “Covariance Scaled Tracking for Monocular 3D Body Tracking,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2001.
[15] M. Black and A. Jepson, “A Probabilistic Framework for Matching Temporal Trajectories,” Proc. Seventh IEEE Int'l Conf. Computer Vision, vol. 22, pp. 176-181, 1999.
[16] T. Zhao, T. Wang, and H. Shum, “Learning a Highly Structured Motion Model for 3D Human Tracking,” Proc. Fifth Asian Conf. Computer Vision, 2002.
[17] J. Cheng and J. Moura, “Capture and Representation of Human Walking in Live Video Sequence,” IEEE Trans. Multimedia, vol. 1, no. 2, pp. 144-156, 1999.
[18] C. Bregler, “Learning and Recognizing Human Dynamics in Video Sequences,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 1997.
[19] T. Zhao and R. Nevatia, “3D Tracking of Human Locomotion: A Tracking as Recognition Approach,” Proc. 16th Int'l Conf. Pattern Recognition, 2002.
[20] V. Pavlovic, J. Rehg, T. Cham, and K. Murphy, “A Dynamic Bayesian Network Approach to Figure Tracking Using Learned Dynamic Models,” Proc. Seventh IEEE Int'l Conf. Computer Vision, 1999.
[21] S.M. Oh, J.M. Rehg, T. Balch, and F. Dellaert, “Learning and Inference in Parametric Switching Linear Dynamic Systems,” Proc. 10th IEEE Int'l Conf. Computer Vision, 2005.
[22] T.D. Seeley, “The Tremble Dance of the Honeybee: Message and Meanings,” Behavioral Ecology and Sociobiology, vol. 31, pp. 375-383, 1992.
[23] A. Feldman and T. Balch, “Automatic Identification of Bee Movement Using Human Trainable Models of Behavior,” Math. and Algorithms of Social Insects, Dec. 2003.
[24] S.M. Oh, J.M. Rehg, T. Balch, and F. Dellaert, “Parameterized Duration Modeling for Switching Linear Dynamic Systems,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2006.
[25] M. Isard and A. Blake, “A Mixed-State Condensation Tracker with Automatic Model-Switching,” Proc. Sixth IEEE Int'l Conf. Computer Vision, 1998.
[26] S. Blackman and R. Popoli, Design and Analysis of Modern Tracking Systems. Artech House, 1999.
[27] S. Fine, Y. Singer, and N. Tishby, “The Hierarchical Hidden Markov Model: Analysis and Applications,” Machine Learning, vol. 32, no. 1, pp. 41-62, 1998.
[28] X. Koutsoukos and P. Antsaklis, “Hierarchical Control of Piecewise Linear Hybrid Dynamical Systems Based on Discrete Abstractions,” ISIS technical report, Feb. 2001.
[29] A. Blake, B. North, and M. Isard, “Learning Multi-Class Dynamics,” Advances in Neural Information Processing Systems, pp.389-395, 1999.
[30] B. Juang and L. Rabiner, “A Probabilistic Distance Measure for Hidden Markov Models,” AT&T Technical J., vol. 64, pp. 391-408, 1985.
[31] N. Vaswani, “Additive Change Detection in Nonlinear Systems with Unknown Change Parameters,” IEEE Trans. Signal Processing, 2006.
[32] N. Vaswani, “Change Detection in Partially Observed Nonlinear Dynamic Systems with Unknown Change Parameters,” Am. Control Conf., 2004.
[33] N. Gordon, D. Salmond, and A. Smith, “Novel Approach to Non-Linear/Non-Gaussian Bayesian State Estimation,” Proc. IEE Radar and Signal Processing, vol. 140, pp. 107-113, 1993.
[34] D. Freedman and M. Turek, “Illumination-Invariant Tracking via Graph Cuts,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, 2005.
[35] Y. Xu and A. Roy-Chowdhury, “Integrating Motion, Illumination and Structure in Video Sequences, with Applications in Illumination-Invariant Tracking,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 5, May 2006.
6 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool