This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Analyzing and Capturing Articulated Hand Motion in Image Sequences
December 2005 (vol. 27 no. 12)
pp. 1910-1922
Ying Wu, IEEE
John Lin, IEEE
Capturing the human hand motion from video involves the estimation of the rigid global hand pose as well as the nonrigid finger articulation. The complexity induced by the high degrees of freedom of the articulated hand challenges many visual tracking techniques. For example, the particle filtering technique is plagued by the demanding requirement of a huge number of particles and the phenomenon of particle degeneracy. This paper presents a novel approach to tracking the articulated hand in video by learning and integrating natural hand motion priors. To cope with the finger articulation, this paper proposes a powerful sequential Monte Carlo tracking algorithm based on importance sampling techniques, where the importance function is based on an initial manifold model of the articulation configuration space learned from motion-captured data. In addition, this paper presents a divide-and-conquer strategy that decouples the hand poses and finger articulations and integrates them in an iterative framework to reduce the complexity of the problem. Our experiments show that this approach is effective and efficient for tracking the articulated hand. This approach can be extended to track other articulated targets.

[1] V. Athitsos and S. Sclaroff, “Estimating 3D Hand Pose from a Cluttered Image,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 432-439, June 2003.
[2] A. Blake and M. Isard, Active Contours. London: Springer-Verlag, 1998.
[3] M. Brand, “Shadow Puppetry,” Proc. IEEE Int'l Conf. Computer Vision, vol. II, pp. 1237-1244, 1999.
[4] C. Bregler and S. Omohundro, “Nonlinear Image Interpolation Using Manifold Learning,” Advances in Neural Information Processing Systems 7, G. Tesauro, D. Touretzky, and T. Leen, eds., Cambridge, Mass.: MIT Press, 1995.
[5] T.-J. Cham and J. Rehg, “A Multiple Hypothesis Approach to Figure Tracking,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 239-244, 1999.
[6] E. Chao, K. An, W. Cooney, and R. Linscheid, Biomechanics of the Hand: A Basic Research Study. Mayo Foundation, Minn.: World Scientific, 1989.
[7] J. Deutscher, A. Blake, and I. Reid, “Articulated Body Motion Capture by Annealed Particle Filtering,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 126-133, 2000.
[8] Sequential Monte Carlo Methods in Practice, A. Doucet, N.D. Freitas, and N. Gordon, eds., New York: Springer-Verlag, 2001.
[9] T. Heap and D. Hogg, “Towards 3D Hand Tracking Using a Deformable Model,” Proc. IEEE Int'l Conf. Automatic Face and Gesture Recognition, pp. 140-145, 1996.
[10] T. Heap and D. Hogg, “Wormholes in Shape Space: Tracking through Discontinuous Changes in Shape,” Proc. IEEE Int'l Conf. Computer Vision, pp. 344-349, Jan. 1998.
[11] N. Howe, M. Leventon, and W. Freeman, “Bayesian Reconstruction of 3D Human Motion from Single-Camera Vision,” Proc. Neural Information Processing Systems, 2000.
[12] M. Isard and A. Blake, “Contour Tracking by Stochastic Propagation of Conditional Density,” Proc. European Conf. Computer Vision, pp. 343-356, 1996.
[13] M. Isard and A. Blake, “ICondensation: Unifying Low-Level and High-Level Tracking in a Stochastic Framework,” Proc. European Conf. Computer Vision, vol. 1, pp. 767-781, June 1998.
[14] S. Ju, M. Black, and Y. Yacoob, “Cardboard People: A Parametrized Model of Articulated Motion,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 38-44, Oct. 1996.
[15] J.J. Kuch and T.S. Huang, “Vision-Based Hand Modeling and Tracking for Virtual Teleconferencing and Telecollaboration,” Proc. IEEE Int'l Conf. Computer Vision, pp. 666-671, June 1995.
[16] J. Lee and T. Kunii, “Model-Based Analysis of Hand Posture,” IEEE Computer Graphics and Applications, vol. 15, pp. 77-86, Sept. 1995.
[17] J. Lin, “Visual Hand Tracking and Gesture Analysis,” PhD thesis, Dept. of Electrical and Computer Eng., Univ. of Illinois at Urbana-Champaign, Urbana, 2004.
[18] J. Lin, Y. Wu, and T.S. Huang, “Capturing Human Hand Motion in Image Sequences,” Proc. IEEE Workshop Motion and Video Computing, pp. 99-104, Dec. 2002.
[19] J. Liu and R. Chen, “Sequential Monte Carlo Methods for Dynamic Systems,” J. Am. Statistical Assoc., vol. 93, pp. 1032-1044, 1998.
[20] J. Liu, R. Chen, and T. Logvinenko, “A Theoretical Framework for Sequential Importance Sampling and Resampling,” Sequential Monte Carlo in Practice, A. Doucet, N. de Freitas, and N. Gordon, eds. New York: Springer-Verlag, 2000.
[21] S. Lu, D. Metaxas, D. Samaras, and J. Oliensis, “Using Multiple Cues for Hand Tracking and Model Refinement,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 443-450, June 2003.
[22] J. MacCormick and M. Isard, “Partitioned Sampling, Articulated Objects, and Interface-Quality Hand Tracking,” Proc. European Conf. Computer Vision, vol. 2, pp. 3-19, 2000.
[23] A. Mulder, “Design of Three-Dimensional Virtual Instruments with Gestural Constraints for Musical Applications,” PhD thesis, Simon Fraser Univ., Canada, 1998.
[24] V. Pavlovic, R. Sharma, and T.S. Huang, “Visual Interpretation of Hand Gestures for Human Computer Interaction: A Review,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 677-695, July 1997.
[25] J. Rehg and T. Kanade, “ Model-Based Tracking of Self-Occluding Articulated Objects,” Proc. IEEE Int'l Conf. Computer Vision, pp. 612-617, 1995.
[26] R. Rosales and S. Sclaroff, “Inferring Body Pose without Tracking Body Parts,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 2, pp. 721-727, 2000.
[27] J. Segen and S. Kumar, “Shadow Gesture: 3D Hand Pose Estimation Using a Single Camera,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 479-485, 1999.
[28] N. Shimada, K. Kimura, Y. Shirai, and Y. Kuno, “Hand Posture Estimation by Combining 2-D Appearance-Based 3-D Model-Based Approaches,” Proc. Int'l Conf. Pattern Recognition, vol. 3, pp. 709-712, 2000.
[29] N. Shimada, Y. Shirai, Y. Kuno, and J. Miura, “Hand Gesture Estimation and Model Refinement Using Monocular Camera-Ambiguity Limitation by Inequality Constraints,” Proc. Third Conf. Face and Gesture Recognition, pp. 268-273, 1998.
[30] B. Stenger, P. Mendonca, and R. Cipolla, “Model Based 3D Tracking of an Articulated Hand,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 310-315, Dec. 2001.
[31] B. Stenger, A. Thayananthan, P. Torr, and R. Cipolla, “Filtering Using a Tree-Based Estimator,” Proc. IEEE Int'l Conf. Computer Vision, vol. II, pp. 1063-1070, Oct. 2003.
[32] E. Sudderth, M. Mandel, W. Freeman, and A. Willsky, “Visual Hand Tracking Using Nonparametric Belief Propagation,” Proc. Workshop Generative Model Based Vision, June 2004.
[33] A. Thayananthan, B. Stenger, P. Torr, and R. Cipolla, “Learning a Kinematic Prior for Tree-Based Filtering,” Proc. British Machine Vision Conf., vol. 2, pp. 589-598, 2003.
[34] C. Tomasi and T. Kanade, “Shape and Motion from Image Streams under Orthography— A Factorized Method,” Int'l J. Computer Vision, vol. 9, pp. 137-154, 1992.
[35] C. Tomasi, S. Petrov, and A. Sastry, “3D Tracking = Classification + Interpolation,” Proc. IEEE Int'l Conf. Computer Vision, vol. 2, pp. 1441-1448, Oct. 2003.
[36] J. Triesch and C. von der Malsburg, “Classification of Hand Postures against Complex Backgrounds Using Elastic Graph Matching,” Image and Vision Computing, vol. 20, pp. 937-943, 2002.
[37] Y. Wu, G. Hua, and T. Yu, “Switching Observation Models for Contour Tracking in Clutter,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. I, pp. 295-302, June 2003.
[38] Y. Wu and T.S. Huang, “Capturing Articulated Human Hand Motion: A Divide-and-Conquer Approach,” Proc. IEEE Int'l Conf. Computer Vision, pp. 606-611, Sept. 1999.
[39] Y. Wu and T.S. Huang, “View-Independent Recognition of Hand Postures,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. II, pp. 88-94, June 2000.
[40] Y. Wu and T.S. Huang, “Hand Modeling, Analysis and Recognition for Vision-Based Human Computer Interaction,” IEEE Signal Processing Magazine, vol. 18, pp. 51-60, May 2001.
[41] Y. Wu, J. Lin, and T.S. Huang, “Capturing Natural Hand Articulation,” Proc. IEEE Int'l Conf. Computer Vision, vol. II, pp. 426-432, July 2001.
[42] Z. Zhang, “Iterative Point Matching for Registration of Free-Form Curves and Surfaces,” Int'l J. Computer Vision, vol. 13, pp. 119-152, 1994.

Index Terms:
Index Terms- Motion, tracking, video analysis, statistical computing, probabilistic algorithms, face and gesture recognition.
Citation:
Ying Wu, John Lin, Thomas S. Huang, "Analyzing and Capturing Articulated Hand Motion in Image Sequences," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 12, pp. 1910-1922, Dec. 2005, doi:10.1109/TPAMI.2005.233
Usage of this product signifies your acceptance of the Terms of Use.