This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models
April 2000 (vol. 22 no. 4)
pp. 322-336

Abstract—An improved technique for 3D head tracking under varying illumination conditions is proposed. The head is modeled as a texture mapped cylinder. Tracking is formulated as an image registration problem in the cylinder's texture map image. The resulting dynamic texture map provides a stabilized view of the face that can be used as input to many existing 2D techniques for face recognition, facial expressions analysis, lip reading, and eye tracking. To solve the registration problem in the presence of lighting variation and head motion, the residual error of registration is modeled as a linear combination of texture warping templates and orthogonal illumination templates. Fast and stable on-line tracking is achieved via regularized, weighted least-squares minimization of the registration error. The regularization term tends to limit potential ambiguities that arise in the warping and illumination templates. It enables stable tracking over extended sequences. Tracking does not require a precise initial fit of the model; the system is initialized automatically using a simple 2D face detector. The only assumption is that the target is facing the camera in the first frame of the sequence. The formulation is tailored to take advantage of texture mapping hardware available in many workstations, PCs, and game consoles. The nonoptimized implementation runs at about 15 frames per second on a SGI O2 graphic workstation. Extensive experiments evaluating the effectiveness of the formulation are reported. The sensitivity of the technique to illumination, regularization parameters, errors in the initial positioning, and internal camera parameters are analyzed. Examples and applications of tracking are reported.

[1] The Flock of Birds. Ascension Technology Corp., P.O. Box 527, Burlington, Vt. 05402.
[2] A. Azarbayejani,T. Starner,B. Horowitz,, and A. Pentland,“Visually controlled graphics,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 602-605, June 1993. (Special Section on 3D Modeling in Image Analysis and Synthesis).
[3] S. Basu, I. Essa, and A. Pentland, “Motion Regularization for Model-Based Head Tracking,” Proc. Int'l Conf. Pattern Recognition, 1996.
[4] S. Birchfield, “An Elliptical Head Tracker,” Proc. 31st Asilomar Conf. Signals, Systems, and Computers, Nov. 1997.
[5] M.J. Black and A.D. Jepson, “Eigentracking: Robust Matching and Tracking of Articulated Objects Using a View-Based Representation,” Int'l J. Computer Vision, vol. 26, no. 1, pp. 63-84, Jan. 1998.
[6] M.J. Black and Y. Yacoob, "Tracking and Recognizing Rigid and Non-Rigid Facial Motions Using Local Parametric Model of Image Motion," Proc. Int'l Conf. Computer Vision, pp. 374-381,Cambridge, Mass., 1995.
[7] M.J. Black and Y. Yacoob, “Recognizing Facial Expressions in Image Sequences Using Local Parameterized Models of Image Motion,” Int'l J. Computer Vision, vol. 25, no. 1, pp. 23-48, Oct. 1997.
[8] T.F. Cootes, G.J. Edwards, and C.J. Taylor, “Active Appearance Models,” Proc. European Conf. Computer Vision, pp. 484-498, June 1998.
[9] J.L. Crowley and F. Beard, “Multimodal Tracking of Faces for Video Communications,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 640-645, 1997.
[10] D. DeCarlo and D. Metaxas, “The Integration of Optical Flow and Deformable Models: Applications to Human Face Shape and Motion Estimation,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '96), pp. 231-238, 1996.
[11] F. Dellaert, C. Thorpe, and S. Thrun, “Super-Resolved Texture Tracking of Planar Surface Patches,” Proc. IEEE/RSJ Int'l Conf. Intelligent Robotic Systems, 1998.
[12] F. Dellaert, S. Thrun, and C. Thorpe, “Jacobian Images of Super-Resolved Texture Maps for Model-Based Motion Estimation and Tracking,” Proc. Fourth Workshop Applications of Computer Vision, pp. 2-7, 1998.
[13] I.A. Essa and A.P. Pentland, “Coding, Analysis, Interpretation, and Recognition of Facial Expressions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 757-763, July 1997.
[14] P. Fieguth and D. Terzopoulos, Color-Based Tracking of Heads and Other Objects at Video Frame Rates Proc. IEEE Conf. Computer Vision and Pattern Recognition, June 1997.
[15] M. Gleicher, “Projective Registration with Difference Decomposition,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1997.
[16] G. Hager and P. Belhumeur, “Efficient Region Tracking with Parametric Models of Geometry and Illumination,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 10, pp. 1025-1039, Oct. 1998.
[17] P. Hallinan, "A Low-Dimensional Representation of Human Faces for Arbitrary Lighting Conditions," Proc. IEEE Conf. Computer Vision and Pattern Recognition, 1994, pp. 995-999.
[18] B.K.P. Horn, “Closed-Form Solution of Absolute Orientation Using Unit Quaternions,” J. Optical Soc. of Am. A, vol. 4, no. 4, Apr. 1987.
[19] T. Horprasert, Y. Yacoob, and L.S. Davis, “Computing 3-D Head Orientation from a Monocular Image,” Proc. Int'l Conf. Automatic Face and Gesture Recognition, pp. 242-247, Oct. 1996.
[20] M. Isard and A. Blake, “A Mixed-State Condensation Tracker with Automatic Model Switching,” Proc. Sixth Int'l Conf. Computer Vision, pp. 107-112, 1998.
[21] T. Jebara and A. Pentland, “Parameterized Structure from Motion for 3D Adaptive Feedback Tracking of Faces,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 144-150, June 1997.
[22] K. Toyama and G. Hager, "Incremental Focus of Attention for Robust Vision-Based Tracking," Int'l J. of Computer Vision, Vol. 35, No. 1, 1999, pp. 45-63.
[23] M. La Cascia, J. Isidoro, and S. Sclaroff, "Head Tracking via Robust Registration in Texture Map Images," Proc. Int'l Conf. Computer Vision and Pattern Recognition (CVPR 98), IEEE CS Press, Los Alamitos, Calif., 1998, pp. 508-514.
[24] M. La Cascia and S. Sclaroff, “Fast, Reliable Head Tracking Under Varying Illumination,” Proc. Conf. Computer Vision and Pattern Recognition, 1999.
[25] H. Li, P. Roivainen, and R. Forchheimer, "3D Motion Estimation in Model-Based Facial Image Coding," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 545-555, June 1993.
[26] B. Moghaddam and A. Pentland, “Probabilistic Visual Learning for Object Representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 7, pp. 696-710, July 1997.
[27] N. Oliver, F. Berard, J. Coutaz, and A. Pentland, “LAFTER: Lips and Face Tracker,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 100-110, San Juan, Puerto Rico, 1997.
[28] A. Rosenfeld, ed., Multiresolution Image Processing and Analysis. New York: Springer-Verlag, 1984.
[29] H. Rowley, S. Baluja, and T. Kanade, "Neural Network-Based Face Detection," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 1, Jan. 1998, pp. 23-38.
[30] A. Schödl, A. Haro, and I. Essa, “Head Tracking Using a Textured Polygonal Model,” Proc. 1998 Workshop Perceptual User Interfaces, 1998.
[31] S. Sclaroff and J. Isidoro, “Active Blobs,” Proc. Sixth Int'l Conf. Computer Vision, pp. 1146-1153, 1998.
[32] A. Shashua, “Geometry and Photometry in 3D Visual Recognition,” PhD dissertation, Dept. of Brain and Cognitive Sciences, Massachusetts Inst. of Technology, Cambridge, Nov. 1992.
[33] D. Terzopoulos, "Image Analysis Using Multigrid Relaxation Methods," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, no. 2, pp. 129-139, Mar. 1986.
[34] D. Terzopoulos and K. Waters, "Analysis and Synthesis of Facial Image Sequences Using Physical and Anatomical Models," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 6, pp. 569-579, 1993.
[35] Y. Yacoob and L.S. Davis, “Recognizing Human Facial Expression from Long Image Sequences Using Optical Flow,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 18, no. 6, pp. 636-642, June 1996.
[36] A.L. Yuille, D.S. Cohen, and P.W. Hallinan, “Feature Extraction from Faces Using Deformable Templates,” Proc. Int'l Conf. Pattern Recognition, 1994.

Index Terms:
Visual tracking, real-time vision, illumination, motion estimation, computer human interfaces.
Citation:
Marco La Cascia, Stan Sclaroff, Vassilis Athitsos, "Fast, Reliable Head Tracking under Varying Illumination: An Approach Based on Registration of Texture-Mapped 3D Models," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 22, no. 4, pp. 322-336, April 2000, doi:10.1109/34.845375
Usage of this product signifies your acceptance of the Terms of Use.