This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Eye Gaze Correction with Stereovision for Video-Teleconferencing
July 2004 (vol. 26 no. 7)
pp. 956-960
ASCII Text
x 
 
Ruigang Yang, Zhengyou Zhang, "Eye Gaze Correction with Stereovision for Video-Teleconferencing," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 956-960, July, 2004.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2004.27,
author = {Ruigang Yang and Zhengyou Zhang},
title = {Eye Gaze Correction with Stereovision for Video-Teleconferencing},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {26},
number = {7},
issn = {0162-8828},
year = {2004},
pages = {956-960},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2004.27},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Eye Gaze Correction with Stereovision for Video-Teleconferencing
IS - 7
SN - 0162-8828
SP956
EP960
EPD - 956-960
A1 - Ruigang Yang,
A1 - Zhengyou Zhang,
PY - 2004
KW - Stereoscopic vision
KW - eye-gaze correction
KW - model-based tracking
KW - head tracking
KW - pose determination.
VL - 26
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 

Abstract—The lack of eye contact in desktop video teleconferencing substantially reduces the effectiveness of video contents. While expensive and bulky hardware is available on the market to correct eye gaze, researchers have been trying to provide a practical software-based solution to bring video-teleconferencing one step closer to the mass market. This paper presents a novel approach: Based on stereo analysis combined with rich domain knowledge (a personalized face model), we synthesize, using graphics hardware, a virtual video that maintains eye contact. A 3D stereo head tracker with a personalized face model is used to compute initial correspondences across two views. More correspondences are then added through template and feature matching. Finally, all the correspondence information is fused together for view synthesis using view morphing techniques. The combined methods greatly enhance the accuracy and robustness of the synthesized views. Our current system is able to generate an eye-gaze corrected video stream at five frames per second on a commodity 1 GHz PC.

[1] T.J. Cham, S. Krishnamoorthy, and M. Jones, Analogous View Transfer for Gaze Correction in Video Sequences Proc. Int'l Conf. Automation, Robotics, Control and Vision, 2002.
[2] D.H. Douglas and T.K. Peucker, Algorithms for the Reduction of the Number of Points Required to Represent a Digitized Line or Its Caricature Canadian Cartographer, vol. 10, no. 2, pp. 112-122, 1973.
[3] J. Gemmell, C.L. Zitnick, T. Kang, K. Toyama, and S. Seitz, Gaze-Awareness for Videoconferencing: A Software Approach IEEE Multimedia, vol. 7, no. 4, pp. 26-35, Oct. 2000.
[4] M. Jones, Multidimensional Morphable Models: A Framework for Representing and Matching Object Classes Int'l J. Computer Vision, vol. 29, no. 2, pp. 107-131, Aug. 1998.
[5] M. Kass, A. Witkin, and D. Terzopoulos, Snake: Active Contour Models Int'l J. Computer Vision, vol. 1, no. 4, pp. 321-331, 1987.
[6] R. Kollarits, C. Woodworth, J. Ribera, and R. Gitlin, An Eye-Contact Camera/Display System for Videophone Applications Using a Conventional Direct-View LCD SID Digest, 1995.
[7] J. Liu, I. Beldie, and M. Wopking, A Computational Approach to Establish Eye-Contact in Videocommunication Proc. Int'l Workshop Stereoscopic and Three Dimensional Imaging (IWS3DI), pp. 229-234, 1995.
[8] Z. Liu, Z. Zhang, C. Jacobs, and M. Cohen, Rapid Modeling of Animated Faces From Video J. Visualization and Computer Animation, vol. 12, no. 4, pp. 227-240, 2001.
[9] C. Loop and Z. Zhang, “Computing Rectifying Homographies for Stereo Vision,” Proc. Computer Vision and Pattern Recognition '99, vol. I, pp. 125-131, 1999.
[10] L. Mhlbach, B. Kellner, A. Prussog, and G. Romahn, The Importance of Eye Contact in a Videotelephone Service Proc. 11th Int'l Symp. Human Factors in Telecomm., 1985.
[11] M. Ott, J. Lewis, and I. Cox, Teleconferencing Eye Contact Using a Virtual Camera Proc. Conf. Human Factors in Computing Systems, pp. 119-110, 1993.
[12] S. Pollard, J. Porrill, J. Mayhew, and J. Frisby, Disparity Gradient, Lipschitz Continuity, and Computing Binocular Correspondance Robotics Research: Proc. Third Int'l Symp., O.D. Faugeras and G. Giralt, eds., vol. 30, pp. 19-26, 1986.
[13] J. Shi and C. Tomasi, Good Features to Track Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 593-600, 1994.
[14] R. Stokes, "Human Factors and Appearance Design Considerations of the Mod II PicturePhone Station Set," IEEE Trans. Communication Technology, Vol. COM-17, No. 2, April 1969, pp. 318-323.
[15] R. Yang and Z. Zhang, Eye Gaze Correction with Stereovision for Video-Teleconferencing Proc. European Conf. Computer Vision, pp. 479-494, 2002.
[16] R. Yang and Z. Zhang, Model-Based Head Pose Tracking With Stereovision Proc. Fifth IEEE Int'l Conf. Automatic Face and Gesture Recognition, May 2002.
[17] Z. Zhang, A Flexible New Technique for Camera Calibration IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 11, pp. 1330-1334, Nov. 2000.

Index Terms:
Stereoscopic vision, eye-gaze correction, model-based tracking, head tracking, pose determination.
Citation:
Ruigang Yang, Zhengyou Zhang, "Eye Gaze Correction with Stereovision for Video-Teleconferencing," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 7, pp. 956-960, July 2004, doi:10.1109/TPAMI.2004.27
Usage of this product signifies your acceptance of the Terms of Use.