This Article 
 Bibliographic References 
 Add to: 
Robust and Accurate Visual Echo Cancelation in a Full-duplex Projector-Camera System
October 2008 (vol. 30 no. 10)
pp. 1831-1840
In this paper we study the problem of "visual echo" in a full-duplex projector-camera system for tele-collaboration applications. Visual echo is defined as the appearance of projected contents observed by the camera. It can potentially saturate the projected contents, similar to audio echo in telephone conversation. Our approach to visual echo cancelation includes an off-line calibration procedure that records the geometric and photometric transfer between the projector and the camera in a look-up table. During run-time, projected contents in the captured video are identified using the calibration information and suppressed, therefore achieving the goal of canceling visual echo. Our approach can accurately handle full color images under arbitrary reflectance of display surfaces and photometric response of the projector or camera. It is robust to geometric registration errors and quantization effect, therefore particularly effective for high-frequency contents such as texts and hand drawings. We demonstrate the effectiveness of our approach with a variety of real images in a full-duplex projector-camera system.

[1] M. Brown, A. Majumder, and R. Yang, “Camera-Based Calibration Techniques for Seamless Multiprojector Displays,” IEEE Trans. Visualization and Computer Graphics, vol. 11, no. 2, pp. 193-206, Mar./Apr. 2005.
[2] T.J. Cham, J. Rehg, R. Sukthankar, and G. Sukthankar, “Shadow Elimination and Occluder Light Suppression for Multi-Projector Displays,” Proc. IEEE Computer Vision and Pattern Recognition (CVPR '03), 2003.
[3] H. Chen, R. Sukthankar, and G. Wallace, “Scalable Alignment of Large-Format Multi-Projector Displays Using Camera Homography Trees,” Proc. 13th IEEE Visualization Conf. (VIS '02), pp. 339-346, 2002.
[4] P.E. Debevec and J. Malik, “Recovering High Dynamic Range Radiance Maps from Photographs,” Proc. ACM SIGGRAPH '97, pp.369-378, 1997.
[5] M. Flagg, J. Summet, and J. Rehg, “Improving the Speed of Virtual Rear Projection: A GPU-Centric Architecture,” Proc. IEEE Int'l Workshop Projector-Camera Systems (Procams '05), 2005.
[6] K. Fujii, M.D. Grossberg, and S.K. Nayar, “A Projector-Camera System with Real-Time Photometric Adaptation for Dynamic Environments,” Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '05), vol. 1, pp. 814-821, 2005.
[7] D. Hall, C. Le Gal, J. Martin, O. Chomat, T. Kapuscinski, and J. Crowley, “Magicboard: A Contribution to an Intelligent Office Environment,” Proc. Seventh Int'l Symp. Intelligent Robotic Systems (SIRS '99), 1999.
[8] R. Hartley and A. Zisserman, Multiple-View Geometry. Cambridge Univ. Press, 2000.
[9] C. Jaynes, B. Seales, K. Calvert, Z. Fei, and J. Griffioen, “The Metaverse: A Collection of Inexpensive, Self-Configuring, Immersive Environments,” Proc. Seventh Int'l Workshop Immersive Projection Technology/Ninth Eurographics Workshop Virtual Environments, 2003.
[10] C. Jaynes, S. Webb, and R. Steele, “Camera-Based Detection and Removal of Shadows from Interactive Multiprojector Displays,” IEEE Trans. Visualization and Computer Graphics, vol. 10, no. 3, pp.290-301, May/June 2004.
[11] A. Majumder, D. Jones, M. McCrory, M.E. Papka, and R. Stevens, “Using a Camera to Capture and Correct Spatial Photometric Variation in Multi-Projector Displays,” Proc. IEEE Int'l Workshop Projector-Camera Systems (Procams '03), 2003.
[12] A. Majumder and R. Stevens, “Color Nonuniformity in Projection-Based Displays: Analysis and Solutions,” IEEE Trans. Visualization and Computer Graphics, vol. 10, no. 2, Apr.-June 2003.
[13] E. Mynatt, T. Igarashi, W. Edwards, and A. LaMarca, “Flatland: New Dimensions in the Whiteboard,” Proc. ACM Conf. Human Factors in Computing Systems (CHI '99), pp. 346-353, 1999.
[14] C.S. Pinhanez, “The Everywhere Displays Projector: A Device to Create Ubiquitous Graphical Interfaces,” Proc. Third Int'l Conf. Ubiquitous Computing (Ubicomp '01), 2001.
[15] R. Raskar, M. Brown, R. Yang, W. Chen, G. Welch, H. Towles, B. Seales, and H. Fuchs, “Multi-Projector Displays Using Camera-Based Registration,” Proc. 10th IEEE Visualization Conf. (VIS '99), pp. 161-168, 1999.
[16] R. Raskar, J. van Baar, P. Beardsley, T. Willwacher, S. Rao, and C. Forlines, “ilamps: Geometrically Aware and Self-Configuring Projectors,” Proc. ACM Siggraph '03, 2003.
[17] R. Raskar, G. Welch, M. Cutts, A. Lake, L. Stesin, and H. Fuchs, “The Office of the Future: A Unified Approach to Image-Based Modeling and Spatially Immersive Displays,” Computer Graphics, vol. 32, Annual Conference Series, pp. 179-188, 1998.
[18] R. Surati, “Scalable Self-Calibrating Display Technology for Seamless Large-Scale Displays,” PhD dissertation, Dept. of Computer Science, Massachusetts Inst. of Tech nology, 1998.
[19] Y. Sato, Y. Kobayashi, and H. Koike, “Fast Tracking of Hands and Fingertips in Infrared Images for Augmented Desk Interface,” Proc. Fourth IEEE Int'l Conf. Automatic Face and Gesture Recognition (AFGR '00), 2000.
[20] M.C. Stone, “Color and Brightness Appearance Issues in Tiled Displays,” IEEE Computer Graphics and Applications, vol. 21, no. 5, pp. 58-66, 2001.
[21] N. Takao, J. Shi, and S. Baker, “Telegraffiti: A Camera-Projector Based Remote Sketching System with Hand-Based User Interface and Automatic Session Summarization,” Int'l J. Computer Vision, vol. 53, no. 2, pp. 115-133, 2003.
[22] N. Takao, J. Shi, and S. Baker, “Tele-Grafitti,” Technical Report CMU-RI-TR-02-10, Robotics Inst., Carnegie Mellon Univ., 2002.
[23] N. Takao, S. Baker, and J. Shi, “Steady-State Feedback Analysis of Tele-Graffiti,” Proc. IEEE Int'l Workshop Projector-Camera Systems (Procams '03), Oct. 2003.
[24] P. Wellner, “Interacting with Paper on the Digital Desk,” Comm. ACM, vol. 36, no. 7, pp. 86-97, 1993.
[25] R. Yang, D. Gotz, J. Hensley, H. Towles, and M. Brown, “PixelFlex: A Reconfigurable Multi-Projector Display System,” Proc. 12th IEEE Visualization Conf. (VIS '01), pp. 167-174, 2001.
[26] K.-J. Yoon and I.-S. Kweon, “Locally Adaptive Support-Weight Approach for Visual Correspondence Search,” Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition (CVPR '05), pp. 924-931, 2005.
[27] Z. Zhang, “Computer Vision Techniques for Remote Collaboration Using Physical Whiteboards, Projectors and Cameras,” Computer Vision for Interactive and Intelligent Environments, C.Jaynes and R. Collins, eds., pp. 109-122, IEEE Computer Society, 2006.
[28] H. Zhou, Z. Zhang, and T. Huang, “Visual Echo Cancellation in a Projector-Camera-Whiteboard System,” Proc. Int'l Conf. Image Processing (ICIP '04), pp. 2885-2888, 2004.

Index Terms:
projector-camera system, teleconferencing, visual echo cancelation, collaboration, camera calibration
Miao Liao, Ruigang Yang, Zhengyou Zhang, "Robust and Accurate Visual Echo Cancelation in a Full-duplex Projector-Camera System," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 10, pp. 1831-1840, Oct. 2008, doi:10.1109/TPAMI.2007.70828
Usage of this product signifies your acceptance of the Terms of Use.