This Article 
 Bibliographic References 
 Add to: 
Camera-Based Detection and Removal of Shadows from Interactive Multiprojector Displays
May/June 2004 (vol. 10 no. 3)
pp. 290-301

Abstract—Front-projection displays are a cost-effective and increasingly popular method for large format visualization and immersive rendering of virtual models. New approaches to projector tiling, automatic calibration, and color balancing have made multiprojector display systems feasible without undue infrastructure changes and maintenance. As a result, front-projection displays are being used to generate seamless, visually immersive worlds for virtual reality and visualization applications with reasonable cost and maintenance overhead. However, these systems suffer from a fundamental problem: Users and other objects in the environment can easily and inadvertently block projectors, creating shadows on the displayed image. Shadows occlude potentially important information and detract from the sense of presence an immersive display may have conveyed. We introduce a technique that detects and corrects shadows in a multiprojector display while it is in use. Cameras observe the display and compare observations with an expected image to detect shadowed regions. These regions are transformed to the appropriate projector frames, where corresponding pixel values are increased and/or attenuated. In display regions where more than one projector contributes to the image, shadow regions are eliminated.

[1] T. Cham, R. Sukthankar, J. Rehg, and G.G. Sukthankar, Shadow Elimination and Occluder Light Suppression for Multi-Projector Displays technical report, Cambridge Research Laboratory, Mar. 2002.
[2] H. Chen, R. Sukthankar, G. Wallace, and T. Cham, Calibrating Scalable Multi-Projector Displays Using Camera Homography Trees Computer Vision and Pattern Recognition, 2001.
[3] Y. Chen, D. Clark, A. Finkelstein, T. Housel, and K. Li, Automatic Alignment of High-Resolution Multi-Projector Displays Using an Un-Calibrated Camera Proc. IEEE Visualization 2000, pp. 125-130, Oct. 2000.
[4] Y. Chen, D. Clark, A. Finkelstein, T. Housel, and K. Li, Methods for Acheiving Seamlessness on High-Resolution Displays Using Uncalibrated Cameras Proc. IEEE Visualization, 2000.
[5] Z. Chen and S. Y. Ho, Incremental Model Building of Polyhedral Objects Using Structured Light Pattern Recognition, vol. 26, no. 1, pp. 33-46, 1993.
[6] L. Childers, T. Disz, R. Olson, M.E. Papka, R. Stevens, and T. Udesh, Access Grid: Immersive Group-to-Group Collaborative Visualization technical report, Argone Nat'l Laboratories Tech-Report, May 2000.
[7] C. Cruz-Neira, D.J. Sandin, and T.A. DeFanti, Surround-Screen Projection-Based Virtual Reality: The Design and Implementation of the CAVE Computer Graphics (SIGGRAPH '93 Proc.), J.T. Kajiya, ed., vol. 27, pp. 135-142, Aug. 1993.
[8] M. Czernuszenko, D. Pape, D. Sandin, T. DeFanti, L. Dawe, and M. Brown, The ImmersaDesk and Infinity Wall Projection-Based Virtual Reality Displays Computer Graphics, May 1997.
[9] O.D. Faugeras, Three-Dimensional Computer Vision: A Geometric Approach. MIT Press, 1993.
[10] T. Funkhouser and K. Li, "Large-Format Displays," IEEE Computer Graphics and Applications, vol. 20, no. 4, July/Aug. 2000, pp. 20-21.
[11] M. Herald, I. Judson, J. Paris, and R. Stevens, Developing Tiled Projection Display Systems Proc. Fourth Int'l Immersive Projection Technology Workshop, June 2000.
[12] M. Hereld, I.R. Judson, and R.L. Stevens, "Tutorial: Introduction to Building Projection-based Tiled Display Systems," IEEE Computer Graphics and Applications, Vol. 20, No. 4, July/Aug. 2000, pp. 22-28.
[13] G. Humphreys, M. Eldridge, I. Buck, G. Stoll, M. Everett, and P. Hanrahan, WireGL: A Scalable Graphics System for Clusters ACM Proc. SIGGRAPH, pp. 129-140, July 2001.
[14] C. Jaynes, B. Seales, K. Calvert, Z. Fei, and J. Griffieon, The Metaverse a Collection of Inexpensive, Self-Configuring, Immersive Environments Proc. Int'l Workshop Immersive Projection Technology and Virtual Environments, pp. 115-123, 2003.
[15] C. Jaynes, S. Webb, and R.M. Steele, A Scalable Framework for High-Resolution Immersive Displays Int'l J. Inst. Electronics and Telecomm. Engineers (IETE), vol. 48, nos. 3-4, pp. 273-280, Aug. 2002.
[16] C. Jaynes, S. Webb, R.M. Steele, M. Brown, and B. Seales, Dynamic Shadow Removal from Front Projection Displays Proc. IEEE Visualization, pp. 175-182, Oct. 2001.
[17] A. Majumder, Z. He, H. Towles, and G. Welch, Color Calibration of Projectors for Large Tiled Displays Proc. IEEE Visualization 2000, Oct. 2000.
[18] A. Majumder and G. Welch, Computer Graphics Optique: Optical Superposition of Projected Computer Graphics Proc. Fifth Immersive Projection Technology Workshop, in conjunction with Seventh Eurographics Workshop Virtual Environments, May 2001.
[19] PowerWall, powerwall.html, year?
[20] D. Ramakrishnan, Inverse Super Resolution Reconstruction: Superposition of Projected Imagery in the Framebuffer Optique master's thesis, Computer Science Dept., Univ. of Kentucky, 2003.
[21] R. Raskar et al., "Multi-Projector Displays Using Camera-Based Registration," Proc. of IEEE Visualization 1999, ACM Press, New York, Oct. 1999.
[22] R. Raskar, J. van Baar, P. Beardsley, T. Willwacher, S. Rao, and C. Forlines, ilamps: Geometrically Aware and Self-Configuring Projectors ACM Trans. Graphics, Proc. ACM SIGGRAPH 2003, pp. 809-817, 2003.
[23] R. Raskar, G. Welch, and H. Fuchs, Seamless Projection Overlaps Using Image Warping and Intensity Blending Proc. Fourth Int'l Conf. Virutal Systems and Multimedia, Nov. 1998.
[24] R. Yang, D. Gotz, J. Hensly, H. Towles, and M. Brown, Pixelflex: A Reconfigurable, Multiprojector Display Proc. IEEE Visualization, Oct. 2001.
[25] F. Schoffel, W. Kresse, S. Muller, and M. Unbescheiden, Do IPT Systems Fulfill Application Requirements? A Study on Luminance on Large-Scale Immersive Projection Devices Proc. Third Int'l IPT Workshop, pp. 281-292, 1999.
[26] R.M. Steele and C. Jaynes, Iterative Estimation for Active Subpixel Calibration: A Statistical Analysis of Accuracy and Behavior Proc. Int'l Workshop Statistical Analysis in Computer Vision, 2003.
[27] R.M. Steele, S. Webb, and C. Jaynes, Monitoring and Correction of Geometric Distortion in Projected Displays Proc. Central European Conf. Computer Vision, Visualization, and Graphics, pp. 429-436, Feb. 2001.
[28] R. Sukthankar, T. Cham, and G. Sukthankar, Dynamic Shadow Elimination for Multi-Projector Displays Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), 2001.
[29] J. Summet, R. Somani, G. Abowd, and J. Rehg, Interactive Walls: Addressing the Challenges of Large-Scale Interactive Surfaces Technical Report, git-gvu-02-35, Computer Science Dept., Georgia Inst. of Tech nology, 2002.
[30] R. Surati, Scalable Self-Calibrating Display Technology for Seamless Large-Scale Displays PhD thesis, Computer Science and Electrical Eng. Dept., Massachussetts Inst. of Tech nology, 1999.
[31] D.S. Tan and R. Pausch, Prex-Emptive Shadows: Eliminating the Blinding Light from Projectors Proc. CHI Conf. Human Factors in Computing Systems, 2002.
[32] J. Underkoffler, B. Ullmer, and H. Ishi, Emancipated Pixels: Real-World Graphics in the Luminous Room Proc. ACM SIGGRAPH, pp. 385-392, 1999.
[33] G. Wallace, H. Chen, and K. Li, Color Gamut Matching for Tiled Display Walls Proc. Immersive Projection Technology and Virtual Environments Workshop, pp. 293-302, 2003.
[34] G. Welch, H. Fuchs, R. Raskar, M. Brown, and H. Towles, Projected Imagery in Your Office in the Future IEEE Computer Graphics and Applications, pp. 62-67, July/Aug. 2000.
[35] R. Yang and G. Welch, Automatic Projector Display Surface Estimation Using Every-Day Imagery Proc. Ninth Int'l Conf. in Central Europe on Computer Graphics, Visualization and Computer Vision, 2001.

Index Terms:
Immersive display, multiple projector display, camera calibration, shadows.
Christopher Jaynes, Stephen Webb, R. Matt Steele, "Camera-Based Detection and Removal of Shadows from Interactive Multiprojector Displays," IEEE Transactions on Visualization and Computer Graphics, vol. 10, no. 3, pp. 290-301, May-June 2004, doi:10.1109/TVCG.2004.1272728
Usage of this product signifies your acceptance of the Terms of Use.