This Article 
 Bibliographic References 
 Add to: 
On the Removal of Shadows from Images
January 2006 (vol. 28 no. 1)
pp. 59-68
This paper is concerned with the derivation of a progression of shadow-free image representations. First, we show that adopting certain assumptions about lights and cameras leads to a 1D, gray-scale image representation which is illuminant invariant at each image pixel. We show that as a consequence, images represented in this form are shadow-free. We then extend this 1D representation to an equivalent 2D, chromaticity representation. We show that in this 2D representation, it is possible to relight all the image pixels in the same way, effectively deriving a 2D image representation which is additionally shadow-free. Finally, we show how to recover a 3D, full color shadow-free image representation by first (with the help of the 2D representation) identifying shadow edges. We then remove shadow edges from the edge-map of the original image by edge in-painting and we propose a method to reintegrate this thresholded edge map, thus deriving the sought-after 3D shadow-free image.

[1] G.J. Klinker, S.A. Shafer, and T. Kanade, “A Physical Approach to Color Image Understanding,” Int'l J. Computer Vision, vol. 4, pp. 7-38, 1990.
[2] M.J. Swain and D.H. Ballard, “Color Indexing,” Int'l J. Computer Vision, vol. 7, no. 1, pp. 11-32, 1991.
[3] H. Jiang and M. Drew, “Tracking Objects with Shadows,” Proc. Int'l Conf. Multimedia and Expo, 2003.
[4] H. Barrow and J. Tenenbaum, “Recovering Intrinsic Scene Characteristics from Images,” Computer Vision Systems, A. Hanson and E. Riseman, eds., pp. 3-26, Academic Press, 1978.
[5] Y. Weiss, “Deriving Intrinsic Images from Image Sequences,” Proc. Int'l Conf. Computer Vision, pp. II: 68-75, 2001.
[6] E.H. Land, “Lightness and Retinex Theory,” J. Optical Soc. Am., A, vol. 61, pp. 1-11, 1971.
[7] B.K. Horn, “Determining Lightness from an Image,” Computer Graphics and Image Processing, vol. 3, pp. 277-299, 1974.
[8] A. Blake, “On Lightness Computation in the Mondrian World,” Proc. Wenner-Gren Conf. Central and Peripheral Mechanisms in Colour Vision, pp. 45-59, 1983.
[9] G. Brelstaff and A. Blake, “Computing Lightness,” Pattern Recognition Letters, vol. 5, pp. 129-138, 1987.
[10] A. Hurlbert, “Formal Connections between Lightness Algorithms,” J. Optical Soc. Am., A, vol. 3, no. 10, pp. 1684-1693, 1986.
[11] E.H. Land, “The Retinex Theory of Color Vision,” Scientific Am., pp. 108-129, 1977.
[12] L.T. Maloney and B.A. Wandell, “Color Constancy: A Method for Recovering Surface Spectral Reflectance,” J. Optical Soc. Am., A, vol. 3, no. 1, pp. 29-33, 1986.
[13] G. Buchsbaum, “A Spatial Processor Model for Object Colour Perception,” J. Franklin Inst., vol. 310, pp. 1-26, 1980.
[14] D. Forsyth, “A Novel Algorithm for Colour Constancy,” Int'l J. Computer Vision, vol. 5, no. 1, pp. 5-36, 1990.
[15] G.D. Finlayson, S.D. Hordley, and P.M. Hubel, “Color by Correlation: A Simple, Unifying Framework for Color Constancy,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1209-1221, Nov. 2001.
[16] D.H. Brainard and W.T. Freeman, “Bayesian Method for Recovering Surface and Illuminant Properties from Photosensor Responses,” Proc. IS&T/SPIE Symp. Electronic Imaging Science and Technology, vol. 2179, pp. 364-376, 1994.
[17] V. Cardei, “A Neural Network Approach to Colour Constancy,” PhD dissertation, School of Computing Science, Simon Fraser Univ., 2000.
[18] K. Barnard, “Practical Colour Constancy,” PhD dissertation, School of Computing Science, Simon Fraser Univ., 2000.
[19] S. Tominaga and B.A. Wandell, “Standard Surface-Reflectance Model and Illuminant Estimation,” J. Optical Soc. Am. , A, vol. 6, no. 4, pp. 576-584, 1996.
[20] H.-C. Lee, “Method for Computing Scene-Illuminant Chromaticity from Specular Highlights,” Color, G.E. Healey, S.A. Shafer, and L.B. Wolff, eds., pp. 340-347, Boston: Jones and Bartlett, 1992.
[21] Th. Gevers and H.M.G. Stokman, “Classifying Color Transitions into Shadow-Geometry, Illumination Highlight or Material Edges,” Proc. Int'l Conf. Image Processing, pp. 521-525, 2000.
[22] B.V. Funt and G.D. Finlayson, “Color Constant Color Indexing,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 17, no. 5, pp. 522-529, May 1995.
[23] G. Finlayson, S. Chatterjee, and B. Funt, “Color Angular Indexing,” Proc. Fourth European Conf. Computer Vision, vol. II, pp. 16-27, 1996.
[24] T. Gevers and A. Smeulders, “Color Based Object Recognition,” Pattern Recognition, vol. 32, pp. 453-464, 1999.
[25] M. Stricker and M. Orengo, “Similarity of Color Images,” Proc. SPIE Conf. Storage and Retrieval for Image and Video Databases III, vol. 2420, pp. 381-392, 1995.
[26] D. Berwick and S.W. Lee, “A Chromaticity Space for Specularity, Illumination Color- and Illumination Pose-Invariant 3-D Object Recognition,” Proc. Sixth Int'l Conf. Computer Vision, 1998.
[27] G. Finlayson, B. Schiele, and J. Crowley, “Comprehensive Colour Image Normalization,” Proc. European Conf. Computer Vision, pp. 475-490, 1998.
[28] G. Finlayson and S. Hordley, “Color Constancy at a Pixel,” J. Optical Soc. Am. A, vol. 18, no. 2, pp. 253-264, UK Patent application no. 0000682.5, under review, British Patent Office, 2001.
[29] J.A. Marchant and C.M. Onyango, “Shadow Invariant Classification for Scenes Illuminated by Daylight,” J. Optical Soc. Am., A, vol. 17, no. 12, pp. 1952-1961, 2000.
[30] R. Hunt, The Reproduction of Colour, fifth ed. Fountain Press, 1995.
[31] J.L.N.J. Romero, J. Hernandez-Andres, and E. Valero, “Testing Spectral Sensitivity of Sensors for Color Invariant at a Pixel,” Proc. Second Computer Graphics, Imaging, and Vision Conf., Apr. 2004.
[32] G. Finlayson and M. Drew, “4-Sensor Camera Calibration for Image Representation Invariant to Shading, Shadows, Lighting and Specularities,” Proc. Int'l Conf. Computer Vision, pp. 473-480, 2001.
[33] B.K. Horn, Robot Vision. MIT Press, 1986.
[34] G. Wyszecki and W. Stiles, Color Science: Concepts and Methods, Quantitative Data and Formulas, second ed. New York: Wiley, 1982.
[35] G.D. Finlayson, M.S. Drew, and C. Lu, “Intrinsic Images by Entropy Minimisation,” Proc. European Conf. Computer Vision, 2004.
[36] M.H. Brill and G. Finlayson, “Illuminant Invariance from a Single Reflected Light,” Color Research and Application, vol. 27 pp. 45-48, 2002.
[37] M. Stokes, M. Anderson, S. Chandrasekar, and R. Motta, “A Standard Default Color Space for the Internet— SRGB,” http://www.color.orgsRGB.html, 1996.
[38] B. Funt, K. Barnard, and L. Martin, “Is Machine Colour Constancy Good Enough?” Proc. Fifth European Conf. Computer Vision, pp. 455-459, June 1998.
[39] M. Drew, C. Chen, S. Hordley, and G. Finlayson, “Sensor Transforms for Invariant Image Enhancement,” Proc. 10th Color Imaging Conf.: Color, Science, Systems, and Applications, pp. 325-329, 2002.
[40] G.D. Finlayson, M.S. Drew, and B.V. Funt, “Spectral Sharpening: Sensor Transformations for Improved Color Constancy,” J. Optical Soc. Am., A, vol. 11, no. 5, pp. 1553-1563, 1994.
[41] M.S. Drew, G.D. Finlayson, and S.D. Hordley, “Recovery of Chromaticity Image Free from Shadows via Illumination Invariance,” Proc. Workshop Color and Photometric Methods in Computer Vision, pp. 32-39, 2003.
[42] B. Funt, M. Drew, and M. Brockington, “Recovering Shading from Color Images,” Proc. Second European Conf. Computer Vision, G. Sandini, ed., pp. 124-132, May 1992.
[43] R. Frankot and R. Chellappa, “A Method for Enforcing Integrability in Shape from Shading Algorithms,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 10, pp. 439-451, 1988.
[44] R. Jain, R. Kasturi, and B. Schunck, Machine Vision. McGraw-Hill, 1995.
[45] D. Comaniciu and P. Meer, “Mean Shift Analysis and Applications,” Proc. Seventh Int'l Conf. Computer Vision, pp. 1197-1203, 1999.
[46] J. Canny, “A Computational Approach to Edge Detection,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 8, pp. 679-698, 1986.

Index Terms:
Index Terms- Shadow removal, illuminant invariance, reintegration.
Graham D. Finlayson, Steven D. Hordley, Cheng Lu, Mark S. Drew, "On the Removal of Shadows from Images," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 1, pp. 59-68, Jan. 2006, doi:10.1109/TPAMI.2006.18
Usage of this product signifies your acceptance of the Terms of Use.