This Article 
 Bibliographic References 
 Add to: 
Recovering Intrinsic Images from a Single Image
September 2005 (vol. 27 no. 9)
pp. 1459-1472
Interpreting real-world images requires the ability distinguish the different characteristics of the scene that lead to its final appearance. Two of the most important of these characteristics are the shading and reflectance of each point in the scene. We present an algorithm that uses multiple cues to recover shading and reflectance intrinsic images from a single image. Using both color information and a classifier trained to recognize gray-scale patterns, given the lighting direction, each image derivative is classified as being caused by shading or a change in the surface's reflectance. The classifiers gather local evidence about the surface's form and color, which is then propagated using the Generalized Belief Propagation algorithm. The propagation step disambiguates areas of the image where the correct classification is not clear from local evidence. We use real-world images to demonstrate results and show how each component of the system affects the results.

[1] H.G. Barrow and J.M. Tenenbaum, “Recovering Intrinsic Scene Characteristics from Images,” Computer Vision Systems, A. Hanson and E. Riseman, eds., pp. 3-26. Academic Press, 1978.
[2] M. Bell and W.T. Freeman, “ Learning Local Evidence for Shading and Reflection,” Proc. Int'l Conf. Computer Vision, 2001.
[3] J. Besag, “Statistical Analysis of Non-Lattice Data,” The Statistician, vol. 24, no. 3, pp. 179-195, Sept. 1975.
[4] G.D. Finlayson, S.D. Hordley, and M.S. Drew, “Removing Shadows from Images,” Proc. European Conf. Computer Vision, pp. 823-836. 2002.
[5] W.T. Freeman, E.C. Pasztor, and O.T. Carmichael, “Learning Low-Level Vision,” Int'l J. Computer Vision, vol. 40, no. 1, pp. 25-47, 2000.
[6] W.T. Freeman and P.A. Viola, “Bayesian Model of Surface Perception,” Advances in Neural Information Processing Systems, Apr. 1998.
[7] Y. Freund and R.E. Schapire, “A Decision-Theoretic Generalization of On-Line Learning and an Application to Boosting,” J. Computer and System Sciences, vol. 55, no. 1, pp. 119-139, 1997.
[8] J. Friedman, T. Hastie, and R. Tibshirami, “Additive Logistic Regression: A Statistical View of Boosting,” The Annals of Statistics, vol. 38, no. 2, pp. 337-374, 2000.
[9] B.V. Funt, M.S. Drew, and M. Brockington, “Recovering shading from Color Images,” Proc. Second European Conf. Computer Vision, G. Sandini, ed., pp. 124-132, May 1992.
[10] J.J. Gibson, The Senses Considered as Perceptual Systems, chapter 10. Houghton Mifflin, 1966.
[11] D. Heeger and J. Bergen, “Pyramid-Based Texture Analysis/Synthesis,” Proc. SIGGRAPH, Computer Graphics, pp 229-238, Aug. 1995.
[12] B.K.P. Horn, Robot Vision, chapter 9. Cambridge, Mass.: MIT Press, 1986.
[13] J. Lafferty, F. Pereira, and A. McCallum, “Conditional Random Fields: Probabilistic Models for Segmenting and Labeling Sequence Data,” Proc. Int'l Conf. Machine Learning (ICML), 2001.
[14] E.H. Land and J.J. McCann, “Lightness and Retinex Theory,” J. Optical Soc. Am., vol. 61, pp. 1-11, 1971.
[15] T. Leung and J. Malik, “Recognizing Surfaces Using Three-Dimensional Textons,” Proc. IEEE Int'l Conf. Computer Vision, 1999.
[16] Y. Matsushita, K. Nishino, K. Ikeuchi, and M. Sakauchi, “Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance,” Proc. 2003 Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 3-10, 2003.
[17] A.V. Oppenheim, R.W. Shafer, and J. Thomas, G. Stockham, “Nonlinear Filtering of Multiplied and Convolved Signals,” IEEE Trans. Audio and Electroacoustics, vol. 16, no. 3, pp. 437-466, Sept. 1968.
[18] A.P. Pentland, “Linear Shape from Shading,” lnt'l J. Computer Vision, vol. 1, no. 4, pp. 153-162, 1990.
[19] J.M. Rubin and W.A. Richards, “Color Vision and Image Intensities: When Are Changes Material,” Biological Cybernetics, vol. 45, pp. 215-226, 1982.
[20] E.P. Simoncelli and W.T. Freeman, “The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation,” Proc. Int'l Conf. Image Processing, vol. 3, pp. 444-447, Oct. 1995
[21] P. Sinha and E.H. Adelson, “Recovering Reflectance in a World of Painted Polyhedra,” Proc. Fourth Int'l Conf. Computer Vision, pp. 156-163, 1993.
[22] K. Tieu and P. Viola, “Boosting Image Retrieval,” Proc. IEEE Computer Vision and Pattern Recognition, vol. 1, pp. 228-235, 2000.
[23] Y. Weiss, “Deriving Intrinsic Images from Image Sequences,” Proc. Int'l Conf. Computer Vision, 2001.
[24] J. Yedidia, W.T. Freeman, and Y. Weiss, “Generalized belief propagation,” Advances in Neural Information Processing Systems 13, T. K. Leen, H.G. Diettrich, and V. Tresp, eds., pp. 689-695. 2001.

Index Terms:
Index Terms- Computer vision, machine learning, reflectance, shading, boosting, belief propagation.
Marshall F. Tappen, William T. Freeman, Edward H. Adelson, "Recovering Intrinsic Images from a Single Image," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 27, no. 9, pp. 1459-1472, Sept. 2005, doi:10.1109/TPAMI.2005.185
Usage of this product signifies your acceptance of the Terms of Use.