This Article 
 Bibliographic References 
 Add to: 
Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance
October 2004 (vol. 26 no. 10)
pp. 1336-1347
Variation in illumination conditions caused by weather, time of day, etc., makes the task difficult when building video surveillance systems of real world scenes. Especially, cast shadows produce troublesome effects, typically for object tracking from a fixed viewpoint, since it yields appearance variations of objects depending on whether they are inside or outside the shadow. In this paper, we handle such appearance variations by removing shadows in the image sequence. This can be considered as a preprocessing stage which leads to robust video surveillance. To achieve this, we propose a framework based on the idea of intrinsic images. Unlike previous methods of deriving intrinsic images, we derive time-varying reflectance images and corresponding illumination images from a sequence of images instead of assuming a single reflectance image. Using obtained illumination images, we normalize the input image sequence in terms of incident lighting distribution to eliminate shadowing effects. We also propose an illumination normalization scheme which can potentially run in real time, utilizing the illumination eigenspace, which captures the illumination variation due to weather, time of day, etc., and a shadow interpolation method based on shadow hulls. This paper describes the theory of the framework with simulation results and shows its effectiveness with object tracking results on real scene data sets.

[1] A. Laurentini,“The visual hull concept for silhouette-based image understanding,” IEEE Trans. on Pattern Analysis and Machine Intelligence, vol. 16, pp. 150-162, 1994.
[2] C. Jiang and M.O. Ward, Shadow Identification Proc. IEEE Int'l Conf. Computer Vision and Pattern Recognition, pp. 606-612, 1992.
[3] D. Koller, K. Daniilidis, and H.H. Nagel, Model-Based Object Tracking in Monocular Image Sequences of Road Traffic Scenes Int'l J. Computer Vision, vol. 10, pp. 257-281, 1993.
[4] E.H. Adelson and A.P. Pentland, The Perception of Shading and Reflectance Perception as Bayesian Inference, pp. 409-423, 1996.
[5] E.H. Land, The Retinex Theory of Color Vision Scientific Am., vol. 237G, no. 6, pp. 108-128, Dec. 1977.
[6] G.D. Finlayson, S.D. Hordley, and M.S. Drew, Removing Shadows from Images Proc. European Conf. Computer Vision, vol. 4, pp. 823-836, 2002.
[7] H.G. Barrow and J.M. Tenenbaum, Recovering Intrinsic Scene Characteristics from Images Computer Vision Systems, pp. 3-26, 1978.
[8] J. Huang and D. Mumford, “Statistics of Natural Images and Models,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, pp. 541-547, 1999.
[9] J. Stauder, R. Mech, and J. Ostermann, Detection of Moving Cast Shadows for Object Segmentation IEEE Trans. Multimedia, vol. 1, no. 1, pp. 65-76, Mar. 1999.
[10] K. Grauman, G. Shakhnarovich, and T. Darrell, A Bayesian Approach to Image-Based Visual Hull Reconstruction Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 187-194, 2003.
[11] K. Toyama, J. Krumm, B. Brumitt, and B. Meyers, “Wallflower: Principles and Practice of Background Maintenance,” Proc. Seventh Int'l Conf. Computer Vision, pp. 255-261, 1999.
[12] K.M. Cheung, S. Baker, and T. Kanade, Shape-from-Silhouette of Articulated Objects and Its Use for Human Body Kinematics Estimation and Motion Capture Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 77-84, 2003.
[13] M. Bell and W.T. Freeman, Learning Local Evidence for Shading and Reflection Proc. Int'l Conf. Computer Vision, vol. 1, pp. 670-677, 2001.
[14] M. Kilger, A Shadow Handler in a Video-Based Real-Time Traffic Monitoring System Proc. IEEE Workshop Applications of Computer Vision, pp. 11-18, 1992.
[15] M. Potmesil, Generating Octree Models of 3D Objects from Their Silhouettes in a Sequence of Images Computer Vision Graphics and Image Processing, vol. 40, pp. 1-20, 1987.
[16] M. Turk and A. Pentland, Eigenfaces for Recognition The J. Cognitive Neuroscience, vol. 3, no. 1, pp. 71-86, 1991.
[17] M.F. Tappen, W.T. Freeman, and E.H. Adelson, Recovering Intrinsic Images from a Single Image Advances in Neural Information Processing Systems 15, MIT Press, 2002.
[18] N. Katayama and S. Satoh, The SR-Tree: An Index Structure for High-Dimensional Nearest Neighbor Queries Proc. 1997 ACM SIGMOD Int'l Conf. Management of Data, pp. 369-380, 1997.
[19] N. Otsu, A Threshold Selection Method from Gray Level Histograms IEEE Trans. Systems, Man, and Cybernetics, vol. 9, pp. 62-66, 1979.
[20] W.H. Press, S.A. Tuekolsky, W.T. Vetterling, and B.P. Flannery, Numerical Recipes in C: The Art of Scientific Computing, second ed. Cambridge Univ. Press, 1992.
[21] Y. Weiss, Deriving Intrinsic Images from Image Sequences Proc. Ninth IEEE Int'l Conf. Computer Vision, pp. 68-75, July 2001.

Index Terms:
Intrinsic images, reflectance, shadow removal, illumination normarization, video surveillance, robust tracking.
Yasuyuki Matsushita, Ko Nishino, Katsushi Ikeuchi, Masao Sakauchi, "Illumination Normalization with Time-Dependent Intrinsic Images for Video Surveillance," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 26, no. 10, pp. 1336-1347, Oct. 2004, doi:10.1109/TPAMI.2004.86
Usage of this product signifies your acceptance of the Terms of Use.