The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.02 - February (2008 vol.30)
pp: 197-213
ABSTRACT
This paper develops a theory of frequency domain invariants in computer vision. We derive novel identities using spherical harmonics, which are the angular frequency domain analog to common spatial domain invariants such as reflectance ratios. These invariants are derived from the spherical harmonic convolution framework for reflection from a curved surface. Our identities apply in a number of canonical cases, including single and multiple images of objects under the same and different lighting conditions. One important case we consider is two different glossy objects in two different lighting environments. For this case, we derive a novel identity, independent of the specific lighting configurations or BRDFs, that allows us to directly estimate the fourth image if the other three are available. The identity can also be used as an invariant to detecttampering in the images.While this paper is primarily theoretical, it has the potential to lay the mathematical foundations for two important practical applications. First, we can develop more general algorithms for inverse rendering problems, which can directly relight and change material properties by transferring the BRDF or lighting from another object or illumination. Second, we can check the consistency of an image, to detect tampering or image splicing.
INDEX TERMS
Frequency Domain Invariants, Spherical harmonic identities, Convolution, Inverse rendering, Relighting, Tampering, Image Forensics
CITATION
Dhruv Mahajan, Ravi Ramamoorthi, Brian Curless, "A Theory Of Frequency Domain Invariants: Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency", IEEE Transactions on Pattern Analysis & Machine Intelligence, vol.30, no. 2, pp. 197-213, February 2008, doi:10.1109/TPAMI.2007.1162
REFERENCES
[1] R. Basri and D. Jacobs, “Photometric Stereo with General, Unknown Lighting,” Proc. Conf. Computer Vision and Pattern Recognition, pp. II-374-II-381, 2001.
[2] R. Basri and D. Jacobs, “Lambertian Reflectance and Linear Subspaces,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 2, pp. 218-233, Feb. 2003.
[3] V. Blanz and T. Vetter, “A Morphable Model for the Synthesis of 3D Faces,” Proc. ACM SIGGRAPH '99, pp. 187-194, 1999.
[4] J. Davis, R. Yang, and L. Wang, “BRDF Invariant Stereo Using Light Transport Constancy,” Proc. Int'l Conf. Computer Vision, pp.436-443, 2005.
[5] R. Gonzalez and R. Woods, Digital Image Processing, second ed. Pearson Education, 2003.
[6] A. Hertzmann, C. Jacobs, N. Oliver, B. Curless, and D. Salesin, “Image Analogies,” Proc. ACM SIGGRAPH '01, pp. 327-340, 2001.
[7] A. Hertzmann and S. Seitz, “Example-Based Photometric Stereo: Shape Reconstruction with General, Varying BRDFs,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 8, pp. 1254-1264, Aug. 2005.
[8] M. Johnson and H. Farid, “Exposing Digital Forgeries by Detecting Inconsistencies in Lighting,” Proc. ACM Multimedia and Security Workshop, pp. 1-10, 2005.
[9] H. Jin, S. Soatto, and A. Yezzi, “Multi-View Stereo Beyond Lambert,” Proc. Conf. Computer Vision and Pattern Recognition, pp.171-178, 2003.
[10] Z. Lin, R. Wang, X. Tang, and H. Shum, “Detecting Doctored Images Using Camera Response Normality and Consistency,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 1087-1092, 2005.
[11] S. Marschner and D. Greenberg, “Inverse Lighting for Photography,” Proc. Color Imaging Conf., pp. 262-265, 1997.
[12] D. Mahajan, R. Ramamoorthi, and B. Curless, “A Theory of Spherical Harmonic Identities for BRDF/Lighting Transfer and Image Consistency,” Proc. European Conf. Computer Vision, vol. IV, pp. 41-55, 2006.
[13] S. Marschner, S. Westin, E. Lafortune, and K. Torrance, “Image-Based BRDF Measurement,” Applied Optics, vol. 39, no. 16, pp.2592-2600, 2000.
[14] S. Mallick, T. Zickler, and P. Belhumeur, “Specularity Removal in Images and Videos: A PDE Approach,” Proc. European Conf. Computer Vision, vol. I, pp. 550-563, 2006.
[15] S. Nayar and R. Bolle, “Reflectance Based Object Recognition,” Int'l J. Computer Vision, vol. 17, no. 3, pp. 219-240, 1996.
[16] T. Ng, S. Chang, and Q. Sun, “Blind Detection of Photomontage Using Higher Order Statistics,” Proc. IEEE Int'l Symp. Circuits and Systems, pp. V-688-V-691, 2004.
[17] S. Narasimhan, V. Ramesh, and S. Nayar, “A Class of Photometric Invariants: Separating Material from Shape and Illumination,” Proc. Int'l Conf. Computer Vision, pp. 1387-1394, 2003.
[18] B. Oh, M. Chen, J. Dorsey, and F. Durand, “Image-Based Modeling and Photo Editing,” Proc. ACM SIGGRAPH '01, pp.433-442, 2001.
[19] R. Ramamoorthi and P. Hanrahan, “A Signal-Processing Framework for Inverse Rendering,” Proc. ACM SIGGRAPH '01, pp. 117-128, 2001.
[20] R. Ramamoorthi and P. Hanrahan, “Frequency Space Environment Map Rendering,” Proc. ACM SIGGRAPH '02, vol. 21, no. 3, pp. 517-526, 2002.
[21] S. Rusinkiewicz, “A New Change of Variables for Efficient BRDF Representation,” Proc. Eurographics Rendering Workshop, 1998.
[22] D. Simakov, D. Frolova, and R. Basri, “Dense Shape Reconstruction of a Moving Object Under Arbitrary, Unknown Lighting,” Proc. Int'l Conf. Computer Vision, pp. 1202-1209, 2003.
[23] P. Sloan, J. Kautz, and J. Snyder, “Precomputed Radiance Transfer for Real-Time Rendering in Dynamic, Low-Frequency Lighting Environments,” ACM Trans. Graphics, vol. 21, no. 3, pp. 527-536, 2002.
[24] I. Sato, Y. Sato, and K. Ikeuchi, “Illumination Distribution from Brightness in Shadows: Adaptive Estimation of Illumination Distribution with Unknown Reflectance Properties in Shadow Regions,” Proc. Int'l Conf. Computer Vision, pp. 875-882, 1999.
[25] N. Wiener, Extrapolation, Interpolation and Smoothing of Stationary Time Series. MIT Press, 1942.
[26] Z. Wen, Z. Liu, and T.S. Huang, “Face Relighting with Radiance Environment Maps,” Proc. Conf. Computer Vision and Pattern Recognition, vol. II, pp. 158-165, 2003.
[27] L. Zhang, G. Phocion, J. Samson, and S. Seitz, “Single View Modeling of Free-Form Scenes,” J. Visualization and Computer Animation, vol. 13, no. 4, pp. 225-235, 2002.
[28] L. Zhang, S. Weng, and D. Samaras, “Face Synthesis and Recognition from a Single Image under Arbitrary Unknown Lighting Using a Spherical Harmonic Basis Morphable Model,” Proc. Conf. Computer Vision and Pattern Recognition, pp. 209-216, 2005.
41 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool