This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape
August 2006 (vol. 28 no. 8)
pp. 1287-1302
ASCII Text
x 
 
Todd Zickler, Ravi Ramamoorthi, Sebastian Enrique, Peter N. Belhumeur, "Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1287-1302, August, 2006.
 
 
BibTex
x
 
@article{ 10.1109/TPAMI.2006.170,
author = {Todd Zickler and Ravi Ramamoorthi and Sebastian Enrique and Peter N. Belhumeur},
title = {Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape},
journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence},
volume = {28},
number = {8},
issn = {0162-8828},
year = {2006},
pages = {1287-1302},
doi = {http://doi.ieeecomputersociety.org/10.1109/TPAMI.2006.170},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Pattern Analysis and Machine Intelligence
TI - Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape
IS - 8
SN - 0162-8828
SP1287
EP1302
EPD - 1287-1302
A1 - Todd Zickler,
A1 - Ravi Ramamoorthi,
A1 - Sebastian Enrique,
A1 - Peter N. Belhumeur,
PY - 2006
KW - Reflectance
KW - BRDF
KW - image synthesis
KW - image-based rendering
KW - radial basis functions.
VL - 28
JA - IEEE Transactions on Pattern Analysis and Machine Intelligence
ER -
 
Three-dimensional appearance models consisting of spatially varying reflectance functions defined on a known shape can be used in analysis-by-synthesis approaches to a number of visual tasks. The construction of these models requires the measurement of reflectance, and the problem of recovering spatially varying reflectance from images of known shape has drawn considerable interest. To date, existing methods rely on either: 1) low-dimensional (e.g., parametric) reflectance models, or 2) large data sets involving thousands of images (or more) per object. Appearance models based on the former have limited accuracy and generality since they require the selection of a specific reflectance model a priori, and while approaches based on the latter may be suitable for certain applications, they are generally too costly and cumbersome to be used for image analysis. We present an alternative approach that seeks to combine the benefits of existing methods by enabling the estimation of a nonparametric spatially varying reflectance function from a small number of images. We frame the problem as scattered-data interpolation in a mixed spatial and angular domain, and we present a theory demonstrating that the angular accuracy of a recovered reflectance function can be increased in exchange for a decrease in its spatial resolution. We also present a practical solution to this interpolation problem using a new representation of reflectance based on radial basis functions. This representation is evaluated experimentally by testing its ability to predict appearance under novel view and lighting conditions. Our results suggest that since reflectance typically varies slowly from point to point over much of an object's surface, we can often obtain a nonparametric reflectance function from a sparse set of images. In fact, in some cases, we can obtain reasonable results in the limiting case of only a single input image.

[1] F. Nicodemus, J. Richmond, J. Hsia, I. Ginsberg, and T. Limperis, “Geometric Considerations and Nomenclature for Reflectance,” Monograph 160, Nat'l Bureau of Standards (US), 1977.
[2] A. Georghiades, “Incorporating the Torrance and Sparrow Model of Reflectance in Uncalibrated Photometric Stereo,” Proc. Int'l Conf. Computer Vision, pp. 816-823, 2003.
[3] V. Blanz and T. Vetter, “Face Recognition Based on Fitting a 3D Morphable Model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 25, no. 9, Sept. 2003.
[4] K.I.K. Hara and K. Nishino, “Light Source Position and Reflectance Estimation from a Single View without the Distant Illumination Assumption,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 27, no. 4, pp. 493-505, Apr. 2005.
[5] D.K. McAllister, A. Lastra, and W. Heidrich, “Efficient Rendering of Spatial Bi-Directional Reflectance Distribution Functions,” Graphics Hardware 2002 (Proc. Eurographics/ACM SIGGRAPH Hardware Workshop), pp. 79-88, 2002.
[6] Y. Sato, M.D. Wheeler, and K. Ikeuchi, “Object Shape and Reflectance Modeling from Observation,” Proc. ACM SIGGRAPH, pp. 379-387, 1997.
[7] Y. Yu, P. Debevec, J. Malik, and T. Hawkins, “Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs,” Proc. ACM SIGGRAPH Conf., pp. 215-224, 1999.
[8] R. Cook and K. Torrance, “A Reflectance Model for Computer Graphics,” Computer Graphics (Proc. ACM SIGGRAPH), vol. 15, no. 3, pp. 307-316, 1981.
[9] M. Oren and S. Nayar, “Generalization of the Lambertian Model and Implications for Machine Vision,” Int'l J. Computer Vision, vol. 14, pp. 227-251, 1996.
[10] R. Lu, J. Koenderink, and A. Kappers, “Optical Properties (Bidirectional Reflection Distribution Functions) of Velvet,” Applied Optics, vol. 37, no. 25, pp. 5974-5984, 1998.
[11] D. Wood, D. Azuma, K. Aldinger, B. Curless, T. Duchamp, D. Salesin, and W. Stuetzle, “Surface Light Fields for 3D Photography,” Proc. ACM SIGGRAPH, pp. 287-296, 2000.
[12] P. Debevec, T. Hawkins, C. Tchou, H. Duiker, W. Sarokin, and M. Sagar, “Acquiring the Reflectance Field of a Human Face,” Proc. ACM SIGGRAPH, pp. 145-156, 2000.
[13] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “Image-Based 3D Photography Using Opacity Hulls,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 21, no. 3, pp. 427-437, 2002.
[14] S. Marschner, S. Westin, E. Lafortune, K. Torrance, and D. Greenberg, “Image-Based BRDF Measurement Including Human Skin,” Rendering Techniques '99 (Proc. Eurographics Rendering Workshop), pp. 139-152, 1999.
[15] W. Matusik, H. Pfister, M. Brand, and L. McMillan, “A Data-Driven Reflectance Model,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 22, no. 3, pp. 759-769, 2003.
[16] H. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, “Image-Based Reconstruction of Spatially Varying Materials,” Rendering Techniques 2001 (Proc. Eurographics Rendering Workshop), 2001.
[17] K. Karner, H. Mayer, and M. Gervautz, “An Image Based Measurement System for Anisotropic Reflectance,” Computer Graphics Forum, vol. 15, no. 3, pp. 119-128, 1996.
[18] A. Hertzmann and S. Seitz, “Shape and Material By Example: A Photometric Stereo Approach,” Proc. IEEE Conf. Computer Vision and Pattern Recognition, 2003.
[19] M. McCool, J. Ang, and A. Ahmad, “Homomorphic Factorization of BRDFs for High-Performance Rendering,” Proc. ACM SIGGRAPH, pp. 171-178, 2001.
[20] R. Jaroszkiewicz and M. McCool, “Fast Extraction of BRDFs and Material Maps from Images,” Proc. Graphics Interface Conf., pp. 1-10, 2003.
[21] B. Cabral, M. Olano, and P. Nemec, “Reflection Space Image Based Rendering,” Proc. ACM SIGGRAPH, p. 165, 1999.
[22] R. Ramamoorthi and P. Hanrahan, “Frequency Space Environment Map Rendering,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 21, no. 3, pp. 517-526. 2002.
[23] T. Zickler, S. Enrique, R. Ramamoorthi, and P. Belhumeur, “Reflectance Sharing: Image-Based Rendering from a Sparse Set of Images,” Proc. Eurographics Symp. Rendering, 2005.
[24] S. Gortler, R. Grzeszczuk, R. Szeliski, and M. Cohen, “The Lumigraph,” Proc. ACM SIGGRAPH, pp. 43-54, 1996.
[25] M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. ACM SIGGRAPH, pp. 31-42, 1996.
[26] C. Christoudias, L. Morency, and T. Darrell, “Light Field Appearance Manifolds,” Proc. European Conf. Computer Vision, vol. 4, pp. 481-493, 2004.
[27] A. Georghiades, P. Belhumeur, and D. Kriegman, “From Few to Many: Illumination Cone Models for Face Recognition under Variable Lighting and Pose,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 6, pp. 643-660, June 2001.
[28] S. Rusinkiewicz, “A New Change of Variables for Efficient BRDF Representation,” Rendering Techniques '98 (Proc. Eurographics Rendering Workshop), pp. 11-22, 1998.
[29] S. Nayar, K. Ikeuchi, and T. Kanade, “Surface Reflection: Physical and Geometric Perspectives,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 13, no. 7, pp. 611-634, July 1991.
[30] J. Chai, S. Chan, H. Shum, and X. Tong, “Plenoptic Sampling,” Proc. ACM SIGGRAPH, pp. 307-318, 2000.
[31] F. Durand, N. Holzschuch, C. Soler, E. Chan, and F. Sillion, “A Frequency Analysis of Light Transport,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 24, no. 3, pp. 1115-1126, 2005.
[32] R. Bracewell, K. Chang, A. Jha, and Y. Wang, “Affine Theorem for Two-Dimensional Fourier Transform,” Electronics Letters, vol. 29, no. 3,p. 304, 1993.
[33] S. Marschner, “Inverse Rendering for Computer Graphics,” PhD dissertation, Cornell Univ., 1998.
[34] D. Terzopoulos, “Multilevel Computational Processes for Visual Surface Reconstruction,” Computer Vision, Graphics and Image Processing, vol. 24, pp. 52-96, 1983.
[35] M. Powell, “The Theory of Radial Basis Function Approximation in 1990,” Advances in Numerical Analysis, vol. II, W. Light, ed., pp. 105-210, 1992.
[36] M. Buhmann, Radial Basis Functions. Cambridge Univ. Press, 2003.
[37] J. Duchon, “Splines Minimizing Rotation-Invariant Semi-Norms in Sobelev Spaces,” Constructive Theory of Functions of Several Variables, W. Schempp and K. Zeller, eds., pp. 85-100, 1997.
[38] C. Micchelli, “Interpolation of Scattered Data: Distance Matrices and Conditionally Positive Definite Functions,” Constructive Approximation, vol. 1, pp. 11-22, 1986.
[39] G. Wahba, “Spline Models for Observational Data,” Proc. CBMS-NSF Regional Conf. Series in Applied Math., no. 59, 1990.
[40] H. Dinh, G. Turk, and G. Slabaugh, “Reconstructing Surfaces Using Anisotropic Basis Functions,” Proc. Int'l Conf. Computer Vision, pp. 606-613, 2001.
[41] G. Beatson, R. Goodsell, and M. Powell, “On Multigrid Techniques for Thin Plate Spline Interpolation in Two Dimensions,” Lectures in Applied Math., vol. 32, pp. 77-97, 1995.
[42] R. Beatson and G. Newsam, “Fast Evaluation of Radial Basis Functions: I,” Computer Math. Applications, vol. 24, pp. 7-19, 1992.
[43] J. Carr, R. Beatson, J. Cherrie, T. Mitchell, W. Fright, B. McCallum, and T. Evans, “Reconstruction and Representation of 3D Objects with Radial Basis Functions,” Proc. ACM SIGGRAPH, pp. 67-76, 2001.
[44] E. Lafortune, S. Foo, K. Torrance, and D. Greenberg, “Nonlinear Approximation of Reflectance Functions,” Proc. ACM SIGGRAPH, pp. 117-126, 1997.
[45] G.J. Ward, “Measuring and Modeling Anisotropic Reflection,” Computer Graphics (Proc. ACM SIGGRAPH), vol. 26, no. 2, pp. 265-272, 1992.
[46] A. Lee, W. Sweldens, P. Schroder, L. Cowsar, and D. Dobkin, “MAPS: Multiresolution Adaptive Parameterization of Surfaces,” Proc. ACM SIGGRAPH, pp. 95-104, 1998.
[47] X. Gu, S. Gortler, and H. Hoppe, “Geometry Images,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 21, no. 3, pp. 355-361, 2002.
[48] S. Shafer, “Using Color to Separate Reflection Components,” COLOR Research Applications, vol. 10, no. 4, pp. 210-218, 1985.
[49] S. Nayar, X. Fang, and T. Boult, “Separation of Reflection Components Using Color and Polarization,” Int'l J. Computer Vision, vol. 21, no. 3, pp. 163-186, 1997.
[50] S. Mersch, “Polarized Lighting for Machine Vision Applications,” Proc. RI/SME Third Ann. Applied Machine Vision Conf., pp. 40-54, Feb. 1984.
[51] J. Meseth, G. Müller, and R. Klein, “Reflectance Field Based Real-Time, High-Quality Rendering of Bidirectional Texture Functions,” Computers and Graphics, vol. 28, no. 1, pp. 105-112, 2004.
[52] S. Nayar, P. Belhumeur, and T. Boult, “Lighting Sensitive Display,” ACM Trans. Graphics, vol. 23, no. 4, p. 963, 2004.
[53] W. Chen, J. Bouguet, M. Chu, and R. Grzeszczuk, “Light Field Mapping: Efficient Representation and Hardware Rendering of Surface Light Fields,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 21, no. 3, pp. 447-456, 2002.
[54] P. Sloan, J. Hall, J. Hart, and J. Snyder, “Clustered Principal Components for Precomputed Radiance Transfer,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 22, no. 3, pp. 382-391, 2003.
[55] D. Nehab, S. Rusinkiewicz, J. Davis, and R. Ramamoorthi, “Efficiently Combining Positions and Normals for Precise 3D Geometry,” ACM Trans. Graphics (Proc. ACM SIGGRAPH), vol. 24, no. 3, 2005.
[56] T. Zickler, P. Belhumeur, and D. Kriegman, “Helmholtz Stereopsis: Exploiting Reciprocity for Surface Reconstruction,” Int'l J. Computer Vision, vol. 49, no. 2/3, pp. 215-227, Sept. 2002.
[57] S. Boivin and A. Gagalowicz, “Image-Based Rendering of Diffuse, Specular and Glossy Surfaces from a Single Image,” Proc. ACM SIGGRAPH, pp. 107-116, 2001.

Index Terms:
Reflectance, BRDF, image synthesis, image-based rendering, radial basis functions.
Citation:
Todd Zickler, Ravi Ramamoorthi, Sebastian Enrique, Peter N. Belhumeur, "Reflectance Sharing: Predicting Appearance from a Sparse Set of Images of a Known Shape," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 8, pp. 1287-1302, Aug. 2006, doi:10.1109/TPAMI.2006.170
Usage of this product signifies your acceptance of the Terms of Use.