|
| This Article | ||
| ||
| Share | ||
| Bibliographic References | ||
| Add to: | ||
 Digg  Furl  Spurl  Blink  Simpy  Google  Del.icio.us  Y!MyWeb | ||
| Search | ||
| ||
| ASCII Text | x | ||
| Ko Nishino, Yoichi Sato, Katsushi Ikeuchi, "Eigen-Texture Method: Appearance Compression and Synthesis Based on a 3D Model," IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 23, no. 11, pp. 1257-1265, November, 2001. | |||
| BibTex | x | ||
| @article{ 10.1109/34.969116, author = {Ko Nishino and Yoichi Sato and Katsushi Ikeuchi}, title = {Eigen-Texture Method: Appearance Compression and Synthesis Based on a 3D Model}, journal ={IEEE Transactions on Pattern Analysis and Machine Intelligence}, volume = {23}, number = {11}, issn = {0162-8828}, year = {2001}, pages = {1257-1265}, doi = {http://doi.ieeecomputersociety.org/10.1109/34.969116}, publisher = {IEEE Computer Society}, address = {Los Alamitos, CA, USA}, } | |||
| RefWorks Procite/RefMan/Endnote | x | ||
| TY - JOUR JO - IEEE Transactions on Pattern Analysis and Machine Intelligence TI - Eigen-Texture Method: Appearance Compression and Synthesis Based on a 3D Model IS - 11 SN - 0162-8828 SP1257 EP1265 EPD - 1257-1265 A1 - Ko Nishino, A1 - Yoichi Sato, A1 - Katsushi Ikeuchi, PY - 2001 KW - Image synthesis KW - texture KW - appearance KW - model-based rendering KW - image-based rendering KW - principle component analysis. VL - 23 JA - IEEE Transactions on Pattern Analysis and Machine Intelligence ER - | |||
Abstract—Image-based and model-based methods are two representative rendering methods for generating virtual images of objects from their real images. However, both methods still have several drawbacks when we attempt to apply them to mixed reality where we integrate virtual images with real background images. To overcome these difficulties, we propose a new method, which we refer to as the Eigen-Texture method. The proposed method samples appearances of a real object under various illumination and viewing conditions, and compresses them in the 2D coordinate system defined on the 3D model surface generated from a sequence of range images. The Eigen-Texture method is an example of a view-dependent texturing approach which combines the advantages of image-based and model-based approaches: No reflectance analysis of the object surface is needed, while an accurate 3D geometric model facilitates integration with other scenes. This paper describes the method and reports on its implementation.
[1] E.H Adelson and J.R. Bergen, “The Plenoptic Function and the Elements of EarlyVision,” Computational Models of Visual Processing, M. Landy and J.A. Movshon eds., pp. 3-20, 1991.
[2] P.J. Besl and N.D. McKay, "A Method for Registration of 3D Shapes," IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14, no. 2, pp. 239-256, Feb. 1992.
[3] S.E. Chen and L. Williams, “View Interpolation for Image Synthesis,” Proc. ACM SIGGRAPH '93, pp. 279-288, Aug. 1993.
[4] W.B. Culbertson, T. Malzbender, and G.G. Slabaugh, “Generalized Voxel Coloring,” Vision Algorithms: Theory and Practice, vol. 1883, pp. 67-74, Sept. 1999.
[5] P.E. Debevec, "Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography," Proc. SIGGRAPH 98, pp. 189-198, July 1998.
[6] P.E. Debevec, C.J. Taylor, and J. Malik, “Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach,” Proc. SIGGRAPH '96, pp. 11-20, Aug. 1996.
[7] P.E. Debevec, Y. Yu, and G. Borshukov, “Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping,” Proc. Ninth Eurographics Workshop Rendering, pp. 105-116, June 1998.
[8] S.J. Gortler, R. Grzeszczuk, R. Szeliski, and M.F. Cohen, “The Lumigraph,” Proc. SIGGRAPH '96, pp. 43-54, 1996.
[9] I. Sato, Y. Sato, and K. Ikeuchi, “Acquiring a Radiance Distribution to Superimpose Virtual Objects onto a Real Scene,” IEEE Trans. Visualization and Computer Graphics, vol. 5, no. 1, pp. 1-12, 1999.
[10] K.N. Kutulakos and S.M. Seitz, “What Do N Photographs Tell Us about 3D Shape?” Technical Report 680, Computer Science Dept., Univ. Rochester, NY, Jan. 1998.
[11] T. Leung and J. Malik, “Recognizing Surfaces Using Three-Dimensional Textons,” Proc. Seventh Int'l Conf. Computer Vision, 1999.
[12] M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. SIGGRAPH '96, pp. 31-42, 1996.
[13] H. Murase and S.K. Nayar, “Visual Learning and Recognition of 3-D Objects from Appearance,” Int'l J. Computer Vision, vol. 14, pp. 5-24, 1995.
[14] K. Nishino, Y. Sato, K. Ikeuchi, “Appearance Compression and Synthesis based on 3D Model for Mixed Reality,” Proc. Seventh Int'l Conf. Computer Vision (ICCV '99), vol. 1, pp. 38-45, Sept. 1999.
[15] K. Nishino, Y. Sato, and K. Ikeuchi, Eigen-Texture Method: Appearance Compression Based on 3D Model Proc. IEEE Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 618-624, June 1999.
[16] K. Pulli, M. Cohen, T. Duchamp, H. Hoppe, L. Shapiro, and W. Stuetzle, “View-Based Rendering: Visualizing Real Objects from Scanned Range and Color Data,” Proc. Rendering Techniques '97, J. Dorsey and P. Slusallek, eds., pp. 23-34, June 1997.
[17] K. Sato and S. Inokuchi, “Range-Imaging System Utilizing Nematic Liquid Crystal Mask,” Proc. First Int'l Conf. Computer Vision, pp. 657-661, 1987.
[18] Y. Sato and K. Ikeuchi, “Temporal-Color Space Analysis of Reflection,” J. Optical Soc. of America, vol. 11, no. 11, pp. 2990-3002, 1994.
[19] Y. Sato, M.D. Wheeler, and K. Ikeuchi, “Object Shape and Reflectance Modeling from Observation,” Proc. SIGGRAPH '97, pp. 379-387, 1997.
[20] S.M. Seitz and C.R. Dyer, “View Morphing,” ACM SIGGRAPH, pp. 21-30, 1996.
[21] S.M. Seitz and C.R. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” Computer Vision and Pattern Recognition, pp. 1067-1073, 1997.
[22] A. Shashua, “Geometry and Photometry in 3D Visual Recognition,” PhD dissertation, Dept. of Brain and Cognitive Sciences, Massachusetts Inst. of Technology, Cambridge, Nov. 1992.
[23] P. Suen and G. Healey, “The Analysis and Recognition of Real-World Textures in Three Dimensions,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 22, no. 5, pp. 491-503, May 2000.
[24] R.Y. Tsai, “A Versatile Camera Calibration Technique for High-Accuracy 3D Machine Vision Metrology Using Off-the-Shelf TV Cameras and Lenses,” IEEE J. Robotics and Automation, vol. 3, no. 4, pp. 323-344, Aug. 1987.
[25] G. Turk and M. Levoy, “Zippered Polygon Meshes from Range Images,” Proc. SIGGRAPH '94, pp. 311-318, 1994.
[26] M. Wheeler, Y. Sato, and K. Ikeuchi, "Consensus Surfaces for Modeling 3D Objects from Multiple Range Images," Proc. IEEE Int'l Conf. Computer Vision, pp. 917-924,Bombay, India, Jan. 1998.
[27] R.J. Woodham, “Photometric Stereo: A Reflectance Map Technique for Determining Surface Orientation from a Single View,” Image Understanding Systems and Industrial Applications, vol. 155, pp. 136-143. Proc. SPIE 22nd Ann. Technical Symp., Aug. 1978.
[28] Z. Zhang, “Modeling Geometric Structure and Illumination Variation of a Scene from Real Images,” Proc. Sixth Int'l Conf. Computer Vision, pp. 1041-1046, July 1998.