This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
Flow-Based Local Optimization for Image-to-Geometry Projection
March 2012 (vol. 18 no. 3)
pp. 463-474
ASCII Text
x 
 
Matteo Dellepiane, Ricardo Marroquim, Marco Callieri, Paolo Cignoni, Roberto Scopigno, "Flow-Based Local Optimization for Image-to-Geometry Projection," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 3, pp. 463-474, March, 2012.
 
 
BibTex
x
 
@article{ 10.1109/TVCG.2011.75,
author = {Matteo Dellepiane and Ricardo Marroquim and Marco Callieri and Paolo Cignoni and Roberto Scopigno},
title = {Flow-Based Local Optimization for Image-to-Geometry Projection},
journal ={IEEE Transactions on Visualization and Computer Graphics},
volume = {18},
number = {3},
issn = {1077-2626},
year = {2012},
pages = {463-474},
doi = {http://doi.ieeecomputersociety.org/10.1109/TVCG.2011.75},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Visualization and Computer Graphics
TI - Flow-Based Local Optimization for Image-to-Geometry Projection
IS - 3
SN - 1077-2626
SP463
EP474
EPD - 463-474
A1 - Matteo Dellepiane,
A1 - Ricardo Marroquim,
A1 - Marco Callieri,
A1 - Paolo Cignoni,
A1 - Roberto Scopigno,
PY - 2012
KW - Computer graphics
KW - image color analysis.
VL - 18
JA - IEEE Transactions on Visualization and Computer Graphics
ER -
 
Matteo Dellepiane, ISTI-CNR, Pisa
Ricardo Marroquim, Universidade Federal do Rio de Janeiro, Rio de Janeiro
Marco Callieri, ISTI-CNR, Pisa
Paolo Cignoni, ISTI-CNR, Pisa
Roberto Scopigno, ISTI-CNR, Pisa
The projection of a photographic data set on a 3D model is a robust and widely applicable way to acquire appearance information of an object. The first step of this procedure is the alignment of the images on the 3D model. While any reconstruction pipeline aims at avoiding misregistration by improving camera calibrations and geometry, in practice a perfect alignment cannot always be reached. Depending on the way multiple camera images are fused on the object surface, remaining misregistrations show up either as ghosting or as discontinuities at transitions from one camera view to another. In this paper we propose a method, based on the computation of Optical Flow between overlapping images, to correct the local misalignment by determining the necessary displacement. The goal is to correct the symptoms of misregistration, instead of searching for a globally consistent mapping, which might not exist. The method scales up well with the size of the data set (both photographic and geometric) and is quite independent of the characteristics of the 3D model (topology cleanliness, parametrization, density). The method is robust and can handle real world cases that have different characteristics: low level geometric details and images that lack enough features for global optimization or manual methods. It can be applied to different mapping strategies, such as texture or per-vertex attribute encoding.

[1] B. Brown and S. Rusinkiewicz, “Global Non-Rigid Alignment of 3-D Scans,” ACM Trans. Graphics , vol. 26, no. 3, p. 21, Aug. 2007.
[2] M.J. Black and P. Anandan, “A Framework for the Robust Estimation of Optical Flow,” Proc. Fourth Int'l Conf. Computer Vision, pp. 231-236, 1993.
[3] C. Theobalt, N. Ahmed, H. Lensch, M. Magnor, and H.-P. Seidel, “Seeing People in Different Light-Joint Shape, Motion, and Reflectance Capture,” IEEE Trans. Visualization and Computer Graphics, vol. 13, no. 4, pp. 663-674, July/Aug. 2007.
[4] D. Bradley, W. Heidrich, T. Popa, and A. Sheffer, “High Resolution Passive Facial Performance Capture,” ACM Trans. Graphics, vol. 29, no. 4, pp. 1-10, 2010.
[5] K. Pulli, S. Piiroinen, T. Duchamp, and W. Stuetzle, “Projective Surface Matching of Colored 3D Scans,” Proc. Fifth Int'l Conf. 3-D Digital Imaging and Modeling (3DIM), pp. 531-538, 2005.
[6] K. Ikeuchi, T. Oishi, J. Takamatsu, R. Sagawa, A. Nakazawa, R. Kurazume, K. Nishino, M. Kamakura, and Y. Okamoto, “The Great Buddha Project: Digitally Archiving, Restoring, and Analyzing Cultural Heritage Objects,” Int'l J. Computer Vision, vol. 75, no. 1, pp. 189-208, 2007.
[7] L. Brunie, S. Lavallée, and R. Szeliski, “Using Force Fields Derived from 3D Distance Maps for Inferring the Attitude of a 3D Rigid Object,” ECCV '92: Proc. Second European Conf. Computer Vision, pp. 670-675, 1992.
[8] H. Lensch, W. Heidrich, and H. Seidel, “Automated Texture Registration and Stitching for Real World Models,” Proc. Eighth Pacific Conf. Computer Graphics and Applications, pp. 317-327, 2000.
[9] L. Liu, I. Stamos, G. Yu, G. Wolberg, and S. Zokai, “Multiview Geometry for Texture Mapping 2D Images Onto 3D Range Data,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 2293-2300, 2006.
[10] T. Franken, M. Dellepiane, F. Ganovelli, P. Cignoni, C. Montani, and R. Scopigno, “Minimizing User Intervention in Registering 2D Images to 3D Models,” The Visual Computer, vol. 21, nos. 8-10, pp. 619-628, http://vcg.isti.cnr.it/Publications/2005 CDFGMS05, Sept. 2005.
[11] J. Dorsey, H. Rushmeier, and F. Sillion, Digital Modeling of Material Appearance, Morgan Kauf./Elsevier, http://artis.inrialpes.fr/ Publications/ 2007DRS07, 2007.
[12] P.A. Lensch, J. Kautz, M. Goesele, W. Heidrich, and H.-P. Seidel, “Image-Based Reconstruction of Spatially Varying Materials,” Proc. 12th Eurographics Workshop Rendering Techniques, pp. 104-115, June 2001.
[13] S.R. Marschner, S.H. Westin, E.P.F. Lafortune, K.E. Torrance, and D.P. Greenberg, “Image-Based Brdf Measurement Including Human Skin,” Proc. 10th Eurographics Workshop Rendering, pp. 139-152, June 1999.
[14] P. Debevec, T. Hawkins, C. Tchou, H.-P. Duiker, W. Sarokin, and M. Sagar, “Acquiring the Reflectance Field of a Human Face,” SIGGRAPH '00: Proc. 27th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 145-156, 2000.
[15] Y. Sato, M.D. Wheeler, and K. Ikeuchi, “Object Shape and Reflectance Modeling from Observation,” Proc. ACM SIGGRAPH '97, pp. 379-387, 1997.
[16] Y. Yu, P. Debevec, J. Malik, and T. Hawkins, “Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs,” Proc. ACM SIGGRAPH '99, pp. 215-224, 1999.
[17] F. Bernardini, I. Martin, and H. Rushmeier, “High-Quality Texture Reconstruction from Multiple Scans,” IEEE Trans. Visualization and Computer Graphics, vol. 7, no. 4, pp. 318-332, Oct.-Dec. 2001.
[18] M. Callieri, P. Cignoni, and R. Scopigno, “Reconstructing Textured Meshes from Multiple Range Rgb Maps,” Proc. Seventh Int'l Fall Workshop Vision, Modeling, and Visualization, pp. 419-426, Nov. 2002.
[19] N. Bannai, A. Agathos, and R. Fisher, “Fusing Multiple Color Images for Texturing Models,” Proc. Second Int'l Symp. 3D Data Processing, Visualization and Transmission (3DPVT '04), pp. 558-565, 2004.
[20] M. Chuang, L. Luo, B.J. Brown, S. Rusinkiewicz, and M. Kazh-dan, “Estimating the Laplace-Beltrami Operator by Restricting 3D Functions,” Proc. Symp. Geometry Processing, pp. 1475-1484, July 2009.
[21] K. Pulli, H. Abi-Rached, T. Duchamp, L. Shapiro, and W. Stuetzle, “Acquisition and Visualization of Colored 3D Objects,” Proc. 14th Int'l Conf. Pattern Recognition (ICPR '98), pp. 11-15, 1998.
[22] A. Baumberg, “Blending Images for Texturing 3D Models,” Proc. British Machine Vision Conf. (BMVC '02), 2002.
[23] V. Rankov, R. Locke, R. Edens, P. Barber, and B. Vojnovic, “An Algorithm for Image Stitching and Blending,” Proc. SPIE, vol. 5701, pp. 190-199, 2005.
[24] M. Callieri, P. Cignoni, M. Corsini, and R. Scopigno, “Masked Photo Blending: Mapping Dense Photographic Data Set on High-Resolution 3D Models,” Computer and Graphics, vol. 32, no. 4, pp. 464-473, http://dx.doi.org/10.1016j.cag.2008.05.004 , http://vcg. isti.cnr.it/Publications/2008 CCCS08, Aug. 2008.
[25] P. Debevec, “Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-Based Graphics with Global Illumination and High Dynamic Range Photography,” SIGGRAPH '98: Proc. 25th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 189-198, 1998.
[26] J. Unger, A. Wenger, T. Hawkins, A. Gardner, and P. Debevec, “Capturing and Rendering with Incident Light Fields,” Proc. 14th Eurographics Workshop Rendering (EGRW '03), pp. 141-149, 2003.
[27] M. Dellepiane, M. Callieri, M. Corsini, P. Cignoni, and R. Scopigno, “Improved Color Acquisition and Mapping on 3D Models via Flash-Based Photography,” ACM J. Computers and Cultural Heritage, vol. 2, no. 4, pp. 1-20, http://vcg.isti.cnr.it/Publications/2010 DCCCS10, Feb. 2010.
[28] T. Takai, A. Hilton, and T. Matsuyama, “Harmonised Texture Mapping,” Proc. 3D Data Processing Visualization and Transmission (3DPVT '10), May 2010.
[29] E. Aganj, P. Monasse, and R. Keriven, “Multi-View Texturing of Imprecise Mesh,” Proc. Asian Conf. Computer Vision, Sept. 2009.
[30] R. Gal, Y. Wexler, E. Ofek, H. Hoppe, and D. Cohen-Or, “Seamless Montage for Texturing Models,” Computer Graphics Forum, vol. 29, no. 2, pp. 479-486, 2010.
[31] M. Eisemann, B.D. Decker, M. Magnor, P. Bekaert, E. de Aguiar, N. Ahmed, C. Theobalt, and A. Sellent, “Floating Textures,” Computer Graphics Forum, vol. 27, no. 2, pp. 409-418, 2008.
[32] P.E. Debevec, C.J. Taylor, and J. Malik, “Modeling and Rendering Architecture from Photographs: A Hybrid Geometry-and Image-Based Approach,” SIGGRAPH '96: Proc. 23rd Ann. Conf. Computer Graphics and Interactive Techniques, pp. 11-20, 1996.
[33] T. Brox, A. Bruhn, N. Papenberg, and J. Weickert, “High Accuracy Optical Flow Estimation Based on a Theory for Warping,” Proc. Eighth European Conf. Computer Vision, vol. 4, pp. 25-36, May 2004.
[34] F. Steinbruecker, T. Pock, and D. Cremers, “Large Displacement Optical Flow Computation without Warping,” Proc. IEEE Int'l Conf. Computer Vision (ICCV), 2009.
[35] “Adam Technology Website,” More Info on: http:/www. adamtech.com.au/, 2011.

Index Terms:
Computer graphics, image color analysis.
Citation:
Matteo Dellepiane, Ricardo Marroquim, Marco Callieri, Paolo Cignoni, Roberto Scopigno, "Flow-Based Local Optimization for Image-to-Geometry Projection," IEEE Transactions on Visualization and Computer Graphics, vol. 18, no. 3, pp. 463-474, March 2012, doi:10.1109/TVCG.2011.75
Usage of this product signifies your acceptance of the Terms of Use.