The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - Jan. (2013 vol.19)
pp: 56-66
A. Colburn , Dept. of Comput. Sci. & Eng., Univ. of Washington, Seattle, WA, USA
A. Agarwala , Adobe Syst., Inc., Seattle, WA, USA
A. Hertzmann , Univ. of Toronto, Toronto, ON, Canada
B. Curless , Dept. of Comput. Sci. & Eng., Univ. of Washington, Seattle, WA, USA
M. F. Cohen , Microsoft Res., Redmond, WA, USA
ABSTRACT
Imagining what a proposed home remodel might look like without actually performing it is challenging. We present an image-based remodeling methodology that allows real-time photorealistic visualization during both the modeling and remodeling process of a home interior. Large-scale edits, like removing a wall or enlarging a window, are performed easily and in real time, with realistic results. Our interface supports the creation of concise, parameterized, and constrained geometry, as well as remodeling directly from within the photographs. Real-time texturing of modified geometry is made possible by precomputing view-dependent textures for all faces that are potentially visible to each original camera viewpoint, blending multiple viewpoints and hole-filling when necessary. The resulting textures are stored and accessed efficiently enabling intuitive real-time realistic visualization, modeling, and editing of the building interior.
INDEX TERMS
solid modelling, computational geometry, data visualisation, image texture, building interior, image-based remodeling methodology, photorealistic visualization, large-scale edits, constrained geometry, photographs, real-time texturing, modified geometry, camera viewpoint, hole-filling, intuitive real-time realistic visualization, Solid modeling, Geometry, Three dimensional displays, Rendering (computer graphics), Real time systems, Visualization, Buildings, modeling packages, solid modelling, computational geometry, data visualisation, image texture, building interior, image-based remodeling methodology, photorealistic visualization, large-scale edits, constrained geometry, photographs, real-time texturing, modified geometry, camera viewpoint, hole-filling, intuitive real-time realistic visualization, Solid modeling, Geometry, Three dimensional displays, Rendering (computer graphics), Real time systems, Visualization, Buildings, visualization systems and software, Image-based rendering
CITATION
A. Colburn, A. Agarwala, A. Hertzmann, B. Curless, M. F. Cohen, "Image-Based Remodeling", IEEE Transactions on Visualization & Computer Graphics, vol.19, no. 1, pp. 56-66, Jan. 2013, doi:10.1109/TVCG.2012.95
REFERENCES
[1] N. Snavely, S.M. Seitz, and R. Szeliski, "Photo Tourism: Exploring Photo Collections in 3D," Proc. ACM SIGGRAPH '06 Papers, http://doi.acm.org/10.11451179352.1141964 , pp. 835-846, 2006.
[2] Y. Furukawa, B. Curless, S.M. Seitz, and R. Szeliski, "Reconstructing Building Interiors from Images," Proc. IEEE 12th Int'l Conf. Computer vision (ICCV), pp. 80-87, Oct. 2009.
[3] M. Pollefeys, L. Van Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, "Visual Modeling with a Hand-Held Camera," Int'l J. Computer Vision, vol. 59, pp. 207-232, http://portal.acm.orgcitation.cfm?id=986694.986705 , Sept. 2004.
[4] N. Cornelis, B. Leibe, K. Cornelis, and L. Gool, "3D Urban Scene Modeling Integrating Recognition and Reconstruction," Int'l J. Computer Vision, vol. 78, pp. 121-141, http://portal.acm.orgcitation.cfm?id=1355822.1355831 , July 2008.
[5] G. Bahmutov, V. Popescu, and M. Mudure, "Efficient Large Scale Acquisition of Building Interiors," Proc. ACM SIGGRAPH '06 Sketches, http://doi.acm.org/10.11451179849.1180008 , 2006.
[6] M. Goesele, N. Snavely, B. Curless, H. Hoppe, and S. Seitz, "Multi-View Stereo for Community Photo Collections," Proc. IEEE 11th Int'l Conf. Computer Vision (ICCV '07), pp. 1-8, oct. 2007.
[7] A.R. Dick, P.H.S. Torr, and R. Cipolla, "Modelling and Interpretation of Architecture from Several Images," Int'l J. Computer Vision, vol. 60, pp. 111-134, http://portal.acm.orgcitation.cfm?id= 993451.996343 , Nov. 2004.
[8] P. Müller, G. Zeng, P. Wonka, and L. Van Gool, "Image-Based Procedural Modeling of Facades," Proc. ACM SIGGRAPH '07 papers, http://doi.acm.org/10.11451275808.1276484 , 2007.
[9] V. Hedau, D. Hoiem, and D. Forsyth, "Recovering the Spatial Layout of Cluttered Rooms," Proc. IEEE 12th Int'l Conf. Computer Vision (ICCV '09), pp. 1849 -1856, Oct. 2009.
[10] D.C. Lee, A. Gupta, M. Hebert, and T. Kanade, "Estimating Spatial Layout of Rooms Using Volumetric Reasoning about Objects and Surfaces," Proc. Adavances in Neural Information Processing Systems (NIPS '10), vol. 24, Nov. 2010.
[11] R. Cipolla and D. Robertson, "3D Models of Architectural Scenes from Uncalibrated Images and Vanishing Points," Proc. 10th Int'l Conf. Image Analysis and Processing (ICIAP '99), pp. 824-829, http://portal.acm.orgcitation.cfm?id=839281.840826 , 1999.
[12] P.E. Debevec, C.J. Taylor, and J. Malik, "Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach," Proc. 23rd Ann. Conf. Computer Graphics and Interactive Techniques, pp. 11-20, http://doi.acm.org/10.1145237170.237191, 1996.
[13] B.M. Oh, M. Chen, J. Dorsey, and F. Durand, "Image-Based Modeling and Photo Editing," Proc. 28th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 433-442, http://doi.acm. org/10.1145383259.383310 , 2001.
[14] A. Criminisi, I. Reid, and A. Zisserman, "Single View Metrology," Int'l J. Computer Vision, vol. 40, pp. 123-148, http://portal.acm. orgcitation.cfm?id=365875.365888 , Nov. 2000.
[15] S. El-Hakim, E. Whiting, and L. Gonzo, "3D Modeling with Reusable and Integrated Building Blocks," Proc. Seventh Conf. Optical 3-D Measurement Techniques, pp. 3-5, http://www.iit-iti. nrc-cnrc.gc.ca/iit-publications-iti/ docsNRC-48228.p df, Jan. 2005.
[16] A. vanden Hengel, A. Dick, T. Thormählen, B. Ward, and P.H.S. Torr, "Videotrace: Rapid Interactive Scene Modelling from Video," Proc. ACM SIGGRAPH '07 Papers, http://doi.acm.org/10.11451275808.1276485 , 2007.
[17] S.N. Sinha, D. Steedly, R. Szeliski, M. Agrawala, and M. Pollefeys, "Interactive 3D Architectural Modeling from Unordered Photo Collections," Proc. ACM SIGGRAPH Asia '08 Papers, pp. 159:1-159:10, http://doi.acm.org/10.11451457515.1409112 , 2008.
[18] L. Nan, A. Sharf, H. Zhang, D. Cohen-Or, and B. Chen, "Smartboxes for Interactive Urban Reconstruction," Proc. ACM SIGGRAPH '10 Papers, pp. 93:1-93:10, http://doi.acm.org/10.11451833349.1778830 , 2010.
[19] S.M. Seitz and K.N. Kutulakos, "Plenoptic Image Editing," Int'l J. Computer Vision, vol. 48, pp. 115-129, http://portal.acm.orgcitation.cfm?id=598431.598504 , July 2002.
[20] D.R. Horn and B. Chen, "Lightshop: Interactive Light Field Manipulation and Rendering," Proc. Symp. Interactive 3D Graphics and Games (I3D '07), pp. 121-128, http://doi.acm.org/10.11451230100.1230121 , 2007.
[21] B. Chen, E. Ofek, H.-Y. Shum, and M. Levoy, "Interactive Deformation of Light Fields," Proc. Symp. Interactive 3D Graphics and Games (I3D '05), pp. 139-146, http://doi.acm.org/10.11451053427.1053450 , 2005.
[22] R. Carroll, A. Agarwala, and M. Agrawala, "Image Warps for Artistic Perspective Manipulation," Proc. ACM SIGGRAPH '10 Papers, pp. 127:1-127:9, http://doi.acm.org/10.11451833349. 1778864 , 2010.
[23] N. Snavely, "Bundler: Structure from Motion for Unordered Image Collections," http:/phototour.cs.washington.edu, http://phototour.cs.washington.edubundler /, 2008.
[24] Y. Furukawa and J. Ponce, "Patch-Based Multi-View Stereo Software," http://grail.cs.washington.edu/softwarepmvs /, http://grail.cs.washington.edu/softwarepmvs /, 2009.
[25] Y. Furukawa, B. Curless, S. Seitz, and R. Szeliski, "Manhattan-World Stereo," Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR '09), pp. 1422-1429, 2009.
[26] C.M. Eastman and S.F. Weiss, "Tree Structures for High Dimensionality Nearest Neighbor Searching," Information Systems, vol. 7, no. 2, pp. 115-122, http://www.sciencedirect.com/science/article/ B6V0G-48TD1M7-6H/234fd7dcac0ef3c2fa82a5eb 688697662 , 1982.
[27] D.S. Johnson, "Approximation Algorithms for Combinatorial Problems," Proc. Fifth Ann. ACM Symp. Theory of Computing (STOC '73), pp. 38-49, http://doi.acm.org/10.1145800125. 804034 , 1973.
[28] H. Buchholz and J. Dollner, "View-Dependent Rendering of Multiresolution Texture-Atlases," Proc. IEEE Visualization Conf., pp. 215-222, 2005.
[29] C. Buehler, M. Bosse, L. McMillan, S. Gortler, and M. Cohen, "Unstructured Lumigraph Rendering," Proc. 28th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 425-432, http://doi.acm.org/10.1145383259.383309, 2001.
[30] P. Pérez, M. Gangnet, and A. Blake, "Poisson Image Editing," Proc. ACM SIGGRAPH '03 Papers, pp. 313-318, http://doi. acm.org/10.11451201775.882269 , 2003.
[31] P. Bhat, C.L. Zitnick, M. Cohen, and B. Curless, "Gradientshop: A Gradient-Domain Optimization Framework for Image and Video Filtering," ACM Trans. Graphics, vol. 29, pp. 1-14, http://doi.acm.org/10.11451731047.1731048 , Apr. 2010.
[32] N. Snavely, R. Garg, S. Seitz, and R. Szeliski, "Finding Paths through the World's Photos," Proc. SIGGRAPH '08, http://portal.acm.orgcitation.cfm?id=1399504.1360614 , Aug. 2008.
[33] Y. Yu, P. Debevec, J. Malik, and T. Hawkins, "Inverse Global Illumination: Recovering Reflectance Models of Real Scenes from Photographs," Proc. 26th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 215-224, http://dx.doi.org/10.1145311535.311559, 1999.
[34] R. Ramamoorthi and P. Hanrahan, "A Signal-Processing Framework for Inverse Rendering," Proc. 28th Ann. Conf. Computer Graphics and Interactive Techniques, pp. 117-128, http://doi.acm. org.offcampus.lib.washington.edu/ 10.1145383259.383271, 2001.
[35] F. Romeiro and T. Zickler, "Blind Reflectometry," Proc. 11th European Conf. Computer Vision (ECCV '10), K. Daniilidis, P. Maragos, and N. Paragios, eds., pp. 45-58, 2010.
[36] J. Unger, S. Gustavson, P. Larsson, and A. Ynnerman, "Free Form Incident Light Fields," Computer Graphics Forum, vol. 27, no. 4, pp. 1293-1301, 2008.
[37] K. Karsch, V. Hedau, D. Forsyth, and D. Hoiem, "Rendering Synthetic Objects into Legacy Photographs," Proc. SIGGRAPH Asia Conf., pp. 157:1-157:12, http://doi.acm.org/10.11452024156. 2024191 , 2011.
39 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool