This Article 
 Bibliographic References 
 Add to: 
Photorealistic Large-Scale Urban City Model Reconstruction
July/August 2009 (vol. 15 no. 4)
pp. 654-669
Charalambos Poullis, University of Southern California, Los Angeles
Suya You, University of Southern California, Los Angeles
The rapid and efficient creation of virtual environments has become a crucial part of virtual reality applications. In particular, civil and defense applications often require and employ detailed models of operations areas for training, simulations of different scenarios, planning for natural or man-made events, monitoring, surveillance, games, and films. A realistic representation of the large-scale environments is therefore imperative for the success of such applications since it increases the immersive experience of its users and helps reduce the difference between physical and virtual reality. However, the task of creating such large-scale virtual environments still remains a time-consuming and manual work. In this work, we propose a novel method for the rapid reconstruction of photorealistic large-scale virtual environments. First, a novel, extendible, parameterized geometric primitive is presented for the automatic building identification and reconstruction of building structures. In addition, buildings with complex roofs containing complex linear and nonlinear surfaces are reconstructed interactively using a linear polygonal and a nonlinear primitive, respectively. Second, we present a rendering pipeline for the composition of photorealistic textures, which unlike existing techniques, can recover missing or occluded texture information by integrating multiple information captured from different optical sensors (ground, aerial, and satellite).

[1] M. Bartels, H. Wei, and D.C. Mason, “DTM Generation from LIDAR Data Using Skewness Balancing,” Proc. Int'l Conf. Pattern Recognition (ICPR '06), pp. 566-569. 2006.
[2] Y. Boykov and M.-P. Jolly, “Interactive Graph Cuts for Optimal Boundary and Region Segmentation of Objects in N-D Images,” Proc. Int'l Conf. Computer Vision (ICCV '01), pp. 105-112, 2001.
[3] Y. Boykov, O. Veksler, and R. Zabih, “Fast Approximate Energy Minimization via Graph Cuts,” Proc. Int'l Conf. Computer Vision (ICCV '99), pp. 377-384, 1999.
[4] P. Debevec, C. Tchou, A. Gardner, T. Hawkins, C. Poullis, J. Stumpfel, A. Jones, N. Yun, P. Einarsson, T. Lundgren, M. Fajardo, and P. Martinez, “Estimating Surface Reflectance Properties of a Complex Scene under Captured Natural Illumination,” Technical Report, Univ. of Southern California, ICT, 2004.
[5] P.E. Debevec, C.J. Taylor, and J. Malik, “Modeling and Rendering Architecture from Photographs: A Hybrid Geometry- and Image-Based Approach,” Proc. ACM Conf. Computer Graphics, pp. 11-20, Aug. 1996.
[6] R.O. Duda and P.E. Hart, Pattern Classification. John Wiley and Sons, 2000.
[7] C. Früh and A. Zakhor, “Constructing 3D City Models by Merging Ground-Based and Airborne Views,” Computer Vision and Pattern Recognition, pp. 562-569, 2003.
[8] J. Hu, S. You, and U. Neumann, “Approaches to Large-Scale Urban Modeling,” IEEE Computer Graphics and Applications, vol. 23, no. 6, pp. 62-69, 2003.
[9] S.C. Lee, S.K. Jung, and R. Nevatia, “Automatic Pose Estimation of Complex 3D Building Models,” Proc. IEEE Workshop Application of Computer Vision (WACV '02), pp. 148-152, 2002.
[10] M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. ACM SIGGRAPH '96, pp.31-42, 1996.
[11] A.R. Martinez and G. Drettakis, “View-Dependent Layered Projective Texture Maps,” Proc. Pacific Conf. Computer Graphics and Applications, pp. 492-496, 2003.
[12] P. Merrell, A. Akbarzadeh, L. Wang, P. Mordohai, J.M. Frahm, R.G. Yang, D. Nister, and M. Pollefeys, “Real-Time Visibility-Based Fusion of Depth Maps,” Proc. Int'l Conf. Computer Vision, pp. 1-8, 2007.
[13] R. Nevatia and K.E. Price, “Automatic and Interactive Modeling of Buildings in Urban Environments from Aerial Images,” Proc. Int'l Conf. Image Processing (ICIP '02), vol. 3, pp. 525-528, 2002.
[14] M. Pollefeys, L.J.V. Gool, M. Vergauwen, F. Verbiest, K. Cornelis, J. Tops, and R. Koch, “Visual Modeling with a Hand-Held Camera,” Int'l J. Computer Vision, vol. 59, no. 3, pp. 207-232, 2004.
[15] C. Poullis, A. Gardner, and P. Debevec, “Photogrammetric Modeling and Image-Based Rendering for Rapid Virtual Environment Creation,” Proc. Army Science Conf., pp. I: 1-7, 2004.
[16] C. Poullis, S. You, and U. Neumann, “Rapid Creation of Large-Scale Photorealistic Virtual Environments,” Proc. IEEE Virtual Reality Conf., pp. 153-160, 2008.
[17] D.P. Robertson and R. Cipolla, “Building Architectural Models from Many Views Using Map Constraints,” Lecture Notes in Computer Science, vol. 2351, pp. 155-163, 2002.
[18] G. Schmitt, Architectura et Machina. Vieweg & Sohn, 1993.
[19] G. Turk, “Texture Synthesis on Surfaces,” Proc. ACM SIGGRAPH '01, pp. 347-354, 2001.
[20] S. You, J. Hu, U. Neumann, and P. Fox, “Urban Site Modeling from liDAR,” Proc. Int'l Conf. Computational Science and Applications, 2003.
[21] C. Zach, T. Pock, and H. Bischof, “A Globally Optimal Algorithm for Robust TV-L1 Range Image Integration,” Proc. Int'l Conf. Computer Vision, pp. 1-8, 2007.

Index Terms:
Large-scale modeling, rapid reconstruction, photorealistic model.
Charalambos Poullis, Suya You, "Photorealistic Large-Scale Urban City Model Reconstruction," IEEE Transactions on Visualization and Computer Graphics, vol. 15, no. 4, pp. 654-669, July-Aug. 2009, doi:10.1109/TVCG.2008.189
Usage of this product signifies your acceptance of the Terms of Use.