This paper presents a hybrid modeling system that fuses LiDAR data, an aerial image and ground view images for rapid creation of accurate building models. Outlines for complex building shapes are interactively extracted from a high-resolution aerial image, surface information is automatically fit with a primitive based method from LiDAR data, and high-resolution ground view images are integrated into the model to generate fully textured CAD models. Our method benefits from the merit of each dataset, and evaluation results are presented on a university campus-size model.