We investigate in this paper an original methodology for detecting roof facets through the fusion of aerial images and lidar data (3D point cloud). Based on a hierarchical segmentation of the image, we define a cost function that manages the merging order of regions. It depends on both radiometric similarities of two neighbouring regions as well as on extracted information from lidar data. Considering that lidar data have been filtered into points belonging either to ground or non-ground classes, we define semantic and geometric rules in the binary merging process. Building roof facets are finally detected by selecting a level of generallity for representing roof building components. Some remarks are given concerning the reliability of the integration of lidar and image data. Reconstructed roof facets are finally shown onto complex buildings.