Abstract: This paper presents a knowledge-based approach for automatic 3D building reconstruction. By combining the aerial image analysis with information from GIS maps and domain specific knowledge the complexity of the building reconstruction process can be greatly reduced. The building reconstruction process is described as a tree search in the space of possible building hypotheses. To guide the search of the tree an evaluation function based on mutual information is defined. This evaluation function allows comparison of different building hypotheses obtained by applying a fitting algorithm. The performance of the 3D reconstruction is improved by incorporating geometric constraints.