Abstract: A scene analysis system for the 3-D modeling of objects is presented. It combines surface reconstruction techniques with object recognition for the generation of 3-D models for computer graphic applications. The system permits the insertion of highlevel constraints, like a specific angle between two house walls, in an explicit knowledge base implemented as a semantic net. The applicability of those constraints is proved by asserting and testing hypotheses in an interpretation phase. In the case of rejection a more general constraint or model is selected. The capabilities of the system were shown for the modeling of buildings using depth from stereo and contour information. The system reconstructs the surface of the scene objects using the constraints selected in the prior interpretation.