This paper describes an approach to recovering surface models of complex scenes from the quasi-sparse data returned by a feature based stereo system. The method can be used to merge stereo results obtained from different viewpoints into a single coherent surface mesh. The technique proceeds by exploiting the freespace theorem which provides a principled mechanism for reasoning about the structure of the scene based on quasi-sparse correspondences in multiple image. Effective methods for overcoming the difficulties posed by missing features and outliers are discussed. Results obtained by applying this approach to actual images are presented.