In this paper, we describe our recent efforts in grouping sensory data into meaningful entities. Our grouping philosophy is based on perceptual organization principles using gestalt hypotheses where we impose structural regularity on sensory primitives stemming from a common underlying cause. We present results using field data from UGVs and outline the utility of our research in object recognition and tracking for autonomous vehicle navigation. In addition, we show how the grouping efforts can be useful for constructing symbolic topological maps when data from different sensing modalities are fused in a bottom-up and topdown fashion.