In this paper, we propose a new method to simultaneously achieve segmentation and dense matching in a pair of stereo images. In constrast to conventional methods that are based on similarity or correlation techniques, this method is based on geometry and uses correlations only on a limited number of key points. Stemming from the observation that our environment is abundant in planes, this method focuses on segmentation and matching of planes in an observed scene. Neither prior knowledge about the scene nor camera calibration are needed. Using two uncalibrated images as inputs, the method starts with a rough identification of a potential plane, defined by three points only. Based on these three points, a plane homography is then calculated and, used for validation. Starting from a seed region defined by the original three points, the method grows the current region by successive move/confirmation steps until occlusions and/or surface discontinuity occur. In this case, the homography-based mapping of points between the two images will fail. This failure is detected by the correlation, used in the confirmation process. In particular, this method grows a region even across different colors as long as the region is planar. Experiments on real images validated our method and showed its capability and performance.