The paper presents an agglomerative clustering technique for image segmentation. To initiate agglomeration, a set of homogeneous segments is found in the image using level set analysis (LSA). A relational matrix is then defined establishing relations between neighbouring segments present in the image. These relations are derived based on the intensity and boundary features of the segments. The agglomeration is performed on this relationship based on asymmetric agglomerative clustering criteria. The performance is demonstrated through results on a number of natural images and through cluster validity criteria.