In this paper we present a probabilistic and continuous framework for supervised image category modelling and matching as well as unsupervised clustering of image space into image categories. A generalized GMM-KL framework is described in which each image or image-set (category) is represented as a Gaussian mixture distribution and images (categories) are compared and matched via a probabilistic measure of similarity between distributions. Image-to-category matching is investigated and unsupervised clustering of a random image set into visually coherent image categories is demonstrated.