Incrementally learning from a large number of unlabeled examples continues to be an active area of research in pattern recognition. Active Learning has made great strides in recent years to address this problem, taking advantage of SVMs to develop robust learning systems. Recently, diversity sampling for SVM active learning has garnered much attention. In this work we propose a fundamentally motivated view of diversity for SVM active learning based on an information-theoretic diversity measure. Comparative testing on a database from the small-sample learning problem of image retrieval is done and thoughts for future work are presented.