In this paper, a semantic model — IISM (image internal semantic model) is introduced. Unlike other semantic extracting methods, IISM extracts the semantic information not by image segmentation and image understanding, but by analyzing relevance feedback image retrieval results. For relevance feedback image retrieval system, the images relevant to query are pointed as positive example, otherwise the images irrelevant to query are pointed as negative examples. It is assumed that these positive examples are related in semantic content. IISM computes comprehensive pair-wise mutual information for all images through analyzing the results of relevance feedback image retrieval. An association with a high mutual information means that one image is semantically associated with another. Semantic retrieval and clustering is carried out based on these association relationships.