Semantic annotation of images is an important step to support semantic information extraction and retrieval. However, since the behaviors of annotators could influence the performance of a retrieval system, the modeling of annotators must be considered during the annotation especially in multiple-annotator system. This paper devised an annotator model from three different aspects, expertise, trustworthiness, and diligence. And also conducted experiments to evaluate the annotator model; the experiments showed the proposed annotator model improved the annotation and retrieval significantly.