The exponential growth of Web images has created a compelling need for innovative methods to retrieve and manage them. Automatic image annotation is an effective way for resolving this problem. In this paper, we propose a novel system that automatically annotates images by semantic corpus which is constructed by mining Web image data. It includes three parts: 1) Constructing the semantic annotation corpus by mining 413,006 Web images and their surrounding text collected from several image search engine; 2) Searching for visually similar images in this semantic annotation corpus and extracting candidate annotation terms; 3) Ranking candidate annotation terms to filter out noisy ones. Our system is evaluated using two benchmark image datasets. Experimental results indicate that this approach is effective.