First Latin American Web Congress (LA-WEB'03) SharpSpider: Spidering the Web through Web Services Santiago, Chile November 10-November 12 ISBN: 0-7695-2058-8
Web search engines have become an indispensable utility for Internet users. In the near future, however, Web search engines will not only be expected to provide quality search results, but also to enable applications to search and exploit their index repositories directly. We present here SharpSpider, a distributed, C# spider designed to address the issues of scalability, decentralisation and continuity of a Web crawl. Fundamental to the design of SharpSpider is the publication of an API for use by other services on the network. Such an API grants access to a constantly refreshed index built after successive crawls of the Web.
Citation:
Ken Moody, Marco Palomino, "SharpSpider: Spidering the Web through Web Services," la-web, pp.219, First Latin American Web Congress (LA-WEB'03), 2003 Usage of this product signifies your acceptance of the Terms of Use. | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||