loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
First Latin American Web Congress (LA-WEB'03)
SharpSpider: Spidering the Web through Web Services
Santiago, Chile
November 10-November 12
ISBN: 0-7695-2058-8
Ken Moody, University of Cambridge
Marco Palomino, University of Cambridge
Web search engines have become an indispensable utility for Internet users. In the near future, however, Web search engines will not only be expected to provide quality search results, but also to enable applications to search and exploit their index repositories directly. We present here SharpSpider, a distributed, C# spider designed to address the issues of scalability, decentralisation and continuity of a Web crawl. Fundamental to the design of SharpSpider is the publication of an API for use by other services on the network. Such an API grants access to a constantly refreshed index built after successive crawls of the Web.
Citation:
Ken Moody, Marco Palomino, "SharpSpider: Spidering the Web through Web Services," la-web, pp.219, First Latin American Web Congress (LA-WEB'03), 2003
Usage of this product signifies your acceptance of the Terms of Use.