This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
International Conference on Internet Surveillance and Protection (ICISP'06)
Crawler Detection: A Bayesian Approach
Cap Esterel, Cote d'Azur, France
August 26-August 28
ISBN: 0-7695-2649-7
Athena Stassopoulou, Intercollege, Cyprus
Marios D. Dikaiakos, University of Cyprus, Cyprus
In this paper, we introduce a probabilistic modeling approach for addressing the problem of Web robot detection from Web-server access logs. More specifically, we construct a Bayesian network that classifies automatically access-log sessions as being crawler- or human-induced, by combining various pieces of evidence proven to characterize crawler and human behavior. Our approach uses machine learning techniques to determine the parameters of the probabilistic model. We apply our method to real Web-server logs and obtain results that demonstrate the robustness and effectiveness of probabilistic reasoning for crawler detection.
Citation:
Athena Stassopoulou, Marios D. Dikaiakos, "Crawler Detection: A Bayesian Approach," icisp, pp.16, International Conference on Internet Surveillance and Protection (ICISP'06), 2006
Usage of this product signifies your acceptance of the Terms of Use.