The Community for Technology Leaders
Green Image
Issue No. 08 - August (2009 vol. 21)
ISSN: 1041-4347
pp: 1178-1190
Ruihua Song , Microsoft Research Asia, Beijing
Ji-Rong Wen , Microsoft Research Asia, Beijing
Xiaojie Yuan , Nankai University, Tianjin
Zhicheng Dou , Microsoft Research Asia, Beijing
ABSTRACT
Although personalized search has been under way for many years and many personalization algorithms have been investigated, it is still unclear whether personalization is consistently effective on different queries for different users and under different search contexts. In this paper, we study this problem and provide some findings. We present a large-scale evaluation framework for personalized search based on query logs and then evaluate five personalized search algorithms (including two click-based ones and three topical-interest-based ones) using 12-day query logs of Windows Live Search. By analyzing the results, we reveal that personalized Web search does not work equally well under various situations. It represents a significant improvement over generic Web search for some queries, while it has little effect and even harms query performance under some situations. We propose click entropy as a simple measurement on whether a query should be personalized. We further propose several features to automatically predict when a query will benefit from a specific personalization algorithm. Experimental results show that using a personalization algorithm for queries selected by our prediction model is better than using it simply for all queries.
INDEX TERMS
Web search, personalization, information filtering, performance evaluation.
CITATION
Ruihua Song, Ji-Rong Wen, Xiaojie Yuan, Zhicheng Dou, "Evaluating the Effectiveness of Personalized Web Search", IEEE Transactions on Knowledge & Data Engineering, vol. 21, no. , pp. 1178-1190, August 2009, doi:10.1109/TKDE.2008.172
105 ms
(Ver )