2015 IEEE 31st International Conference on Data Engineering (ICDE) (2015)
Seoul, South Korea
April 13, 2015 to April 17, 2015
Chuanfei Xu , Department of Computing, The Hong Kong Polytechnic University, China
Bo Tang , Department of Computing, The Hong Kong Polytechnic University, China
Man Lung Yiu , Department of Computing, The Hong Kong Polytechnic University, China
Commercial web search engines adopt parallel and replicated architecture in order to support high query throughput. In this paper, we investigate the effect of caching on the throughput in such a setting. A simple scheme, called uniform caching, would replicate the cache content to all servers. Unfortunately, it does not exploit the variations among queries, thus wasting memory space on caching the same cache content redundantly on multiple servers. To tackle this limitation, we propose a diversified caching problem, which aims to diversify the types of queries served by different servers, and maximize the sharing of terms among queries assigned to the same server. We show that it is NP-hard to find the optimal diversified caching scheme, and identify intuitive properties to seek good solutions. Then we present a framework with a suite of techniques and heuristics for diversified caching. Finally, we evaluate the proposed solution with competitors by using a real dataset and a real query log.
Servers, Throughput, Silicon, Indexes, Computer architecture, Search engines, Training
C. Xu, B. Tang and M. L. Yiu, "Diversified caching for replicated web search engines," 2015 IEEE 31st International Conference on Data Engineering (ICDE), Seoul, South Korea, 2015, pp. 207-218.