2018 IEEE 34th International Conference on Data Engineering (ICDE) (2018)
Apr 16, 2018 to Apr 19, 2018
In this paper, we propose a novel pairwise crowd-sourcing model to reduce the uncertainty of top-k ranking using a crowd of domain experts. Given a crowdsourcing task of limited budget, we propose efficient algorithms to select the best object pairs for crowdsourcing that will bring in the highest quality improvement. Extensive experiments show that our proposed solutions outperform a random selection method by up to 30 times in terms of quality improvement of probabilistic top-k ranking queries. In terms of efficiency, our proposed solutions can reduce the elapsed time of a brute-force algorithm from several days to one minute.
crowdsourcing, probability, query processing
X. Lin, J. Xu, H. Hu and F. Zhe, "Reducing Uncertainty of Probabilistic Top-k Ranking via Pairwise Crowdsourcing," 2018 IEEE 34th International Conference on Data Engineering (ICDE), Paris, France, 2018, pp. 1757-1758.