The Community for Technology Leaders
Green Image
Issue No. 11 - Nov. (2013 vol. 25)
ISSN: 1041-4347
pp: 2658-2663
Jianguo Lu , University of Windsor, Windsor
Dingding Li , University of Windsor, Windsor
ABSTRACT
This paper discusses the bias problem when estimating the population size of big data such as online social networks (OSN) using uniform random sampling and simple random walk. Unlike the traditional estimation problem where the sample size is not very small relative to the data size, in big data, a small sample relative to the data size is already very large and costly to obtain. We point out that when small samples are used, there is a bias that is no longer negligible. This paper shows analytically that the relative bias can be approximated by the reciprocal of the number of collisions; thereby, a bias correction estimator is introduced. The result is further supported by both simulation studies and the real Twitter network that contains 41.7 million nodes.
INDEX TERMS
Equations, Estimation, Sociology, Statistics, Mathematical model, Twitter, Information management,size estimation, Big data, online social networks, small sample, bias
CITATION
Jianguo Lu, Dingding Li, "Bias Correction in a Small Sample from Big Data", IEEE Transactions on Knowledge & Data Engineering, vol. 25, no. , pp. 2658-2663, Nov. 2013, doi:10.1109/TKDE.2012.220
193 ms
(Ver 3.3 (11022016))