The Community for Technology Leaders
Green Image
Quantile computation has many applications including data mining and financial data analysis. It has been shown that an \epsilon{\hbox{-}}{\rm{approximate}} summary can be maintained so that, given a quantile query (\phi, \epsilon), the data item at rank \lceil \phi N \rceil may be approximately obtained within the rank error precision \epsilon N over all N data items in a data stream or in a sliding window. However, scalable online processing of massive continuous quantile queries with different \phi and \epsilon poses a new challenge because the summary is continuously updated with new arrivals of data items. In this paper, first we aim to dramatically reduce the number of distinct query results by grouping a set of different queries into a cluster so that they can be processed virtually as a single query while the precision requirements from users can be retained. Second, we aim to minimize the total query processing costs. Efficient algorithms are developed to minimize the total number of times for reprocessing clusters and to produce the minimum number of clusters, respectively. The techniques are extended to maintain near-optimal clustering when queries are registered and removed in an arbitrary fashion against whole data streams or sliding windows. In addition to theoretical analysis, our performance study indicates that the proposed techniques are indeed scalable with respect to the number of input queries as well as the number of items and the item arrival rate in a data stream.
Query processing, online computation, data mining.
Xiaofang Zhou, Qing Zhang, Hongjun Lu, Yidong Yuan, Jeffrey Xu Yu, Jian Xu, Xuemin Lin, "Approximate Processing of Massive Continuous Quantile Queries over High-Speed Data Streams", IEEE Transactions on Knowledge & Data Engineering, vol. 18, no. , pp. 683-698, May 2006, doi:10.1109/TKDE.2006.73
106 ms
(Ver )