The Community for Technology Leaders
Green Image
We propose a novel predictive quantization (PQ) based approach for online summarization of multiple time varying data streams. A synopsis over a sliding window of most recent entries is computed in one pass and dynamically updated in constant time. The correlation between consecutive data elements is effectively taken into account without the need for preprocessing. We extend PQ to multiple streams and propose structures for real-time summarization and querying of a massive number of streams. Queries on any subsequence of a sliding window over multiple streams are processed in real-time. We examine each component of the proposed approach, prediction and quantization, separately and investigate the space-accuracy trade off for synopsis generation. Complementing the theoretical optimality of PQ based approaches, we show that the proposed technique, even for very short prediction windows, significantly outperforms the current techniques for a wide variety of query types on both synthetic and real data sets.
multiple streams, Prediction, quantization, summarization, online update
Fatih Altiparmak, Hakan Ferhatosmanoglu, Ertem Tuncel, "Incremental Maintenance of Online Summaries Over Multiple Streams", IEEE Transactions on Knowledge & Data Engineering, vol. 20, no. , pp. 216-229, February 2008, doi:10.1109/TKDE.2007.190693
86 ms
(Ver )