The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.06 - November/December (2008 vol.12)
pp: 22-29
Kristin Tufte , Portland State University
David Maier , Portland State University
Vassilis Papadimos , Microsoft
ABSTRACT
Memory efficiency is important for processing high-volume data streams. Previous stream-aggregation methods can exhibit excessive memory overhead in the presence of skewed data distributions. Further, data skew is a common feature of massive data streams. The authors introduce the AdaptWID algorithm, which uses adaptive processing to cope with time-varying data skew. AdaptWID models the memory usage of alternative aggregation algorithms and selects between them at runtime on a group-by-group basis. The authors' experimental study using the NiagaraST stream system verifies that the adaptive algorithm improves memory usage while maintaining execution cost and latency comparable to existing implementations.
INDEX TERMS
databases, query processing, data stream management
CITATION
Kristin Tufte, David Maier, Vassilis Papadimos, "AdaptWID: An Adaptive, Memory-Efficient Window Aggregation Implementation", IEEE Internet Computing, vol.12, no. 6, pp. 22-29, November/December 2008, doi:10.1109/MIC.2008.116
REFERENCES
1. J. Li et al., "Semantics and Evaluation Techniques for Window Aggregates in Data Streams," Proc. ACM SIGMOD Int'l Conf. Management of Data (SIGMOD 05), ACM Press, 2005, pp. 311–322.
2. P. Tucker et al., "Exploiting Punctuation Semantics in Continuous Data Streams," Trans. Knowledge and Data Eng., vol. 15, no. 3, 2003, pp. 555–568.
18 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool