The Community for Technology Leaders
Green Image
Issue No. 06 - November/December (2008 vol. 12)
ISSN: 1089-7801
pp: 22-29
Vassilis Papadimos , Microsoft
Jin Li , Portland State University
David Maier , Portland State University
Kristin Tufte , Portland State University
ABSTRACT
Memory efficiency is important for processing high-volume data streams. Previous stream-aggregation methods can exhibit excessive memory overhead in the presence of skewed data distributions. Further, data skew is a common feature of massive data streams. The authors introduce the AdaptWID algorithm, which uses adaptive processing to cope with time-varying data skew. AdaptWID models the memory usage of alternative aggregation algorithms and selects between them at runtime on a group-by-group basis. The authors' experimental study using the NiagaraST stream system verifies that the adaptive algorithm improves memory usage while maintaining execution cost and latency comparable to existing implementations.
INDEX TERMS
databases, query processing, data stream management
CITATION
Vassilis Papadimos, Jin Li, David Maier, Kristin Tufte, "AdaptWID: An Adaptive, Memory-Efficient Window Aggregation Implementation", IEEE Internet Computing, vol. 12, no. , pp. 22-29, November/December 2008, doi:10.1109/MIC.2008.116
136 ms
(Ver )