Issue No.06 - November/December (2008 vol.12)
Kristin Tufte , Portland State University
David Maier , Portland State University
Vassilis Papadimos , Microsoft
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/MIC.2008.116
Memory efficiency is important for processing high-volume data streams. Previous stream-aggregation methods can exhibit excessive memory overhead in the presence of skewed data distributions. Further, data skew is a common feature of massive data streams. The authors introduce the AdaptWID algorithm, which uses adaptive processing to cope with time-varying data skew. AdaptWID models the memory usage of alternative aggregation algorithms and selects between them at runtime on a group-by-group basis. The authors' experimental study using the NiagaraST stream system verifies that the adaptive algorithm improves memory usage while maintaining execution cost and latency comparable to existing implementations.
databases, query processing, data stream management
Kristin Tufte, David Maier, Vassilis Papadimos, "AdaptWID: An Adaptive, Memory-Efficient Window Aggregation Implementation", IEEE Internet Computing, vol.12, no. 6, pp. 22-29, November/December 2008, doi:10.1109/MIC.2008.116