The Community for Technology Leaders
Proceedings 18th International Conference on Data Engineering (2002)
San Jose, California
Feb. 26, 2002 to Mar. 1, 2002
ISBN: 0-7695-1531-2
pp: 0706
Per-Åke Larson , Microsoft
ABSTRACT
Partial preaggregation is a simple data reduction operator that can be applied to aggregation queries. Whenever we group and aggregate on a column set G, we can preaggregate on any column set that functionally determines G. Preaggregation can be used, for example, to reduce the input size to a join. Regular aggregation reduces the input to one record per group. Partial preaggregation exploit the fact that preaggregation need not be complete -if multiple record happen to be output for a group, they will be combined into the same group by the final aggregation. This paper describes a straightforward hash-based algorithm for partial preaggregation, discusses where it can be applied, and derives a mathematical model for estimating the output size. The effectiveness of the technique and the accuracy of the model are shown on both artificial and real data. It is also shown how to reduce memory requirement by combining partial preaggregation with the input phase of a subsequent join or sort operator. Partial preaggregation has been implemented, in part, in Microsoft SQL Server.
INDEX TERMS
CITATION

P. Larson, "Data Reduction by Partial Preaggregation," Proceedings 18th International Conference on Data Engineering(ICDE), San Jose, California, 2002, pp. 0706.
doi:10.1109/ICDE.2002.994787
85 ms
(Ver 3.3 (11022016))