13th IEEE International Symposium on High Performance Distributed Computing (HPDC-13 '04)
Strategies for Using Additional Resources in Parallel Hash-Based Join Algorithms
Honolulu, Hawaii USA
June 04-June 06
ISBN: 0-7803-2175-4
DOI Bookmark:
http://doi.ieeecomputersociety.org/10.1109/HPDC.2004.34
Hash-based join is a compute- and memory-intensive algorithm. It achieves good performance and scales well to large datasets, if sufficient memory is available to hold the hash table and the distribution of computing load across nodes is balanced. In this paper, we compare three adaptive algorithms that start with a partitioning of the hash table across a group of nodes and expand during the hash table building phase to additional resources, when memory on a node is used up. The split-based algorithm partitions the hash table range assigned to the node, on which memory is full, into two segments and assigns one of the segments to a new node in the system. The replication-based algorithm replicates the hash table range on a new node. The hybrid algorithm combines the first and second strategies in order to address each strategy's short comings. We perform an experimental performance evaluation of these algorithms on a PC cluster. Our results show that among the three algorithms, in most cases the hybrid algorithm either performs close to the better of the two or is the best algorithm.
Citation:
Xi Zhang, Tahsin Kurc, Tony Pan, Umit Catalyurek, Sivaramakrishnan Narayanan, Pete Wyckoff, Joel Saltz, "Strategies for Using Additional Resources in Parallel Hash-Based Join Algorithms," hpdc, pp.4-13, 13th IEEE International Symposium on High Performance Distributed Computing (HPDC-13 '04), 2004
Usage of this product signifies your acceptance of the
Terms of Use.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||