The Community for Technology Leaders
High-Performance Distributed Computing, International Symposium on (2004)
Honolulu, Hawaii USA
June 4, 2004 to June 6, 2004
ISSN: 1082-8907
ISBN: 0-7803-2175-4
pp: 4-13
Tony Pan , The Ohio State University, Columbus, OH
Sivaramakrishnan Narayanan , The Ohio State University, Columbus, OH
Pete Wyckoff , Ohio Supercomputer Center, Columbus, OH
Joel Saltz , The Ohio State University, Columbus, OH
Xi Zhang , The Ohio State University, Columbus, OH
Umit Catalyurek , The Ohio State University, Columbus, OH
Tahsin Kurc , The Ohio State University, Columbus, OH
ABSTRACT
Hash-based join is a compute- and memory-intensive algorithm. It achieves good performance and scales well to large datasets, if sufficient memory is available to hold the hash table and the distribution of computing load across nodes is balanced. In this paper, we compare three adaptive algorithms that start with a partitioning of the hash table across a group of nodes and expand during the hash table building phase to additional resources, when memory on a node is used up. The split-based algorithm partitions the hash table range assigned to the node, on which memory is full, into two segments and assigns one of the segments to a new node in the system. The replication-based algorithm replicates the hash table range on a new node. The hybrid algorithm combines the first and second strategies in order to address each strategy's short comings. We perform an experimental performance evaluation of these algorithms on a PC cluster. Our results show that among the three algorithms, in most cases the hybrid algorithm either performs close to the better of the two or is the best algorithm.
INDEX TERMS
null
CITATION
Tony Pan, Sivaramakrishnan Narayanan, Pete Wyckoff, Joel Saltz, Xi Zhang, Umit Catalyurek, Tahsin Kurc, "Strategies for Using Additional Resources in Parallel Hash-Based Join Algorithms", High-Performance Distributed Computing, International Symposium on, vol. 00, no. , pp. 4-13, 2004, doi:10.1109/HPDC.2004.34
108 ms
(Ver )