loading...
 This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
21st International Conference on Data Engineering (ICDE'05)
Dynamic Load Distribution in the Borealis Stream Processor
Tokyo, Japan
April 05-April 08
ISBN: 0-7695-2285-8
Ying Xing, Brown University
Stan Zdonik, Brown University
Jeong-Hyon Hwang, Brown University

Distributed and parallel computing environments are becoming cheap and commonplace. The availability of large numbers of CPU?s makes it possible to process more data at higher speeds. Stream-processing systems are also becoming more important, as broad classes of applications require results in real-time.

Since load can vary in unpredictable ways, exploiting the abundant processor cycles requires effective dynamic load distribution techniques. Although load distribution has been extensively studied for the traditional pull-based systems, it has not yet been fully studied in the context of push-based continuous query processing.

In this paper, we present a correlation based load distribution algorithm that aims at avoiding overload and minimizing end-to-end latency by minimizing load variance and maximizing load correlation. While finding the optimal solution for such a problem is NP-hard, our greedy algorithm can find reasonable solutions in polynomial time. We present both a global algorithm for initial load distribution and a pair-wise algorithm for dynamic load migration.

Citation:
Ying Xing, Stan Zdonik, Jeong-Hyon Hwang, "Dynamic Load Distribution in the Borealis Stream Processor," icde, pp.791-802, 21st International Conference on Data Engineering (ICDE'05), 2005
Usage of this product signifies your acceptance of the Terms of Use.