Issue No.10 - October (2009 vol.20)
Sangeetha Seshadri , Georgia Institute of Technology, Atlanta
Vibhore Kumar , IBM T.J Watson Research Center, Hawthorne
Brian Cooper , Yahoo! Research, Santa Clara
Ling Liu , Georgia Institute of Technology, Atlanta
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/TPDS.2008.232
This paper addresses the problem of optimizing multiple distributed stream queries that are executing simultaneously in distributed data stream systems. We argue that the static query optimization approach of "plan, then deployment” is inadequate for handling distributed queries involving multiple streams and node dynamics faced in distributed data stream systems and applications. Thus, the selection of an optimal execution plan in such dynamic and networked computing systems must consider operator ordering, reuse, network placement, and search space reduction. We propose to use hierarchical network partitions to exploit various opportunities for operator-level reuse while utilizing network characteristics to maintain a manageable search space during query planning and deployment. We develop top-down, bottom-up, and hybrid algorithms for exploiting operator-level reuse through hierarchical network partitions. Formal analysis is presented to establish the bounds on the search space and suboptimality of our algorithms. We have implemented our algorithms in the IFLOW [CHECK END OF SENTENCE] system, an adaptive distributed stream management system. Through simulations and experiments using a prototype deployed on Emulab [CHECK END OF SENTENCE], we demonstrate the effectiveness of our framework and our algorithms.
Computer-communication networks, distributed systems, distributed databases, distributed applications, database management, systems, query processing.
Sangeetha Seshadri, Vibhore Kumar, Brian Cooper, Ling Liu, "A Distributed Stream Query Optimization Framework through Integrated Planning and Deployment", IEEE Transactions on Parallel & Distributed Systems, vol.20, no. 10, pp. 1439-1453, October 2009, doi:10.1109/TPDS.2008.232