The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.08 - August (2010 vol.22)
pp: 1126-1141
Mengmeng Liu , University of Pennsylvania, Philadelphia
Nicholas E. Taylor , University of Pennsylvania, Philadelphia
Wenchao Zhou , University of Pennsylvania, Philadelphia
Zachary G. Ives , University of Pennsylvania, Philadelphia
Boon Thau Loo , University of Pennsylvania, Philadelphia
ABSTRACT
The data management community has recently begun to consider declarative network routing and distributed acquisition: e.g., sensor networks that execute queries about contiguous regions, declarative networks that maintain shortest paths, and distributed and peer-to-peer stream systems that detect transitive relationships among data at the distributed sources. In each case, the fundamental operation is to maintain a view over dynamic network state. This view is typically distributed, recursive, and may contain aggregation, e.g., describing shortest paths or least costly paths. Surprisingly, solutions to computing such views are often domain-specific, expensive, and incomplete. We recast the problem as incremental recursive view maintenance given distributed streams of updates to tuples: new stream data becomes insert operations and tuple expirations become deletions. We develop techniques to maintain compact information about tuple derivability or data provenance. We complement this with techniques to reduce communication: aggregate selections to prune irrelevant aggregation tuples, provenance-aware operators that determine when tuples are no longer derivable and remove them from the view, and shipping operators that reduce the information being propagated while still maintaining correct answers. We validate our work in a distributed setting with sensor and network router queries, showing significant gains in communication overhead without sacrificing performance.
INDEX TERMS
Distributed databases, query processing.
CITATION
Mengmeng Liu, Nicholas E. Taylor, Wenchao Zhou, Zachary G. Ives, Boon Thau Loo, "Maintaining Recursive Views of Regions and Connectivity in Networks", IEEE Transactions on Knowledge & Data Engineering, vol.22, no. 8, pp. 1126-1141, August 2010, doi:10.1109/TKDE.2010.65
REFERENCES
[1] H. Balakrishnan, M.F. Kaashoek, D. Karger, R. Morris, and I. Stoica, "Looking Up Data in P2P Systems," Comm. ACM, vol. 46, no. 2, pp. 43-48, Feb. 2003.
[2] D. Chu, L. Popa, A. Tavakoli, J.M. Hellerstein, P. Levis, S. Shenker, and I. Stoica, "The Design and Implementation of a Declarative Sensor Network System," Proc. SenSys, 2007.
[3] B.T. Loo, J.M. Hellerstein, I. Stoica, and R. Ramakrishnan, "Declarative Routing: Extensible Routing with Declarative Queries," Proc. ACM SIGCOMM Conf. Data Comm., 2005.
[4] A.J. Demers, J. Gehrke, R. Rajaraman, A. Trigoni, and Y. Yao, "The Cougar Project: A Work-in-Progress Report," ACM SIGMOD Record, vol. 32, no. 3, 2003.
[5] S. Madden, M.J. Franklin, J.M. Hellerstein, and W. Hong, "Design of an Acquisitional Query Processor for Sensor Networks," Proc. ACM SIGMOD, 2003.
[6] D. Narayanan, A. Donnelly, R. Mortier, and A. Rowstron, "Delay Aware Querying with Seaweed," Proc. Int'l Conf. Very Large Data Bases (VLDB), 2006.
[7] R. Huebsch, J.M. Hellerstein, N. Lanham, B.T. Loo, S. Shenker, and I. Stoica, "Quering the Internet with PIER," Proc. Int'l Conf. Very Large Data Bases (VLDB), 2003.
[8] T.J. Green, G. Karvounarakis, Z.G. Ives, and V. Tannen, "Update Exchange with Mappings and Provenance," Proc. Int'l Conf. Very Large Data Bases (VLDB), 2007.
[9] B.T. Loo, T. Condie, M. Garofalakis, D.E. Gay, J.M. Hellerstein, P. Maniatis, R. Ramakrishnan, T. Roscoe, and I. Stoica, "Declarative Networking: Language, Execution and Optimization," Proc. ACM SIGMOD, June 2006.
[10] M. Welsh and G. Mainland, "Programming Sensor Networks Using Abstract Regions," Proc. Third Symp. Networked Systems Design and Implementation (NSDI), Mar. 2004.
[11] K. Whitehouse, C. Sharp, E. Brewer, and D. Culler, "Hood: A Neighborhood Abstraction for Sensor Networks," Proc. Int'l Conf. Mobile Systems, Applications And Services (MASN), 2004.
[12] I. Balbin and K. Ramamohanarao, "A Generalization of the Differential Approach to Recursive Query Evaluation," J. Logic Programming, vol. 4, no. 3, pp. 259-262, 1987.
[13] F. Bancilhon, D. Maier, Y. Sagiv, and J.D. Ullman, "Magic Sets and Other Strange Ways to Implement Logic Programs," Proc. ACM SIGACT-SIGMOD Symp. Principles of Database Systems (PODS), 1986.
[14] C. Intanagonwiwat, R. Govindan, and D. Estrin, "Directed Diffusion: A Scalable and Robust Communication Paradigm for Sensor Networks," Proc. MobiCom, 2000.
[15] A. Gupta, I.S. Mumick, and V.S. Subrahmanian, "Maintaining Views Incrementally," Proc. ACM SIGMOD, 1993.
[16] M. Liu, N.E. Taylor, W. Zhou, Z.G. Ives, and B.T. Loo, "Recursive Computation of Regions and Connectivity in Networks," Proc. Int'l Conf. Data Eng. Conf. (ICDE), 2009.
[17] S. Raman and S. McCanne, "A Model, Analysis, and Protocol Framework for Soft State-Based Communication," Proc. ACM SIGCOMM Conf. Data Comm., pp. 15-25, 1999.
[18] P. Buneman, S. Khanna, and W.C. Tan, "Why and Where: A Characterization of Data Provenance," Proc. Int'l Conf. Digital Telecomm. (ICDT), 2001.
[19] J. Cheney, L. Chiticariu, and W.-C. Tan, "Provenance in Databases: Why, How, and Where," Foundations and Trends in Databases, vol. 1, no. 4, pp. 379-474, 2009.
[20] Y. Cui, "Lineage Tracing in Data Warehouses," PhD dissertation, Stanford Univ., 2001.
[21] T.J. Green, G. Karvounarakis, and V. Tannen, "Provenance Semirings," Proc. ACM Symp. Principles of Database Systems (PODS), 2007.
[22] R. Bryant, "Graph-Based Algorithms for Boolean Function Manipulation," IEEE Trans. Computers, vol. 35, no. 8, pp. 677-691, Aug. 1986.
[23] J. Whaley, "JavaBDD Library," http:/javabdd.sourceforge.net, 2010.
[24] M. Liu, N.E. Taylor, W. Zhou, Z.G. Ives, and B.T. Loo, "Recursive Computation of Regions and Connectivity in Networks," Technical Report MS-CIS-08-32, Univ. of Pennsylvania, 2008.
[25] T.T. Christoph Meinel, Algorithms and Data Structures in VLSI Design, pp. 1-267. Springer-Verlag, Aug. 1998.
[26] B. Bollig, M. Lobbing, and I. Wegener, "Simulated Annealing to Improve Variable Orderings for OBDDs," Proc. Int'l Workshop Logic Synthesis, 1995.
[27] R. Drechsler and N.G. amd Bernd Becker, "Learning Heuristics for OBDD Minimization by Evolutionary Algorithms," Proc. Parallel Problem Solving from Nature (PPSN IV), 1996.
[28] O. Grumberg, S. Livne, and S. Markovitch, "Learning to Order BDD Variables in Verification," J. AI Research, vol. 18, pp. 83-116, 2003.
[29] M. Carbin, "Learning Effective BDD Variable Orders for BDD-Based Program Analysis," honors thesis, Stanford Univ., May 2006.
[30] D. Olteanu and J. Huang, "Using OBDDs for Efficient Query Evaluation on Probabilistic Databases," Proc. Scalable Uncertainty Management, 2008.
[31] S. Sudarshan and R. Ramakrishnan, "Aggregation and Relevance in Deductive Databases," Proc. Int'l Conf. Very Large Data Bases (VLDB), 1991.
[32] R. Motwani, J. Widom, A. Arasu, B. Babcock, S. Babu, M. Datar, G. Manku, C. Olston, J. Rosenstein, and R. Varma, "Query Processing, Resource Management, and Approximation in a Data Stream Management System," Proc. The Conf. Innovative Data Systems Research (CIDR), 2003.
[33] S. Chaudhuri and K. Shim, "Including Group-By in Query Optimization," Proc. Int'l Conf. Very Large Data Bases (VLDB), 1994.
[34] N.E. Taylor and Z.G. Ives, "Reliable Storage and Querying for Collaborative Data Sharing Systems," Proc. Int'l Conf. Data Eng. Conf. (ICDE), 2010.
[35] A. Rowstron and P. Druschel, "Pastry: Scalable, Distributed Object Location and Routing for Large-Scale Peer-to-Peer Systems," Proc. Middleware, pp. 329-350, Nov. 2001.
[36] GT-ITM, "Modelling Topology of Large Networks," http://www.cc.gatech.edu/projectsgtitm/, 2010.
[37] D.J. Abadi, D. Carney, U. Cetintemel, M. Cherniack, C. Convey, S. Lee, M. Stonebraker, N. Tatbul, and S. Zdonik, "Aurora: A New Model and Architecture for Data Stream Management," VLDB J., vol. 12, no. 2, pp. 120-139, Aug. 2003.
[38] S. Chandrasekaran, O. Cooper, A. Deshpande, M.J. Franklin, J.M. Hellerstein, W. Hong, S. Krishnamurthy, S. Madden, V. Raman, F. Reiss, and M.A. Shah, "TelegraphCQ: Continuous Dataflow Processing for an Uncertain World," Proc. The Conf. Innovative Data Systems Research (CIDR), 2003.
[39] A. Arasu, S. Babu, and J. Widom, "The CQL Continuous Query Language: Semantic Foundations and Query Execution," VLDB J., vol. 15, no. 2, pp. 121-142, 2006.
[40] O. Benjelloun, A.D. Sarma, A.Y. Halevy, and J. Widom, "ULDBs: Databases with Uncertainty and Lineage." Proc. Int'l Conf. Very Large Data Bases (VLDB), 2006.
60 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool