This Article 
 Bibliographic References 
 Add to: 
On Composing Stream Applications in Peer-to-Peer Environments
August 2006 (vol. 17 no. 8)
pp. 824-837

Abstract—Stream processing has become increasingly important as many emerging applications call for continuous real-time processing over data streams, such as voice-over-IP telephony, security surveillance, and sensor data analysis. In this paper, we propose a composable stream processing system for cooperative peer-to-peer environments. The system can dynamically select and compose stream processing elements located on different peers into user desired applications. We investigate multiple alternative approaches to composing stream applications: 1) global-state-based centralized versus local-state-based distributed algorithms for initially composing stream applications at setup phase. The centralized algorithm performs periodical global state maintenance while the distributed algorithm performs on-demand state collection. 2) Reactive versus proactive failure recovery schemes for maintaining composed stream applications during runtime. The reactive failure recovery algorithm dynamically recomposes a new stream application upon failures while the proactive approach maintains a number of backup compositions for failure recovery. We conduct both theoretical analysis and experimental evaluations to study the properties of different approaches. Our study illustrates the performance and overhead trade-offs among different design alternatives, which can provide important guidance for selecting proper algorithms to compose stream applications in cooperative peer-to-peer environments.

[1] Gnutella, http:/, 2006.
[2] PlanetLab, http:/, 2006.
[3] T.F. Abdelzaher, “An Antomated Profiling Subsystem for QoS-Aware Services,” Proc. IEEE Real-Time Technology and Applications Symp., June 2000.
[4] A. Adya et al., “FARSITE: Federated, Available, and Reliable Storage for an Incompletely Trusted Environment,” Proc. Fifth Symp. Operating Systems Design and Implementation (OSDI '02), Dec. 2002.
[5] D. Andersen, H. Balakrishnan, F. Kaashoek, and R. Morris, “Resilient Overlay Networks,” Proc. 18th ACM Symp. Operating Systems Principles (SOSP), Oct. 2001.
[6] R. Bhagwan, K. Tati, Y. Cheng, S. Savage, and G.M. Voelker, “TotalRecall: System Support for Automated Availability Management,” Proc. ACM/USENIX Symp. Networked Systems Design and Implementation (NSDI), Mar. 2004.
[7] A.P. Black, J. Huang, J. Walpole, and C. Pu, “Infopipes: An Abstraction for Multimedia Streaming,” Multimedia, vol. 8, no. 5, pp. 406-419, 2002.
[8] M. Castro, P. Druschel, A-M Kermarrec, A. Nandi, A. Rowstron, and A. Singh, “SplitStream: High-Bandwidth Multicast in Cooperative Environments,” Proc. ACM Symp. Operating Systems Principles (SOSP), Oct. 2003.
[9] Y. Chu, S.G. Rao, S. Seshan, and H. Zhang, “A Case for End System Multicast,” IEEE J. Selected Areas in Comm. (JSAC), 2002.
[10] X. Fu, W. Shi, A. Akkerman, and V. Karamcheti, “CANS: Composable, Adaptive Network Services Infrastructure,” Proc. Third USENIX Symp. Internet Technologies and Systems, Mar. 2001.
[11] M.R. Garey and D.S. Johnson, “Computers and Intractability,” A Guide to the Theory of NP-Completeness, 1979.
[12] X. Gu and K. Nahrstedt, “Distributed Multimedia Service Composition with Statistical QoS Assurances,” IEEE Trans. Multimedia, 2005.
[13] X. Gu, K. Nahrstedt, R.N. Chang, and C. Ward, “QoS-Assured Service Composition in Managed Service Overlay Networks,” Proc. IEEE 23rd Int'l Conf. Distributed Computing Systems (ICDCS '03), 2003.
[14] X. Gu, K. Nahrstedt, and B. Yu, “SpiderNet: An Integrated Peer-to-Peer Service Composition Framework,” Proc. IEEE Int'l Symp. High-Performance Distributed Computing (HPDC-13), June 2004.
[15] X. Gu, K. Nahrstedt, W. Yuan, D. Wichadakul, and D. Xu, “An XML-Based QoS Enabling Language for the Web,” J. Visual Language and Computing, 2001.
[16] X. Gu, P.S. Yu, and K. Nahrstedt, “Optimal Component Composition for Scalable Stream Processing,” Proc. IEEE 23rd Int'l Conf. Distributed Computing Systems (ICDCS '05), 2005.
[17] M. Hefeeda, A. Habib, B. Botev, D. Xu, and B. Bhargava, “Promise: Peer-to-Peer Media Streaming Using Collectcast,” Proc. ACM Multimedia Conf., Nov. 2003.
[18] N. Hu and P. Steenkiste, “Evaluation and Characterization of Available Bandwidth Probing Techniques,” IEEE J. Special Areas in Comm., special issue on Internet and WWW measurement, mapping, and modeling, vol. 21, no. 6, Aug. 2003.
[19] M. Jain and C. Dovrolis, “End-to-End Available Bandwidth: Measurement Methodology, Dynamics, and Relation with TCP Throughput,” Proc. ACM SIGCOMM Conf., Aug. 2002.
[20] D. Kostic, A. Rodriguez, J. Albrecht, and A. Vahdat, “Bullet: High Bandwidth Data Dissemination Using an Overlay Mesh,” Proc. ACM Symp. Operating Systems Principles (SOSP), Oct. 2003.
[21] P.K. Mckinley, U.I. Padmanabhan, N. Ancha, and S.M. Sadjadi, “Composable Proxy Services to Support Collaboration on the Mobile Internet,” IEEE Trans. Computers, vol. 52, no. 6, June 2003.
[22] T.-W. Ngan, D. Wallach, and P. Druschel, “Enforcing Fair Sharing of Peer-to-Peer Resources,” Proc. Second Int'l Workshop Peer-to-Peer Systems (IPTPS '03), Feb. 2003.
[23] W.T. Ooi and R.V. Renesse, “Distributing Media Transformation over Multiple Media Gateways,” Proc. Ninth ACM Int'l Multimedia Conf., Sept. 2001.
[24] J.P. Loyall, D.E. Bakken, R.E. Schantz, J.A. Zinky, D.A. Karr, R. Vanegas, and K.R. Anderson, “QoS Aspect Languages and Their Runtime Integration,” Lecture Notes in Computer Science, vol. 1511, May 1998.
[25] B. Raman and R.H. Katz, “An Architecture for Highly Available Wide-Area Service Composition,” Computer Comm. J., May 2003.
[26] S. Ratnasamy, P. Francis, M. Handley, R. Karp, and S. Shenker, “A Scalable Content-Addressable Network,” Proc. ACM SIGCOMM 2001 Conf., 2001.
[27] S. Ratnasamy, M. Handley, R. Karp, and S. Shenker, “Topologically-Aware Overlay Construction and Server Selection,” Proc. Infocom 2002 Conf., 2002.
[28] A. Rowstron and P. Druschel, “Pastry: Scalable, Distributed Object Location and Routing for Large-Scale Peer-to-Peer Systems,” Proc. IFIP/ACM Int'l Conf. Distributed Systems Platforms (Middleware), Nov. 2001.
[29] S. Frolund and J. Koistinen, “QML: A Language for Quality of Service Specification,” Technical Report HPL-98-10, Feb. 1998.
[30] A.C. Snoeren, K. Conley, and D.K. Gifford, “Mesh Based Content Routing Using XML,” Proc. 18th ACM Symp. Operating Systems Principles (SOSP), Oct. 2001.
[31] I. Stoica, R. Morris, D. Karger, M.F. Kaashoek, and H. Balakrishnan, “Chord: A Scalable Peer-to-Peer Lookup Service for Internet Applications,” Proc. ACM SIGCOMM Conf., Aug. 2001.
[32] H. Tangmunarunkit, R. Govindan, S. Jamin, S. Shenker, and W. Willinger, “Network Topology Genrators: Degree-Based vs. Structural,” Proc. ACM SIGCOMM Conf., Aug. 2002.
[33] D. Tran, K. Hua, and T. Do, “Zigzag: An Efficient Peer-to-Peer Scheme for Media Streaming,” Proc. IEEE INFOCOM Conf., Apr. 2003.
[34] J. Winick and S. Jamin, “Inet3.0: Internet Topology Generator,” Technical Report UM-CSE-TR-456-02, http://irl.eecs.umich.edujamin/, 2002.
[35] L. Xiao, Z. Xu, and X. Zhang, “Low-Cost and Reliable Mutual Anonymity Protocols in Peer-to-Peer Networks,” IEEE Trans. Parallel and Distributed Systems, vol. 14, no. 9, Sept. 2003.

Index Terms:
Peer-to-peer, stream processing, service composition, resource management, quality-of-service.
Xiaohui Gu, Klara Nahrstedt, "On Composing Stream Applications in Peer-to-Peer Environments," IEEE Transactions on Parallel and Distributed Systems, vol. 17, no. 8, pp. 824-837, Aug. 2006, doi:10.1109/TPDS.2006.107
Usage of this product signifies your acceptance of the Terms of Use.