The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.12 - Dec. (2012 vol.23)
pp: 2280-2288
Rafit Izhak-Ratzin , Palo Alto Networks, Santa Clara
Hyunggon Park , Ewha Womans University, Seoul
Mihaela van der Schaar , University of California Los Angeles, Los Angeles
ABSTRACT
We propose a BitTorrent-like protocol based on an online learning (reinforcement learning) mechanism, which can replace the peer selection mechanisms in the regular BitTorrent protocol. We model the peers' interactions in the BitTorrent-like network as a repeated stochastic game, where the strategic behaviors of the peers are explicitly considered. A peer that applies the reinforcement learning (RL)-based mechanism uses the observations on the associated peers' statistical reciprocal behaviors to determine its best responses and estimate the corresponding impact on its expected utility. The policy determines the peer's resource reciprocations such that the peer can maximize its long-term performance. We have implemented the proposed mechanism and incorporated it into an existing BitTorrent client. Our experiments performed on a controlled Planetlab testbed confirm that the proposed protocol 1) promotes fairness and provides incentives to contributed resources, i.e., high capacity peers improve their download completion time by up to 33 percent, 2) improves the system stability and robustness, i.e., reduces the peer selection fluctuations by 57 percent, and (3) discourages free-riding, i.e., peers reduce their uploads to free-riders by 64 percent as compared to the regular BitTorrent protocol.
INDEX TERMS
Peer to peer computing, Protocols, Learning systems, Resource management, Games, foresighted resource reciprocation strategy, Peer-to-peer (P2P), BitTorrent, reinforcement learning
CITATION
Rafit Izhak-Ratzin, Hyunggon Park, Mihaela van der Schaar, "Online Learning in BitTorrent Systems", IEEE Transactions on Parallel & Distributed Systems, vol.23, no. 12, pp. 2280-2288, Dec. 2012, doi:10.1109/TPDS.2012.90
REFERENCES
[1] IPOQUE, "IPOQUE Internet Measurements 2008-2009," http:/www.ipoque.com/, 2012.
[2] BitTorrent, "BitTorrent," http:/www.bittorrent.com/, 2012.
[3] "Skype," http:/skype.com, 2012.
[4] "PPLive," http:/pplive.com, 2012.
[5] B. Cohen, "Incentives Build Robustness in BitTorrent," Proc. Workshop Economics of Peer-to-Peer Systems (P2PEcon), 2003.
[6] M. Piatek, T. Isdal, T. Anderson, A. Krishnamurthy, and A. Venkataramani, "Do Incentives Build Robustness in BitTorrent?" Proc. Fourth USENIX Symp. Networked Systems Design and Implementation (NSDI), 2007.
[7] A. Bharambe, C. Herley, and V. Padmanabhan, "Analyzing and Improving a BitTorrent Network's Performance Mechanisms," Proc. IEEE INFOCOM, 2006.
[8] A. Legout, N. Liogkas, E. Kohler, and L. Zhang, "Clustering and Sharing Incentives in BitTorrent Systems," SIGMETRICS Performance Evaluation Rev., vol. 35, no. 1, pp. 301-312, 2007.
[9] M. Piatek, T. Isdal, A. Krishnamurthy, and T. Anderson, "One Hop Reputations for P2P File Sharing Workloads," Proc. Fifth USENIX Symp. Networked Systems Design and Implementation (NSDI), 2008.
[10] N. Liogkas, R. Nelson, E. Kohler, and L. Zhang, "Exploiting BitTorrent for Fun (But Not Profit)," Proc. Int'l Workshop Peer-To-Peer Systems (IPTPS), 2006.
[11] T. Locher, P. Moor, S. Schmid, and R. Wattenhofer, "Free Riding in BitTorrent is Cheap," Proc. Fifth Workshop Hot Topics in Networks (HotNets-V), 2006.
[12] M. Sirivianos, J.H. Park, R. Chen, and X. Yang, "Free-Riding in BitTorrent Networks with the Large View Exploit," Proc. Int'l Workshop Peer-To-Peer Systems (IPTPS), 2007.
[13] S. Buchegger and J.-Y. le Boudec, "A Robust Reputation System for P2P and Mobile Ad-Hoc Networks," Proc. Second Workshop Economics of P2P Systems, 2004.
[14] L. Xiong and L. Liu, "PeerTrust: Supporting Reputation-Based Trust for P2P Electronic Communities," IEEE Trans. Knowledge and Data Eng., vol. 16, no. 7, pp. 843-857, July 2004.
[15] R. Izhak-Razin, "Collaboration in BitTorrent Systems," Proc. Eight Int'l IFIP-TC 6 Networking Conf., 2009.
[16] R. Izhak-Razin, N. Liogkas, and R. Majumdar, "Team Incentives in BitTorrent Systems," Proc. 18th Int'l Conf. Computer Comm. and Networks (ICCCN), 2009.
[17] J. Hu and P. Wellman, "Multiagent Reinforcement Learning: Theorectical Framework and an Algorithm," Proc. Int'l Conf. Machine Learning, 1998.
[18] R.S. Sutton and A.G. Barto, Reinforcement Learning: An Introduction. MIT Press, 1998.
[19] A. Vlavianos, M. Iliofotou, and M. Faloutsos, "BiToS: Enhancing BitTorrent for Supporting Streaming Applications," Proc. IEEE INFOCOM, 2006.
[20] Y.R. Choe, D.L. Schuff, J.M. Dyaberi, and V.S. Pai, "Improving VoD Server Efficiency with BitTorrent," Proc. Int'l Conf. Multimedia, 2007.
[21] H. Park, R.I. Ratzin, and M. van der Schaar, "Peer-to-Peer Networks - Protocols, Cooperation and Competition," Streaming Media Architectures, Techniques, and Applications: Recent Advances, C. Zhu, Y. Li, and X. Niu, eds. IGI Global, 2011.
[22] R. Izhak-Ratzin, H. Park, and M. van der Schaar, "Reinforcement Learning in BitTorrent Systems," Proc. IEEE INFOCOM, pp. 406-410, Apr. 2011.
[23] H. Park and M. van der Schaar, "A Framework for Foresighted Resource Reciprocation in P2P Networks," IEEE Trans. Multimedia, vol. 11, no. 1, pp. 101-116, Jan. 2009.
[24] E.J. Wegman, "Maximum Likelihood Estimation of a Probability Density Function," The Indian J. Statistics, vol. 37, pp. 211-224, 1975.
[25] H. Park and M. van der Schaar, "Evolution of Resource Reciprocation Strategies in P2P Networks," IEEE Trans. Signal Processing, vol. 58, no. 3, pp. 1205-1218, Mar. 2010.
[26] "Enhanced-CTorrent," http://www.rahul.net/dholmesctorrent, 2012.
[27] M. Feldman, C. Papadimitriou, J. Chuang, and I. Stoica, "Free-Riding and Whitewashing in Peer-to-Peer Systems," Proc. ACM SIGCOMM Workshop Practice and Theory of Incentives in Networked Systems (PINS), pp. 228-236, Aug. 2004.
[28] M.L. Littman, T.L. Dean, and L.P. Kaelbling, "On the Complexity of Solving Markov Decision Problems," Proc. Conf. Uncertainty in Artificial Intelligence, May 1995.
[29] A. Bavier, M. Bowman, B. Chun, D. Culler, S. Karlin, S. Muir, L. Peterson, T. Roscoe, T. Spalink, and M. Wawrzoniak, "Operating System Support for Planetary-Scale Network Services," Proc. First Conf. Symp. Networked Systems Design and Implementation (NSDI), 2004.
[30] D. Levin, K. LaCurts, N. Spring, and B. Bhattacharjee, "BitTorrent is an Auction: Analyzing and Improving BitTorrent's Incentives," Proc. SIGCOMM, 2008.
29 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool