The Community for Technology Leaders
RSS Icon
Subscribe
Issue No.01 - January (2009 vol.20)
pp: 111-123
Sumantra R. Kundu , University of Texas at Arlington, Arlington
Sourav Pal , University of Texas at Arlington, Arlington
Kalyan Basu , University of Texas at Arlington, Arlington
Sajal K. Das , University of Texas at Arlington, Arlington
ABSTRACT
In networks carrying large volume of traffic, accurate traffic characterization is necessary for understanding the dynamics and patterns of network resource usage. Previous approaches to flow characterization are based on random sampling of the packets (e.g., Cisco's NetFlow) or inferring characteristics solely based on long lived flows (LLFs) or on lossy data structures (e.g., bloom filters, hash tables). However, none of these approaches takes into account the heavy-tailed nature of the Internet traffic and separates the estimation algorithm from the flow measurement architecture.In this paper, we propose an alternate approach to traffic characterization by closely linking the flow measurement architecture with the estimation algorithm. Our measurement framework stores complete information related to short lived flows (SLFs) while collecting partial information related to LLFs. For real-time separation of LLFs and SLFs, we propose a novel algorithm based on typical sequences from Information theory. The distribution (pdf) and sample space of the underlying traffic is estimated using the non-parametric Parzen window technique and likelihood function defined over the Coupon collector problem. We validate the accuracy and performance of our estimation technique using traffic traces from the internal LAN in our laboratory and from National Library for Applied Network Research (NLANR).
INDEX TERMS
Communication/Networking and Information Technology, Special-Purpose and Application-Based Systems
CITATION
Sumantra R. Kundu, Sourav Pal, Kalyan Basu, Sajal K. Das, "An Architectural Framework for Accurate Characterization of Network Traffic", IEEE Transactions on Parallel & Distributed Systems, vol.20, no. 1, pp. 111-123, January 2009, doi:10.1109/TPDS.2008.47
REFERENCES
[1] G. Appenzeller, “Sizing Router Buffers,” PhD dissertation, Dept. Electrical Eng., Stanford Univ., Mar. 2005.
[2] N. Brownlee, “Understanding Internet Traffic Streams: Dragonflies and Tortoises,” IEEE Comm. Magazine, vol. 40, pp. 110-117, Oct. 2002.
[3] J. Cao, W.S. Cleveland, D. Lin, and D.X. Sun, “Internet Traffic Tends towards Poisson and Independent as Load Increases,” Nonlinear Estimation and Classification. pp. 83-109, Springer, 2002.
[4] K.C. Clay, G.C. Polyzos, and H.W. Braun, “Application of Sampling Methodologies to Network Trace Characterization,” Proc. ACM SIGCOMM '93, pp. 13-17, 1993.
[5] G. Cormode and S. Muthukrishnan, “What's Hot and What's Not: Tracking Most Frequent Items Dynamically,” Proc. ACM Ann. ACM Symp. Principles of Distributed Computing (PODC '03), pp.296-306, July 2003.
[6] T.M. Cover and J.A. Thomas, Elements of Information Theory. John Wiley & Sons, 1991.
[7] N. Duffield, C. Lund, and M. Thorup, “Properties and Prediction of Flow Statistics from Sampled Packet Streams,” Proc. ACM SIGCOMM Internet Measurement Workshop (IMW '02), pp. 159-171, Nov. 2002.
[8] N. Duffield, C. Lund, and M. Thorup, “Flow Sampling under Hard Resource Constraints,” Proc. ACM SIGMETRICS '04, pp.85-96, June 2004.
[9] N. Duffield, C. Lund, and M. Thorup, “Estimating Flow Distributions from Sampled Flow Statistics,” Proc. ACM SIGMETRICS '03, pp. 325-336, Aug. 2003.
[10] M. Finkelstein, H.G. Tucker, and J.A. Veeh, “Confidence Intervals for the Number of Unseen Types,” Statistics and Probability Letters, vol. 37, no. 4, pp. 423-430, Mar. 1998.
[11] N. Hohn and D. Veitch, “Inverting Sampled Traffic,” Proc. ACM SIGCOMM Internet Measurement Workshop (IMW '03), pp. 222-233, Oct. 2003.
[12] A. Kumar, J. Xu, O. Spatschek, and L. Li, “Space-Code Bloom Filter for Efficient Per-Flow Traffic Measurement,” Proc. IEEE INFOCOM '03, Aug. 2003.
[13] A. Kumar, M. Sung, J. Xu, and L. Wang, “Data Streaming Algorithms for Efficient and Accurate Estimation of Flow Size Distribution,” Proc. ACM SIGMETRICS '03, pp. 177-188, Aug. 2003.
[14] A. Lall, V. Sekhar, M. Ogihara, J. Xu, and H. Zhang, “Data Streaming Algorithms for Estimating Entropy of Network Traffic,” Proc. ACM SIGMETRICS '06, pp. 145-156, June 2006.
[15] T. Mori, M. Uchida, R. Kawahara, J. Pan, and S. Goto, “Identifying Elephant Flows through Periodically Sampled Packets,” Proc. ACM SIGCOMM Workshop Internet Measurement Workshop (IMW'04), pp. 115-120, 2004.
[16] E. Parzen, “On Estimation of a Probability Density Function and Mode,” The Annals of Math. Statistics, vol. 33, pp. 1065-1076, 1962.
[17] S. Ramabhadran and G. Varghese, “Efficient Implementation of a Statistics Counter Architecture,” Proc. ACM SIGMETRICS '03, pp.261-271, 2003.
[18] S. Raudys, “On the Effectiveness of Parzen Window Classifier,” Informatics, vol. 2, no. 3, pp. 435-454, 1991.
[19] J.G. Shantikumar and J. Buzacott, “On the Approximations to the Single-Server Queue,” Int'l J. Production Research, pp. 761-773, 1980.
[20] D. Shah, S. Iyer, B. Prabhakar, and N. McKeown, “Maintaining Statistics Counters in Router Line Cards,” IEEE Micro, vol. 22, no. 1, pp. 76-81, Jan. 2002.
[21] B.W. Silverman, Density Estimation for Statistics and Data Analysis. Chapman and Hall, 1986.
[22] K. Zheng, H. Che, Z. Wang, and B. Liu, “TCAM-Based Distributed Parallel Packet Classification Algorithm with Range-Matching Solution,” Proc. IEEE INFOCOM '05, pp. 293-303, Mar. 2005.
[23] Cisco NetFlow, http:/www.cisco.com, 2008.
[24] NoBL SRAMs and Bus Contention, Cypress, http:/www.cypress. com, July 2001.
[25] Netgear GSM 7224, Netgear, http://www.netgear.com/products/detailsGSM7224.php . 2008.
[26] Network Search Engine (NSE) Family, NetLogic MicroSystem, http://netlogicmicro.com/productsnse.html , 2008.
[27] Intel Pro/1000 MT Dual Port Server Adapter, Intel, http://www.intel.comnetwork/, 2008.
[28] NLANR AMP, http://pma.nlanr.netSpecial/, 2008.
[29] http:/www.tcpdump.org, 2008.
[30] http://www.freshports.org/nettcpdstat/, 2008.
[31] http://www.comlab.uni-rostock.de/research tools.html, 2008.
7 ms
(Ver 2.0)

Marketing Automation Platform Marketing Automation Tool