This Article 
 Bibliographic References 
 Add to: 
A Transport-Friendly NIC for Multicore/Multiprocessor Systems
April 2012 (vol. 23 no. 4)
pp. 607-615
Wenji Wu, Fermilab, Batavia
Phil DeMar, Fermilab, Batavia
Matt Crawford, Fermilab, Batavia
Receive side scaling (RSS) is an NIC technology that provides the benefits of parallel receive processing in multiprocessing environments. However, RSS lacks a critical data steering mechanism that would automatically steer incoming network data to the same core on which its application thread resides. This absence causes inefficient cache usage if an application thread is not running on the core on which RSS has scheduled the received traffic to be processed and results in degraded performance. To remedy the RSS limitation, Intel's Ethernet Flow Director technology has been introduced. However, our analysis shows that Flow Director can cause significant packet reordering. Packet reordering causes various negative impacts in high-speed networks. We propose an NIC data steering mechanism to remedy the RSS and Flow Director limitations. This data steering mechanism is mainly targeted at TCP. We term an NIC with such a data steering mechanism “A Transport-Friendly NIC” (A-TFN). Experimental results have proven the effectiveness of A-TFN in accelerating TCP/IP performance.

[1] P. Willmann et al., "An Evaluation of Network Stack Parallelization Strategies in Modern Operating Systems," Proc. USENIX Ann. Technical Conf., pp. 91-96, 2006.
[2] J.D. Salehi, J.F. Kurose, and D. Towsley, "The Effectiveness of Affinity-Based Scheduling in Multiprocessor Network Protocol Processing (Extended Version)," IEEE/ACM Trans. Networking, vol. 4, no. 4, pp. 516-530, Aug. 1996.
[3] A. Foong et al., "An In-Depth Analysis of the Impact of Processor Affinity on Network Performance," Proc. IEEE Int'l Conf. Networks, 2004.
[4] J. Hye-Churn et al., "MiAMI: Multi-Core Aware Processor Affinity for TCP/IP over Multiple Network Interfaces," Proc. IEEE Symp. High Performance Interconnects, 2009.
[5] R. Huggahalli et al., "Direct Cache Access for High Bandwidth Network I/O," Proc. 32nd Ann. Int'l Symp. Computer Architecture, 2005.
[6] Microsoft Corporation, Receive-Side Scaling Enhancements in Windows Server 2008, download/ a/d/f/adf1347d-08dc-41a4-9084-623b1194d4b2 RSS Server2008.docx, Nov. 2008.
[7] Intel Corporation, Intel 82599 10gbe Controller Datasheet, datashts82599_ datasheet.pdf , Nov. 2010.
[8] W. Wu et al., "Sorting Reordered Packets with Interrupt Coalescing," Computer Networks, vol. 53, no. 15, pp. 2646-2662, 2009.
[9] M. Laor and L. Gendel, "The Effect of Packet Reordering in a Backbone Link on Application Throughput," IEEE Network, vol. 16, no. 5, pp. 28-36, Sept./Oct. 2002.
[10] P. McManus Performance Tradeoffs of TCP Selective Acknowledgement, libraryl-tcp-sack/, 2011.
[11] M.E. Russinovich et al., Microsoft Windows Internals, fourth ed. Microsoft Press, 2004.
[12] W. Wu et al., "The Performance Analysis of Linux Networking - Packet Receiving," Computer Comm., vol. 30, pp. 1044-1057, 2007.
[13] W. Wu and M. Crawford, "Potential Performance Bottleneck in Linux TCP," Int'l J. Comm. Systems, vol. 20, no. 11, pp. 1263-1283, 2007.
[14] Ixgbe Device Driver Readme, README.txt, Nov. 2010.
[15] NGINX Website, http:/, 2011.
[16] LIGHTTPD Website, http:/, 2011.
[17] T. Brecht et al., "Evaluating Network Processing Efficiency with Processor Partitioning and Asynchronous I/O," Proc. ACM SIGOPS/EuroSys European Conf. Computer Systems (EuroSys), 2006.
[18] G. Regnier et al., "ETA: Experience with an Intel Xeon Processor as a Packet Processing Engine," IEEE Micro, vol. 24, no. 1, pp. 24-31, Jan./Feb. 2004.
[19] Intel Corporation "Intel VMDq Technology," 2008.
[20], 2011.
[21], 2011.
[22], 2011.

Index Terms:
Network interface card, protocol stack, multicore, high performance networking, TCP/IP.
Wenji Wu, Phil DeMar, Matt Crawford, "A Transport-Friendly NIC for Multicore/Multiprocessor Systems," IEEE Transactions on Parallel and Distributed Systems, vol. 23, no. 4, pp. 607-615, April 2012, doi:10.1109/TPDS.2011.195
Usage of this product signifies your acceptance of the Terms of Use.