This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A New Hierarchical Data Cache Architecture for iSCSI Storage Server
April 2009 (vol. 58 no. 4)
pp. 433-447
Jun Wang, University of Central Florida, Orlando
Xiaoyu Yao, Microsoft, Seattle
Christopher Mitchell, University of Central Florida, Orlando
Peng Gu, University of Central Florida, Orlando
With the emergence of data-intensive applications, recent years have seen a fast-growing volume of I/O traffic propagated through the local I/O interconnect bus. This raises up a question for storage servers on how to resolve such a potential bottleneck. In this paper, we present a hierarchical Data Cache Architecture called DCA to effectively slash local interconnect traffic and thus boost the storage server performance. A popular iSCSI storage server architecture is chosen as an example. DCA is composed of a read cache in NIC called NIC cache and a read/write unified cache in host memory called Helper cache. The NIC cache services most portions of read requests without fetching data via the PCI bus, while the Helper cache 1) supplies some portions of read requests per partial NIC cache hit, 2) directs cache placement for NIC cache, and 3) absorbs most transient writes locally. We develop a novel State-Locality-Aware cache Placement algorithm called SLAP to improve the NIC cache hit ratio for mixed read and write workloads. To demonstrate the effectiveness of DCA, we develop a DCA prototype system and evaluate it with an open source iSCSI implementation under representative storage server workloads. Experimental results showed that DCA can boost iSCSI storage server throughput by up to 121 percent and reduce the PCI traffic by up to 74 percent compared with an iSCSI target without DCA.

[1] J. Satran, K. Meth, C. Sapuntzakis, and M. Chadalapaka, Internet Engineering Task Force, RFC 3720:Internet Small Computer System Interface (iSCSI), RFC 3720.
[2] B. Phillips, “Have Storage Area Networks Come of Age?” Computer, vol. 31, no. 7, pp. 10-12, July 1998.
[3] G. Gibson and R. Meter, “Network Attached Storage Architecture,” Comm. ACM, vol. 43, no. 11, pp. 37-45, Nov. 2000.
[4] Intel Developer Network for PCI Express$^{\ast}$ Architecture, http://www.intel.com/technology/pciexpress/ devnet/docsWhatisPCIExpress.pdf, 2008.
[5] C.J. Thompson, S. Hahn, and M. Oskin, “Using Modern Graphics Architectures for General-Purpose Computing: A Framework and Analysis,” Proc. 35th Ann. ACM/IEEE Int'l Symp. Microarchitecture (MICRO '02), pp. 306-317, 2002.
[6] Z. Fan, F. Qiu, A. Kaufman, and S. Yoakum-Stover, “GPU Cluster for High Performance Computing,” Proc. ACM/IEEE Conf. Supercomputing (SC '04), p. 47, 2004.
[7] W.C. Feng, J. Hurwitz, H.B. Newman, S. Ravot, R.L. Cottrell, O. Martin, F. Coccetti, C. Jin, D. Wei, and S. Low, “Optimizing 10-Gigabit Ethernet in Networks of Workstations, Clusters, and Grids: ACase Study,” Proc. ACM/IEEE Conf. Supercomputing (SC '03), 2003.
[8] H. Kim, V. Pai, and S. Rixner, “Increasing Web Server Throughput with Network Interface Data Caching,” Proc. 10th Int'l Conf. Architectural Support for Programming Languages and Operating Systems (ASPLOS '02), Oct. 2002.
[9] M. Zhang and Q. Yang, “BUCS—A Bottom-Up Cache Structure for Networked Storage Servers,” Proc. 33rd Int'l Conf. Parallel Processing (ICPP '04), pp. 310-317, Aug. 2004.
[10] T.M. Wong and J. Wilkes, “My Cache or Yours? Making Storage More Exclusive,” Proc. General Track of the Usenix Ann. Technical Conf., pp. 161-175, 2002.
[11] Z. Chen, Y. Zhou, and K. Li, “Eviction-Based Cache Placement for Storage Caches,” Proc. General Track of the Usenix Ann. Technical Conf., pp. 269-281, 2003.
[12] S. Jiang and X. Zhang, “ULC: A File Block Placement and Replacement Protocol to Effectively Exploit Hierarchical Locality in Multi-Level Buffer Caches,” Proc. 24th Int'l Conf. Distributed Computing Systems (ICDCS '04), pp. 168-177, Mar. 2004.
[13] Intel iSCSI Reference Implementation, http://sourceforge.net/projectsintel-iscsi /, 2008.
[14] G. Yadgar, M. Factor, and A. Schuster, “Karma: Know-It-All Replacement for a Multilevel Cache,” Proc. Fifth Usenix Conf. File and Storage Technologies (FAST '07), p. 25, 2007.
[15] UNH-iSCSI Project, http:/unh-iscsi.sourceforge.net/, 2008.
[16] Adaptec iSCSI Card7211c Data Sheet, http://graphics.adaptec.com/pdfsiscsi_7211_datasheet.pdf , 2008.
[17] Cello99 Trace, http://tesla.hpl.hp.compublic_software/, 2008.
[18] TPC-D, http://www.tpc.org/tpcddefault.asp, 2008.
[19] P. Radkov, L. Yin, P. Goyal, and P. Sarkar, “A Performance Comparison of NFS and iSCSI for IP-Networked Storage,” Proc. Third Usenix Conf. File and Storage Technologies (FAST '04), pp. 101-114, May 2004.
[20] G. Peng, S. Sharma, and T. Chiueh, “A Case for Network-Centric Buffer Cache Organization,” Proc. 11th IEEE Symp. High Performance Interconnects (Hot Interconnects '03), pp. 66-71, Aug. 2003.
[21] X. He, Q. Yang, and M. Zhang, “Introducing SCSI-to-IP Cache for Storage Area Networks,” Proc. 31st Int'l Conf. Parallel Processing (ICPP '02), pp. 203-210, Aug. 2002.
[22] X. He, Q. Yang, and M. Zhang, “A Caching Strategy to Improve iSCSI Performance,” Proc. 27th IEEE Ann. Conf. Local Computer Networks (LCN '02), Nov. 2002.

Index Terms:
NIC, cache, PCI, iSCSI, storage server.
Citation:
Jun Wang, Xiaoyu Yao, Christopher Mitchell, Peng Gu, "A New Hierarchical Data Cache Architecture for iSCSI Storage Server," IEEE Transactions on Computers, vol. 58, no. 4, pp. 433-447, April 2009, doi:10.1109/TC.2008.166
Usage of this product signifies your acceptance of the Terms of Use.