This Article 
 Bibliographic References 
 Add to: 
PRESS: A Clustered Server Based on User-Level Communication
May 2005 (vol. 16 no. 5)
pp. 385-395

Abstract—In this paper, we propose and evaluate a cluster-based network server called PRESS. The server relies on locality-conscious request distribution and a standard for user-level communication to achieve high performance and portability. We evaluate PRESS by first isolating the performance benefits of three key features of user-level communication: low processor overhead, remote memory accesses, and zero-copy transfers. Next, we compare PRESS to servers that involve less intercluster communication, but are not as easily portable. Our results for an 8-node server cluster and five WWW traces demonstrate that user-level communication can improve performance by as much as 52 percent compared to a kernel-level protocol. Low processor overhead, remote memory writes, and zero-copy all make nontrivial contributions toward this overall gain. Our results also show that portability in PRESS causes no throughput degradation when we exploit user-level communication extensively.

[1] S. Araki, A. Bilas, C. Dubnicki, J. Edler, K. Konishi, and J. Philbin, “User-Space Communication: A Quantitative Study,” Proc. Supercomputing, Nov. 1998.
[2] M. Aron, P. Druschel, and W. Zwaenepoel, “Efficient Support for P-HTTP in Cluster-Based Web Servers,” Proc. USENIX Ann. Technical Conf., June 1999.
[3] M. Aron, D. Sanders, P. Druschel, and W. Zwaenepoel, “Scalable Content-Aware Request Distribution in Cluster-Based Network Servers,” Proc. USENIX Ann. Technical Conf., June 2000.
[4] M. Banikazemi, V. Moorthy, L. Herger, D.K. Panda, and B. Abali, “Efficient Virtual Interface Architecture Support for the IBM SP Switch-Connected NT Clusters,” Proc. 14th Int'l Parallel and Distributed Processing Symp., 2000.
[5] T. Berners-Lee, R. Fielding, and H. Frystyk, “RFC 1945: HyperText Transfer Protocol— HTTP/1.0,” technical report, HTTP Working Group, May 1996.
[6] R. Bianchini and E.V. Carrera, “Analytical and Experimental Evaluation of Cluster-Based WWW Servers,” World Wide Web J., vol. 3, no. 4, pp. 215-229, Dec. 2000.
[7] M. Blumrich, K. Li, R. Alpert, C. Dubnicki, E. Felten, and J. Sandberg, “Virtual Memory Mapped Network Interface for the SHRIMP Multicomputer,” Proc. 21st Ann. Int'l Symp. Computer Architecture, pp. 142-153, Apr. 1994.
[8] N.J. Boden, D. Cohen, R.E. Felderman, A.E. Kulawik, C.L. Seitz, J.N. Seizovic, and W.-K. Su, “Myrinet: A Gigabit per Second Local Area Network,” IEEE Micro, vol. 15, no. 1, pp. 29-36, Feb. 1995.
[9] P. Bozeman and B. Saphir, “A Modular High Performance Implementation of the Virtual Interface Architecture,” Proc. Second Extreme Linux Workshop, June 1999.
[10] P. Buonadonna, A. Geweke, and D. Culler, “An Implementation and Analysis of the Virtual Interface Architecture,” Proc. Supercomputing, Nov. 1998.
[11] V. Cardellini, E. Casalicchio, M. Colajanni, and P. Yu, “The State of the Art in Locality Distributed Web-Server Systems,” Technical Report RC22209, IBM, Oct. 2001.
[12] E.V. Carrera and R. Bianchini, “Efficiency vs. Portability in Cluster-Based Network Servers,” Proc. Eighth Symp. Principles and Practice of Parallel Programming, pp. 113-122, June 2001.
[13] E.V. Carrera, S. Rao, L. Iftode, and R. Bianchini, “User-Level Communication in Cluster-Based Servers,” Proc. Eighth Int'l Symp. High-Performance Computer Architecture, pp. 275-286, Feb. 2002.
[14] Compaq Corp., Intel Corp., and Microsoft Corp., Virtual Interface Architecture Specification, Version 1.0, 1997.
[15] C. Dubnicki, L. Iftode, E. Felten, and K. Li, “Software Support for Virtual Memory-Mapped Communication,” Proc. 10th Int'l Parallel Processing Symp., Apr. 1996.
[16] D. Dunning, G. Regnier, G. McAlpine, D. Cameron, B. Shubert, F. Berry, A.M. Merritt, E. Gronke, and C. Dodd, “The Virtual Interface Architecture,” IEEE Micro, vol. 18, no. 2, pp. 66-76, Mar. 1998.
[17] Emulex, cLAN Cluster Switch, 2001.
[18] R. Fielding, J. Gettys, J. Mogul, H. Frystyk, L. Masinter, P. Leach, and T. Berners-Lee, “RFC 2616: HyperText Transfer Protocol— HTTP/1. 1,” technical report, HTTP Working Group, June 1999.
[19] V. Holmedahl, B. Smith, and T. Yang, “Cooperative Caching of Dynamic Content on a Distributed Web Server,” Proc. Seventh IEEE Int'l Symp. High Performance Distributed Computing, pp. 243-250, July 1998.
[20] J. Liu, J. Wu, S. Kinis, P. Wyckoff, and D.K. Panda, “High Performance RDMA-Based MPI Implementation over InfiniBand,” Proc. 17th Ann. ACM Int'l Conf. on Supercomputing, June 2003.
[21] K. Magoutis, S. Addetia, A. Fedorova, M. Seltzer, J.S. Chase, A. Gallatin, R. Kisley, R. Wickremesinghe, and E. Gabber, “Structure and Performance of the Direct Access File System,” Proc. USENIX Ann. Technical Conf., June 2002.
[22] R. Martin, A. Vahdat, D. Culler, and T. Anderson, “Effects of Communication Latency, Overhead, and Bandwidth in a Cluster Architecture,” Proc. 24th Int'l Symp. Computer Architecture, June 1997.
[23] K. Nagaraja, N. Krishnan, R. Bianchini, R.P. Martin, and T.D. Nguyen, “Evaluating the Impact of Communication Architecture on the Performability of Cluster-Based Services,” Proc. Ninth Symp. High Performance Computer Architecture (HPCA-9), Feb. 2003.
[24] Foundry Networks, “ServerIron,” http:/www.foundrynetworks. com, May 2003.
[25] V.S. Pai, M. Aron, G. Banga, M. Svendsen, P. Druschel, W. Zwaenepoel, and E. Nahum, “Locality-Aware Request Distribution in Cluster-Based Network Servers,” Proc. Eighth ACM Conf. Architectural Support for Programming Languages and Operating Systems, pp. 205-216, Oct. 1998.
[26] V.S. Pai, P. Druschel, and W. Zwaenepoel, “Flash: An Efficient and Portable Web Server,” Proc. USENIX Ann. Technical Conf., June 1999.
[27] S. Pakin, M. Karamcheti, and A. Chien, “Fast Messages (FM): Efficient, Portable Communication for Workstation Clusters and Massively-Parallel Processors,” IEEE Parallel and Distributed Technology, vol. 5, no. 2, pp. 60-73, Apr. 1997.
[28] M. Rangarajan and L. Iftode, “Software Distributed Shared Memory over Virtual Interface Architecture: Implementation and Performance,” Proc. Third Extreme Linux Workshop, 2000.
[29] Y. Saito, B.N. Bershad, and H.M. Levy, “Manageability, Availability and Performance in Porcupine: A Highly Scalable, Cluster-Based Mail Service,” Proc. 17th ACM Symp. Operating Systems Principles, Dec. 1999.
[30] E. Speight, H. Abdel-Shafi, and J.K. Bennett, “Realizing the Performance Potential of the Virtual Interface Architecture,” Proc. Int'l Conf. Supercomputing, pp. 184-192, June 1999.
[31] R. Stets, S. Dwarkadas, L. Kontothanassis, U. Rencuzogullari, and M. Scott, “The Effect of Network Total Order, Broadcast, and Remote-Write Capability on Network-Based Shared Memory Computing,” Proc. Sixth Int'l Symp. High-Performance Computer Architecture, Jan. 2000.
[32] W. Tang, L. Cherkasova, L. Russell, and M. Mutka, “Modular TCP Handoff Design in Streams Based TCP/IP Implementation,” Proc. First Int'l Conf. Networking, pp. 71-80, July 2001.
[33] H. Tesuka, A. Hori, and Y. Ishikawa, “PM: A High-Performance Communication Library for Multi-User Parallel Environments,” Technical Report TR-96015, Real World Computing Parnership, Nov. 1996.
[34] The InfiniBand Trade Assoc., InfiniBand 1.0 Specifications, http:/, Oct. 2000.
[35] T. von Eicken, A. Basu, V. Buch, and W. Vogels, “U-Net: A User-Level Network Interface for Parallel and Distributed Computing,” Proc. 15th ACM Symp. Operating Systems Principles, pp. 40-53, Dec. 1995.
[36] K.G. Yocum, J.S. Chase, A.J. Gallatin, and A.R. Lebeck, “Cut-Through Delivery in Trapeze: An Exercise in Low Latency Messaging,” Proc. IEEE Symp. High-Performance Distributed Computing, Aug. 1997.
[37] K.H. Yum, E.J. Kim, and C.R. Das, “QoS Provisioning in Clusters: An Investigation of Router and NIC Design,” Proc. 28th Ann. Int'l Symp. Computer Architecture, June 2001.
[38] Y. Zhou, A. Bilas, S. Jagannathan, C. Dubnicki, J.F. Philbin, and K. Li, “Experiences with VI Communication for Database Storage,” Proc. 29th Int'l Symp. Computer Architecture, May 2002.
[39] H. Zhu, B. Smith, and T. Yang, “Scheduling Optimization for Resource-Intensive Web Requests on Server Clusters,” Proc. 11th Ann. ACM Symp. Parallel Algorithms and Architectures, pp. 13-22, June 1999.

Index Terms:
Servers, communication architecture, performance.
Enrique V. Carrera, Ricardo Bianchini, "PRESS: A Clustered Server Based on User-Level Communication," IEEE Transactions on Parallel and Distributed Systems, vol. 16, no. 5, pp. 385-395, May 2005, doi:10.1109/TPDS.2005.60
Usage of this product signifies your acceptance of the Terms of Use.