This Article 
 Bibliographic References 
 Add to: 
Structuring Communication Software for Quality-of-Service Guarantees
October 1997 (vol. 23 no. 10)
pp. 616-634

Abstract—A growing number of real-time applications require quality-of-service (QoS) guarantees from the underlying communication subsystem. The communication subsystem (host and network) must support real-time communication services to provide the required QoS of these applications. In this paper, we propose architectural mechanisms for structuring host communication software to provide QoS guarantees. In particular, we present and evaluate a QoS-sensitive communication subsystem architecture for end hosts that provides real-time communication support for generic network hardware. This architecture provides services for managing communication resources for guaranteed-QoS (real-time) connections, such as admission control, traffic enforcement, buffer management, and CPU and link scheduling. The design of the architecture is based on three key goals: maintenance of QoS-guarantees on a per-connection basis, overload protection between established connections, and fairness in delivered performance to best-effort traffic.

Using this architecture we implement real-time channels, a paradigm for real-time communication services in packet-switched networks. The proposed architecture features a process-per-channel model that associates a channel handler with each established channel. The model employed for handler execution is one of "cooperative" preemption, where an executing handler yields the CPU to a waiting higher-priority handler at well-defined preemption points. The architecture provides several configurable policies for protocol processing and overload protection. We present extensions to the admission control procedure for real-time channels to account for cooperative preemption and overlap between protocol processing and link transmission at a sending host. We evaluate the implementation to demonstrate the efficacy with which the architecture maintains QoS guarantees on outgoing traffic while adhering to the stated design goals. The evaluation also demonstrates the need for specific features and policies provided in the architecture. In subsequent work, we have refined this architecture and used it to realize a full-fledged guaranteed-QoS communication service that performs QoS-sensitive resource management for outgoing as well as incoming traffic.

[1] C.M. Aras, J.F. Kurose, D.S. Reeves, and H. Schulzrinne, “Real-Time Communication in Packet-Switched Networks,” Proc. IEEE, vol. 82, no. 1, pp. 122-139, Jan. 1994.
[2] H. Zhang, “Service Disciplines for Guaranteed Performance Service in Packet-Switching Networks,” Proc. IEEE, vol. 83, pp. 1374-1396, Oct. 1995.
[3] D. Ferrari and D.C. Verma,“A scheme for real-time channel establishment in wide-area networks, IEEE J. Selected Areas in Comm., vol. 8, no. 3, pp. 368-379, Apr. 1990.
[4] D.D. Kandlur, K.G. Shin, and D. Ferrari, “Real-Time Communication in Multi-Hop Networks,” IEEE Trans. Parallel and Distributed Systems, vol. 5, no. 10, pp. 1,044-1,056, Oct. 1994.
[5] A. Mehra, A. Indiresan, and K.G. Shin, “Resource Management for Real-Time Communication: Making Theory Meet Practice,” Proc. IEEE Real-Time Technology and Applications Symp., pp. 130–138, 1996.
[6] N.C. Hutchinson and L.L. Peterson, “The x-Kernel: An Architecture for Implementing Network Protocols,” IEEE Trans. Software Eng., vol. 17, no. 1, pp. 64-76, Jan. 1991.
[7] A. Mehra, A. Indiresan, and K. Shin, "Resource Management for Real-Time Communication: Making Theory Meet Practice," in submission to IEEE Trans. Networking, June 1997.
[8] K.K. Ramakrishnan, "Performance Considerations in Designing Network Interfaces," IEEE J. Selected Areas in Comm., vol. 11, no. 2, pp. 203-219, Feb. 1993.
[9] ARMADA: A Real-Time Middleware Architecture for Distributed Applications. pub/papers/compgeohttp://www. cs. utexas. edu/ users/cpg/TempEst/doc/ / /.
[10] A. Mehra, A. Shaikh, T. Abdelzaher, Z. Wang, and K. Shin, "Realizing Guaranteed-QoS Communication Services on a Micro-Kernel Operating System, Aug. 1997.
[11] C.W. Mercer, S. Savage, and H. Tokuda, "Processor Capacity Reserves for Multimedia Operating Systems," Proc.. IEEE Int'l Conf. Multimedia Computng and Systems, May 1994.
[12] C. Lee, K. Yoshida, C. Mercer, and R. Rajkumar, "Predictable Communication Protocol Processing in Real-Time Mach," Proc.. Second Real-Time Technology and Applications Symp., June 1996.
[13] D.D. Clark and D.L. Tennenhouse, "Architectural Considerations for a New Generation of Protocols," Computer Comm. Rev., vol. 20, no. 4, Sept. 1990.
[14] C. Maeda and B.N. Bershad, "Protocol Service Decomposition for High-Performance Networking," Proc. 14th ACM Symp. Operating Systems Principles, ACM, 1993, pp. 244-255.
[15] C. Thekkath, T. Nguyen, E. Moy, and E. Lazoswka, "Implementing Network Protocols at User Level," IEEE/ACM Trans. Networking, vol. 1, no. 15, Oct. 1993.
[16] P. Druschel, L.L. Peterson, and B.S. Davie, "Experiences with a High-Speed Network Adaptor: A Software Perspective," Proc. SIGCOMM 94, ACM Press, New York, 1994, pp. 2-13.
[17] C.A. Waldspurger, “Lottery and Stride Scheduling: Flexible Proportional-Share Resource Management,” PhD thesis, Technical Report, MIT/LCS/TR-667, Laboratory for Computer Science,Massachusetts Inst. of Tech nology, Sept. 1995.
[18] I. Stoica, H. Abdel-Wahab, K. Jeffay, S.K. Baruah, J.E. Gehrke, and C.G. Plaxton, “A Proportional Share Resource Allocation Algorithm for Real-Time, Time-Shared Systems,” Proc. IEEE Real-Time Systems Symp., pp. 288-299, 1996.
[19] P. Goyal, X. Guo, and H. Vin, “A Hierarchical CPU Scheduler for Multimedia Operating Systems,” Proc. Second Usenix Symp. Operating System Design and Implementation, Oct. 1996.
[20] A. Mehra, Z. Wang, and K. Shin, "Self-Parameterizing Protocol Stacks for Guaranteed Quality of Service," Aug. 1997.
[21] R.L. Cruz, "A Calculus for Network Delay and a Note on Topologies of Interconnection Networks," PhD thesis, Univ. of Illinois at Urbana-Champaign, July 1995. available as Technical Report UILU-Eng-87-2246.
[22] D.P. Anderson, R. Wahbe, S. Tzou, R. Govindan, and M. Andrews, “Support for Continuous Media in the Dash System,” Proc. Int'l Conf. Distributed Computing Systems 10, pp. 54-61, May 1990.
[23] C.L. Liu and J.W. Layland, “Scheduling Algorithms for Multiprogramming in a Hard Real-Time Environment,” J. ACM, vol. 20, no. 1, pp. 40-61, 1973.
[24] D.C. Schmidt and T. Suda, "Transport System Architecture Service for High-Performance Communications Systems," IEEE J. Selected Areas in Comm., vol. 11, no. 4, pp. 489-506, May 1993.
[25] J. Liebeherr, D.E. Wrege, and D. Ferrari, "Exact Admission Control for Networks with Bounded Delay Services," IEEE/ACM Trans. Networking, vol. 4, pp. 885-901, Dec. 1996.
[26] A. Indiresan, A. Mehra, and K. Shin, "Design Tradeoffs in Implementing Real-Time Channels on Bus-Based Multiprocessor Hosts," Technical Report CSE-TR-238-95, Univ. of Michigan, Apr. 1995.
[27] T.-Y. Huang, J.W.-S. Liu, and D. Hull, “A Method for Bounding the Effect of DMA I/O Interference on Program Execution Time,” Proc. 17th Real-Time Systems Symp., pp. 275–285, Dec. 1996.
[28] M. Bjorkman and P. Gunningberg, "Locking Effects in Multiprocessor Implementation of Protocols," Proc. SIGCOMM, pp. 74-83,Ithaca, N.Y., 1993.
[29] E.M. Nahum, D.J. Yates, J.F. Kurose, and D. Towsley, "Performance Issues in Parallelized Network Protocols," Proc.. USENIX Symp. Operating Systems Design and Implementation, pp. 125-137, Nov. 1994.
[30] J.C. Mogul and A. Borg, “The Effect of Context Switches on Cache Performance,” Proc. Fourth Int'l Conf. Architectural Support for Programming Languages and Operating Systems, pp. 75-84, Santa Clara, Calif., Apr. 1991.
[31] J. Liedtke, H. Härtig, and M. Hohmuth, “OS-Controlled Cache Predictability for Real-Time Systems,” Proc. Third Real-Time Technology and Applications Symp., June 1997.
[32] C.-G. Lee, J. Hahn, Y.-M. Seo, S.L. Min, R. Ha, S. Hong, C.Y. Park, M. Lee, and C.S. Kim, “Analysis of Cache-related Preemption Delay in Fixed-Priority Preemptive Scheduling,” Proc. 17th Real-Time Systems Symp., pp. 264–274, Dec. 1996.
[33] R. Gopalakrishnan and G.M. Parulkar, "Bringing Real-Time Scheduling Theory and Practice Closer for Multimedia Computing," Proc.. ACM SIGMETRICS, pp. 1-12, May 1996.
[34] A. Mehra, A. Indiresan, and K.G. Shin, “Structuring Communication for Quality of Service Guarantees,” Proc. IEEE Real-Time Systems Symp., pp. 144-154, Dec. 1996.
[35] A.T. Campbell, C. Aurrecoechea, and L. Hauw, "A Review of QoS Architectures," Multimedia Systems J., 1996.
[36] D.D. Clark,S. Shenker,, and L. Zhang,“Supporting real-time applications inan integrated services packet network: Architecture and mechanism,” Proc. ACM SIGCOMM, pp. 14-26, 1992.
[37] R. Braden, D. Clark, and S. Shenker, "Integrated Services in the Internet Architecture: An Overview," Request for Comments RFC 1963, Xerox PARC, July 1994.
[38] S. Shenker, C. Partridge, and R. Guerin, "Specification of Guaranteed Quality of Service," Request for Comments RFC 2212, Sept. 1997.
[39] S. Jamin, P. Danzig, S. Shenker, and L. Zhang, “A Measurement-Based Admission Control Algorithm for Integrated Services Packet Networks,” Proc. SIGCOMM Symp. Comm. Architectures and Protocols, 1995.
[40] J. Wroclawski, "Specification of Controlled-Load Network Element Service," Request for Comments RFC 2211, Sept. 1997.
[41] S. Shenker, D. Clark, and L. Zhang, "A Scheduling Service Model and a Scheduling Architecture for an Integrated Services Packet Network," Working Paper, Xerox PARC, Aug. 1993.
[42] S. Floyd and V. Jacobson, “Link-Sharing and Resource Management Models for Packet Networks,” IEEE Trans. Networking, vol. 3, no. 4, pp. 365-386, Aug. 1995.
[43] L. Zhang, S. Deering, D. Estrin, S. Shenker, and D. Zappala, "RSVP: A New Resource Reservation Protocol," IEEE Network, vol. 7, no. 5, pp. 8-18, Sept. 1993.
[44] L. Delgrossi and L. Berger, "Internet Stream Protocol Version 2(ST-2) protocol specification—version ST2+," Request for Comments RFC 1819, ST2 Working Group, Aug. 1995.
[45] M. Borden, E. Crawley, B. Davie, and S. Batsell, "Integration of Real-Time Services in an IP-ATM Network Architecture," Request for Comments RFC 1821, Bay Networks, Bellcore, NRL, Aug. 1995.
[46] M. Perez, F. Liaw, A. Mankin, E. Hoffman, D. Grossman, and A. Malis, "ATM Signaling Support for IP Over ATM," Request for Comments RFC 1755, ISI, Fore, MotoralCodex, Ascom Timeplex, Feb. 1995.
[47] A. Banerjea, D. Ferrari, B. Mah, M. Moran, D. Verma, and H. Zhang, “The Tenet Real-Time Protocol Suite: Design, Implementation, and Experiences,” IEEE/ACM Trans. Networking, vol. 4, no. 1, pp. 1-10, Feb. 1996.
[48] K. Nahrstedt and J. Smith, “Design, Implementation, and Experiences with the OMEGA end-Point Architecture,” IEEE J. Selected Areas in Comm., Sept. 1996.
[49] K. Nahrstedt and J. Smith, “The QoS Broker,” IEEE Multimedia, vol. 2, no. 1, pp. 53-67, 1995.
[50] A. Cambell, G. Coulson, and D. Hutchison, “A Quality of Service Architecture,” ACM Computer Comm. Review, Apr. 1994.
[51] A.T. Campbell and G. Coulson, "QoS Adaptive Transports: Delivering Scalable Media to the Desktop," IEEE Network, pp. 18-27, Mar./Apr. 1997.
[52] T. Barzilai, D. Kandlur, A. Mehra, D. Saha, and S. Wise, "Design and Implementation of an RSVP-Based Quality of Service Architecture for Integrated Services Internet," Proc. Int'l Conf. Distributed Computing Systems, May 1997.
[53] R. Ahuja, S. Keshav, and H. Saran, "Design, Implementation, and Performance of a Native Mode ATM Transport Layer," Proc. IEEE INFOCOM, pp. 206-214, Mar. 1996.
[54] G. Coulson, A. Campbell, P. Robin, G.S. Blair, M. Papathomous, and D. Shepherd, "The Design of a QoS-Controlled ATM-Based Communications System in Chorus," IEEE J. Seleted Areas in Comm., vol. 13, no. 4, pp. 686-699, May 1995.
[55] R. Gopalakrishnan and G.M. Parulkar, "A Real-Time Upcall Facility for Protocol Processing with QoS Guarantees," ACM Symp. Operating Systems Principles, (Poster Session), ACM Press, New York, 1995, p. 231; .
[56] D.K.Y. Yau and S.S. Lam, "An Architecture Towards Efficient OS Suport for Distributed Multimedia," Proc. Multimedia Computing and Networking (MMCN '96), Jan. 1996.
[57] D. Mosberger, L.L. Peterson, P.G. Bridges, and S. O'Malley, "Analysis of Techniques to Improve Protocol Processing Latency," Proc. ACM SIGCOMM, pp. 73-84, Oct. 1996.
[58] T. Blackwell, "Speeding Up Protocols for Small Messages," Proc ACM SIGCOMM, pp. 85-95, Oct. 1996.
[59] R. van Renesse, “Masking the Overhead of Protocol Layering,” Proc. ACM SIGCOMM '96, pp. 96-104, Aug. 1996.
[60] A. Edwards, G. Watson, J. Lumley, D. Banks, C. Clamvokis, and C. Dalton, "User-Space Protocols Deliver High Performance to Applications on a Low-Cost Gb/s LAN, Proc. ACM SIGCOMM, pp. 14-24,London, Aug. 1994.
[61] T. von Eicken et al., "U-Net: A User-Level Network Interface for Parallel and Distributed Computing," Proc. 15th ACM Symp. OS Principles, ACM Press, New York, 1995, pp. 40-53.
[62] P.A. Steenkiste, "A Systematic Approach to Host Interface Design for High-Speed Networks," Computer, vol. 26, no. 3, pp. 47-57, Mar. 1994.
[63] P. Druschel, M.B. Abbott, M. Pagels, and L.L. Peterson, "Network Subsystem Design: A Case for an Integrated Data Path, IEEE Network, pp. 8-17, July 1993.
[64] M. Uyhara, B.N. Bershad, C. Maeda, and J.E.B. Moss, "Efficient Packet Demultiplexing for Multiple Endpoints and Large Messages," Proc. Winter USENIX, 1994.
[65] M.L. Bailey, B. Gopal, M.A. Pagels, L.L. Peterson, and P. Sarkar, "PATHFINDER: A Pattern-Based Packet Classifier," Proc ACM SIGCOMM, pp. 115-123,London, Aug. 1994.
[66] D.R. Engler and M.F. Kaashoek, "DPF: Fast, Flexible Message Demultiplexing Using Dynamic Code Generation," Proc. ACM SIGCOMM '96 Conf. Computer Comm. Review, pp. 53-59, Stanford Univ., Stanford, Calif., Aug. 1996.
[67] J. Mogul and K.K. Ramakrishnan, "Eliminating Receive Livelock in an Interrupt-Driven Kernel" Winter USENIX Conf. Jan. 1996.
[68] M.Ajmone Marsan,S. Donatelli,F. Neri,, and U. Rubino,“On the construction of abstract GSPNs: Anexercise in modeling,” Proc. Fourth Int’l Workshop Petri Nets and Performance Models, pp. 2-17,Melbourne, Australia, Dec.2-5, 1991.
[69] D. Mosberger and L. Peterson, "Making Paths Explicit in the Scout Operating System," Proc. Second Symp. Operating System Design and Implementation, 1996, pp. 153-167.
[70] F. Travostino, E. Menze, and F. Reynolds, “Paths: Programming with System Resources in Support of Real-Time Distributed Applications,” Proc. IEEE Workshop Object-Oriented Real-Time Dependable Systems, Feb. 1996.
[71] D.I. Katcher, H. Arakawa, and J.K. Strosnider, “Engineering and Analysis of Fixed Priority Schedulers,” IEEE Trans. Software Eng., vol. 19, no. 9, pp. 920–934, Sept. 1993.
[72] A. Burns, K. Tindell, and A. Wellings, “Effective Analysis for Engineering Real-Time Fixed Priority Schedulers,” IEEE Trans. Software Eng., vol. 21, no. 5, pp. 475-480, May 1995.
[73] K.A. Kettler, D.I. Katcher, and J.K. Strosnider, "A Modeling Methdology for Real-Time/Multimedia Operating Systems," Proc. Real-Time Technology and Applications Symp., pp. 15-26, May 1995.
[74] C.W. Mercer and H. Tokuda, "Preemptibility in Real-Time Operating Systems," Proc. Real-Time Systems Symp., Dec. 1992.
[75] D.P. Anderson, L. Delgrossi, and R.G. Herrtwicyh, "Structure and Scheduling in Real-Time Protocol Implementations," Technical Report TR-90-021, Int'l Computer Science Inst., Berkeley, Calif., June 1990.
[76] R. Govindan and D.P. Anderson, “Scheduling and IPC Mechanisms for Continuous Media,” Proc. 13th ACM Symp. Operating Systems Principles, pp. 68-80, 1991.
[77] H. Kaneko, J.A. Stankovic, S. Sen, and K. Ramamritham, "Integrated Scheduling of Multimedia and Hard Real-Time Tasks," Proc. 17th Real-Time Systems Symp. pp. 206-217, Dec. 1996.
[78] O. Hagsand and P. Sjodin, "Workstation Support for Real-Time Multimedia Communication," Winter USENIX Conf. second edition, pp. 133-142, Jan. 1994.
[79] C. Vogt, R.G. Herrtwich, and R. Nagarajan, "HeiRAT: The Heidelberg Resource Administration Technique Design Philosophy and Goals," Research Report 43.9213, IBM Research Division, IBM European Netrworking Center, Heidelberg, Germany, 1992.
[80] A. Indiresan, A. Mehra, and K. Shin, "The END: An Emulated Network Device for Evaluating Adapter Design," Proc. Third Int'l Workshop on Performability Modeling of Computer and Communication Systems (PMCCS3), pp. 90-94, Sept. 1996.
[81] A. Mehra and K. Shin, "QoS-Sensitive Protocol Processing in Shared-Memory Multiprocessor Multimedia Servers," Proc. Third IEEE Workshop on Architecture and Implementation of High-Performance Communication Subsystems, pp. 163-169, Aug. 1995.

Index Terms:
Real-time communication, traffic enforcement, QoS-sensitive resource management, CPU, and link scheduling.
Ashish Mehra, Atri Indiresan, Kang G. Shin, "Structuring Communication Software for Quality-of-Service Guarantees," IEEE Transactions on Software Engineering, vol. 23, no. 10, pp. 616-634, Oct. 1997, doi:10.1109/32.637145
Usage of this product signifies your acceptance of the Terms of Use.