This Article 
 Bibliographic References 
 Add to: 
Engineering Dynamic Real-Time Distributed Systems: Architecture, System Description Language, and Middleware
January 2002 (vol. 28 no. 1)
pp. 30-57

This paper presents an architectural framework and algorithms for engineering dynamic real-time distributed systems using commercial off-the-shelf technologies. In the proposed architecture, a real-time system application is developed in a general-purpose programming language. Further, the architectural-level description of the system such as composition and interconnections of application software and hardware, and the operational requirements of the system such as timeliness and survivability are specified in a system description language. The specification of the system is automatically translated into an intermediate representation (IR) that models the system in a platform-independent manner. The IR is augmented with dynamic measurements of the system by a language runtime system to produce a dynamic system model. The dynamic model is used by resource management middleware strategies to perform resource management that achieves the timeliness and survivability requirements. The middleware techniques achieve the timeliness and survivability requirements through runtime monitoring and failure detection, diagnosis, and dynamic resource allocation. We present two classes of algorithms—predictive and availability-based—for performing resource allocation. To validate the viability of the approach, we use a real-time benchmark application that functionally approximates dynamic real-time command and control systems. The benchmark is specified in the system description language and the effectiveness of the architecture in achieving its design goals is examined through a set of experiments. The experimental characterizations illustrate that the middleware is able to achieve the desired timeliness requirements during a number of load situations. Furthermore, the results indicate that availability-based allocation algorithms perform resource allocation less frequently, whereas the predictive algorithms give a better steady state performance for the application.

[1] A. Atlas and A. Bestavros, Statistical Rate Monotonic Scheduling Proc. 19th IEEE Real-Time Systems Symp., pp. 123-132, 1998.
[2] R. Allen and D. Garlan, "A Formal Basis for Architectural Connection," ACM Trans. Software Eng. and Methodology, July 1997.
[3] H. Ben-Abdallah, I. Lee, and J.-Y. Choi, “A Graphical Language with Formal Semantics for the Specification and Analysis of Real-Time Systems,” Proc. IEEE Real-Time Systems Symp., pp. 276-286, 1995.
[4] International Standard ANSI/ISO/IEC-8652:1995, Ada 95 Reference Manual, Intermetrics, Inc., Jan. 1995.
[5] T.P. Baker, “Stack-Based Scheduling of Real-Time Processes,” J. Real-Time Systems, vol. 3, no. 1, pp. 67-100, 1991.
[6] T.E. Bihari and P. Gopinath, "Object-Oriented Real-Time Systems: Concepts and Examples," Computer, vol. 25, no. 12, pp. 25-32, Dec. 1992.
[7] S. Brandt, G. Nutt, T. Berk, and J. Mankovich, “A Dynamic Quality of Service Middleware Agent for Mediating Application Resource Usage,” Proc. IEEE Real-Time Systems Symp., pp. 307-317, Dec. 1998.
[8] R.K. Clark, “Scheduling Dependent Real-Time Activities,” PhD thesis, Carnegie Mellon Univ., CMU-CS-90-155, 1990.
[9] J.-Y. Choi, I. Lee, and H.-L. Xie, “The Specification and Schedulability Analysis of Real-Time Systems using ACSR,” Proc. IEEE Real-Time Systems Symp., pp. 266–275, Dec. 1995.
[10] S. Cheng, J. Stankovic, and K. Ramamritham, “Dynamic Scheduling of Groups of Tasks with Precedence Constraints in Distributed Hard Real-Time Systems,” Proc. IEEE Real-Time Systems Symp., 1986.
[11] F. Jahanian, R. Rajkumar, and S.C.V. Raju, “Runtime Monitoring of Timing Constraints in Distributed Real-Time Systems,” Real-Time Systems, vol. 7, no. 3, pp. 247–273, 1994.
[12] R. Gerber and S. Hong,“Semantics-based compiler transformations for enhancedschedulability,” Proc. of IEEE Real-Time Systems Symp., pp. 232-242.Los Alamitos, Calif.: IEEE CS Press, December 1993.
[13] M. Gergeleit, J. Kaiser, and H. Streich, “Checking Timing Constraints in Distributed Object-Oriented Programs,” Proc. Object-Oriented Real-Time Systems Workshop, Oct. 1995.
[14] N. Gehani and K. Ramamritham, “Real-Time Concurrent C: A Language for Programming Dynamic Real-Time Systems,” J. Real-Time Systems, vol. 3, no. 4, pp. 377–405, Dec. 1991.
[15] The U.S. Naval Surface Warfare Center, “High Performance Distributed Computing,” 1998. Available at .
[16] C.J. Hou and K.G. Shin, “Allocation of Periodic Task Modules with Precedence and Deadline Constraints in Distributed Real-Time Systems,” Proc. Real-Time Systems Symp., pp. 146-155, Dec. 1992.
[17] D. Hull, A. Shankar, K. Nahrstedt, and J.W.S. Liu, “An End-to-End QoS Model and Management Architecture,” Proc. IEEE Workshop Middleware for Distributed Real-Time Systems and Services, pp. 82–89, 1997.
[18] Y. Ishikawa, H. Tokuda, and C.W. Mercer, “An Object-Oriented Real-Time Programming Language,” Computer, vol. 25, no. 10, pp. 66–73, Oct. 1992.
[19] E.D. Jensen, “Adaptive Real-Time Distributed Computer Systems,” Real-Time Computing, Proc. NATO Advanced Study Institute, W.A. Halang and A.D. Stoyenko eds., Oct. 1992
[20] F. Jahanian and A. K.-L. Mok,“Safety analysis of timing properties in real-time systems,”IEEE Trans. Software Eng., vol. SE-12, pp. 890–904, Sept. 1986.
[21] B.C. Kao, “Scheduling in Distributed Soft Real-Time Systems with Autonomous Components,” PhD thesis, Princeton Univ., Nov. 1995.
[22] G. Koob, “Quorum,” Proc. Darpa ITO General PI Meeting, vol. A, pp. 59–87, Oct. 1996.
[23] H. Kopetz, A. Damm, C. Koza, M. Mulazzani, W. Schwabi, C. Senft, and R. Zainlinger, "Distributed Fault-Tolerant Real-Time Systems: The MARS Approach," IEEE Micro, pp. 25-58, Feb. 1989.
[24] R.B. Kieburtz and J.L. Hennessy, “Tomal—A High-Level Programming Language for Microprocessor Process Control Applications,” ACM SIGPLAN Notices, pp. 127–134, Apr. 1976.
[25] K.B. Kenny and K.J. Lin, “Building Flexible Real-Time Systems Using the Flex Language,” Computer, pp. 70–78, May 1991.
[26] T.W. Kuo and A.K. Mok, Load Adjustment in Adaptive Real-Time Systems Proc. IEEE Trans. Computers, vol. 16, no. 12, Dec. 1997.
[27] E. Kligerman and A. Stoyenko, "Real-Time Euclid: A Language for Reliable Real-Time Systems," IEEE Trans. Software Eng., vol. 12, no. 9, pp. 940-949, Sept. 1986.
[28] J.P. Lehoczky, Real-Time Queueing Theory Proc. 17th IEEE Real-Time Systems Symp. (RSS '96), pp. 186-195, Dec. 1996.
[29] I. Lee and V. Gehlot, “Language Constructs for Distributed Real-Time Systems,” Proc. IEEE Real-Time Systems Symp., Dec. 1985.
[30] C.L. Liu and J.W. Layland, “Scheduling Algorithms for Multiprogramming in a Hard Real-Time Environment,” J. ACM, vol. 20, no. 1, pp. 40-61, 1973.
[31] K.J. Lin and S. Natarajan, “Expressing and Maintaining Timing Constraints in Flex,” Proc. Ninth IEEE Real-Time Systems Symp., pp. 96–105, Dec. 1988.
[32] C. D. Locke, "Best-Effort Decision Making for Real-Time Scheduling," PhD thesis, Computer Science Dept., Carnegie Mellon Univ., 1986.
[33] J.P. Lehoczky and S. Ramos-Thuel, “An Optimal Algorithm for Scheduling Soft-Aperiodic Tasks in Fixed-Priority Preemptive Systems,” Proc. Real-Time Systems Symp., pp. 110-123, 1992.
[34] J.P. Lehoczky, L. Sha, and J.K. Strosnider, “Enhanced Aperiodic Responsiveness in Hard Real-time Environments,” Proc. IEEE Real-Time Systems Symp., 1987.
[35] T. Martin, “Real-Time Programming Language Pearl—Concept and Characteristics,” Proc. Second Int'l Computer Software and Applications Conf., pp. 301–306, 1978.
[36] J. Magee, N. Dulay, S. Eisenbach, and J. Kramer, “Specifying Distributed Software Architectures,” Proc. Fifth European Software Eng. Conf., Sept. 1995.
[37] D.L. Mills, Improved Algorithms for Synchronizing Computer Network Clocks IEEE/ACM Trans. Networking, vol. 3, pp. 245-254, June 1995.
[38] V. Nirkhe, S. Tripathi, and A. Agrawala, Language Support for the Maruti Real-Time System Proc. Real-Time Systems Symp., pp. 257-266, Dec. 1990.
[39] Defense Advanced Research Projects Agency (DARPA) ITO, “Quorum,” Available at projlist.html, Aug. 1997.
[40] B. Ravindran, “Modeling and Analysis of Complex, Dynamic Real-Time Systems,” PhD thesis, Dept. of Computer Science and Eng., Univ. of Texas at Arlington, Aug. 1998.
[41] I. Ripoll, A. Crespo, and A.G. Fornes, “An Optimal Algorithm for Scheduling Soft Aperiodic Tasks in Dynamic Priority Preemptive Systems,” IEEE Trans. Software Eng., vol. 23, no. 6, pp. 388–400, June 1997.
[42] R. Rajkumar, C. Lee, J. Lehoczky, and D. Siewiorek, A Resource Allocation Model for QoS Management Proc. 19th IEEE Real-Time Systems Symp. (RTSS), pp. 298-307, 1997.
[43] D.I. Rosu, K. Schwan, S. Yalamanchili, and R. Jha, “On Adaptive Resource Allocation for Complex Real-Time Applications,” Proc. 18th IEEE Real-Time Systems Symp., pp. 320–329, Dec. 1997.
[44] K. Ramamritham, J.A. Stankovic, and W. Zhao, “Distributed Scheduling of Tasks with Deadlines and Resource Requirements,” Trans. Computers, vol. 38, no. 8, Aug. 1989.
[45] S. Ramos-Thuel and J.P. Lehoczky, “On-Line Scheduling of Hard Deadline Aperiodic Tasks in Fixed-Priority Systems,” Proc. Real-Time Systems Symp., pp. 160-171, 1993.
[46] B. Ravindran, L.R. Welch, and B. Shirazi, “Resource Management Middleware for Dynamic, Dependable Real-Time Systems,” J. Real-Time Systems, vol. 20, no. 2, pp. 183-196, Mar. 2001.
[47] M. Spuri and G. Buttazzo, “Scheduling Aperiodic Tasks in Dynamic Priority Systems,” J. Real-Time Systems, vol. 10, pp. 179–210, Mar. 1996.
[48] K.G. Shin and C.-J. Hou, “Design and Evaluation of Effective Load Sharing in Distributed Real-Time Systems,” IEEE Trans. Software Eng., vol. 15, no. 7, pp. 875–889, July 1989.
[49] A.C. Shaw, "Reasoning About Time in Higher-Level Language Software," IEEE Trans. Software Eng., vol. 15, no. 7, pp. 875-889, 1989.
[50] K.G. Shin, "HARTS: A Distributed Real-Time Architecture," Computer, vol. 24, no. 5, pp. 25-34, May 1991.
[51] D.B. Stewart and P.K. Khosla, “Mechanisms for Timing Error Detection and Handling,” Comm. ACM, vol. 40, no. 1, Jan. 1997.
[52] L. Sha, M.H. Klein, and J.B. Goodenough, “Rate Monotonic Analysis for Real-Time Systems,” Scheduling and Resource Management, A.M. van Tilborg and G.M. Koob ed., pp. 129–156, 1991.
[53] J. Sun and J.W.S. Liu, “Bounding Completion Times of Jobs with Arbitrary Release Times and Variable Execution Times,” Proc. IEEE Real-Time Systems Symp., Dec. 1996.
[54] B. Sprunt,J. Lehoczky,, and L. Sha,“Exploiting unused periodic time for aperiodic service using the extened priority exchange algorithm,” Proc. Ninth Real-Time System Symp., pp. 251-258, 1988.
[55] S. Sommer and J. Potter, “Operating System Extensions for Dynamic Real-Time Applications,” Proc. IEEE Real-Time Systems Symp., pp. 45-50, Dec. 1996.
[56] J.A. Stankovic and K. Ramamritham, “The Spring Kernel: A New Paradigm for Real-Time Systems,” IEEE Software, pp. 62-72, May 1991.
[57] J.A. Stankovic, K. Ramamritham, and S. Cheng, “Evaluation of a Flexible Task Scheduling Algorithm for Distributed Hard Real-Time Systems,” IEEE Trans. Computers, vol. 34, no. 12, pp. 1130–1141, Dec. 1985.
[58] B. Sprunt, L. Sha, and J. Lehoczky, “Aperiodic Task Scheduling in Hard Real-time Systems,” J. Real-Time Systems, vol. 1, no. 1, pp. 27–60, 1989.
[59] J. Stankovic, A. Burns, K. Jeffay, M. Jones, G. Koob, I. Lee, J. Lehoczky, J. Liu, A. Mok, K. Ramamritham, J. Ready, L. Sha, and A. van Tilborg, “Strategic Directions in Real-Time and Embedded Systems,” ACM Computing Surveys, vol. 28, no. 4, pp. 751-763, Dec. 1996.
[60] T.-S. Tia et al., Probabilistic Performance Guarantee for Real-Time Tasks with Varying Computation Times Proc. IEEE EEE Real-Time Technology and Applications Symp. (RTAS), pp. 164-173, 1995.
[61] T.S. Tia, J.W.S. Liu, and M. Shankar, “Algorithms and Optimality of Scheduling Aperiodic Requests in Fixed-Priority Preemptive Systems,” J. Real-Time Systems, 1995.
[62] R.N. Taylor, N. Medvidovic, K.M. Anderson, E.J. Whitehead Jr., J.E. Robbins, K.A. Nies, P. Oreizy, and D.L. Dubrow, “A Component-Based Architectural Style for GUI Software,” IEEE Trans. Software Eng., vol. 22, no. 6, June 1996.
[63] J.P.C. Verhoosel, “Pre-Run-Time Scheduling of Distributed Real-Time Systems: Models and Algorithms,” PhD Thesis, Eindhoven Univ. of Technology, The Netherlands, Jan. 1995.
[64] L.R. Welch, “Large Grain, Dynamic Control System Architectures,” Proc. Joint Workshop Parallel and Distributed Real-Time Systems, pp. 22–26, Apr. 1997.
[65] L.R. Welch, B. Ravindran, R.D. Harrison, L. Madden, M.W. Masters, and W. Mills, “Challenges in Engineering Distributed Shipboard Control Systems,” Proc. Work-In-Progress Session, 17th IEEE Real-Time Systems Symp., pp. 19–22, Dec. 1996.
[66] L.R. Welch, B. Ravindran, B. Shirazi, and C. Bruggeman, “Specification and Modeling of Dynamic, Distributed Real-Time Systems,” Proc. IEEE Real-Time Systems Symp., pp. 72-81, Dec. 1998.
[67] L.R. Welch and B.A. Shirazi, “A Dynamic Real-Time Benchmark for Assessment of QoS and Resource Management Technology,” Proc. Fifth IEEE Real-Time Technology and Applications Symp., June 1999.
[68] L.R. Welch, A.D. Stoyenko, and T.J. Marlowe, “Response Time Prediction for Distributed Periodic Processes Specified in CaRT-Spec,” Control Eng. Practice, vol. 3, no. 5, pp. 651–664, May 1995.
[69] L.R. Welch, B. Shirazi, B. Ravindran, and F. Kamangar, “Instrumentation Support for Dynamic Resource Management of Real-Time, Distributed Control Systems,” Int'l J. Parallel and Distributed Systems and Networks, vol. 2, no. 3, pp. 105–117, 1999.
[70] J. Xu and D.L. Parnas, “Scheduling Processes with Release Times, Deadlines, Precedence and Exclusion Relations,” IEEE Trans. Software Eng., vol. 16, no. 3, pp. 360-369, Mar. 1990.

Index Terms:
process control systems, command and control, real-time systems, real-time resource management, timeliness, survivability, scalability, quality of service, middleware, system software
B. Ravindram, "Engineering Dynamic Real-Time Distributed Systems: Architecture, System Description Language, and Middleware," IEEE Transactions on Software Engineering, vol. 28, no. 1, pp. 30-57, Jan. 2002, doi:10.1109/32.979988
Usage of this product signifies your acceptance of the Terms of Use.