This Article 
 Bibliographic References 
 Add to: 
Requirements-Based Monitors for Real-Time Systems
February 2002 (vol. 28 no. 2)
pp. 146-158

Before designing safety- or mission-critical real-time systems, a specification of the required behavior of the system should be produced and reviewed by domain experts. After the system has been implemented, it should be thoroughly tested to ensure that it behaves correctly. This is best done using a monitor, a system that observes the behavior of a target system and reports if that behavior is consistent with the requirements. Such a monitor can be used both as an oracle during testing and as a supervisor during operation. Monitors should be based on the documented requirements of the system. If the target system is required to monitor or control real-valued quantities, then the requirements, which are expressed in terms of the monitored and controlled quantities, will allow a range of behaviors to account for errors and imprecision in observation and control of these quantities. Even if the controlled variables are discrete valued, the requirements must specify the timing tolerance. Because of the limitations of the devices used by the monitor to observe the environmental quantities, there is unavoidable potential for false reports, both negative and positive. This paper discusses design of monitors for real-time systems, and examines the conditions under which a monitor will produce false reports. We describe the conclusions that can be drawn when using a monitor to observe system behavior.

[1] B. Alpern and F.B. Schneider, “Defining Liveness,” Information Processing Letters, vol. 21, pp. 181–185, Oct. 1985.
[2] M. Auguston and P. Fritzson, “PARFORMAN—An Assertions Language for Specifying Behavior When Debugging Parallel Applications,” Int'l J. Software Engineering and Knowledge Eng., vol. 6, no. 4, pp. 609–640, 1996.
[3] L. Baresi and M. Young, “Test Oracles,” Technical Report CIS-TR-01-02, Dept. of Computer and Information Science, Univ. of Oregon, Aug. 2001, .
[4] ZUM, The Z Formal Specification Notation, no. 967,Lecture Notes in Computer Science, J.P. Bowen and M. Hinchey, eds., 1995.
[5] M. Brockmeyer, F. Jahanian, C. Heitmeyer, and B. Labaw, “An Approach to Monitoring and Assertion-Checking of Real Time Specifications in Modechart,” Proc. Workshop Parallel and Distributed Real-Time Systems, pp. 236–243, Apr. 1996.
[6] M. Diaz, G. Juanole, and J. Courtiat, “Observer—A Concept for Formal On-Line Validation of Distributed Systems,” IEEE Trans. Software Eng., vol. 20, no. 12, pp. 900–912, Dec. 1994.
[7] L.K. Dillon and Y.S. Ramakrishna, “Generating Oracles from Your Fravorite Temporal Logic Specificaitons,” Proc. Symp. Foundations of Software Eng., Oct. 1996.
[8] L.K. Dillon and Q. Yu, “Oracles for Checking Temporal Properties of Concurrent Systems,” Proc. Symp. Foundations of Software Eng., pp. 140–153, Dec. 1994.
[9] S. Fickas and M. Feather, “Requirements Monitoring in Dynamic Environments,” Proc. Int'l Symp. Requirements Eng. (RE'95), pp. 140–147, Mar. 1995.
[10] C. Fidge, “Fundamentals of Distributed System Observation,” IEEE Software, vol. 13, no. 6, pp. 77–83, Nov. 1996.
[11] C.A. Gunter, E.L. Gunter, M. Jackson, and P. Zave, “A Reference Model for Requirements and Specifications,” IEEE Software, pp. 37–43, May/June 2000.
[12] C.L. Heitmeyer, A. Bull, C. Gasarch, and B.G. Labaw, “SCR*: A Toolset for Specifying and Analyzing Requirements,” Proc. Conf. Computer Assurance (COMPASS), pp. 109–122, June 1995.
[13] K.L. Heninger, “Specifying Software Requirements for Complex Systems: New Techniques and their Application,” IEEE Trans. Software Eng., vol. 6, no. 1, pp. 2–13, Jan. 1980.
[14] K.L. Heninger, D.L. Parnas, J.E. Shore, and J. Kallander, “Software Requirements for the A-7E Aircraft,” Technical Report MR 3876, Naval Research Lab., 1978.
[15] M. Hlady, R. Kovacevic, J.J. Li, B.R. Pekilis, D. Prairie, T. Savor, and R.E. Seviora, “An Approach to Automatic Detection of Software Failures,” Proc. Int'l Symp. Software Reliability Eng. (ISSRE), pp. 314–323, Oct. 1995.
[16] H. Hörcher, “Improving Software Tests Using Z Specifications,” in Bowen and Hinchey [4], pp. 152–166,
[17] F. Jahanian, R. Rajkumar, and S.C.V. Raju, “Runtime Monitoring of Timing Constraints in Distributed Real-Time Systems,” Real-Time Systems, vol. 7, no. 3, pp. 247–273, 1994.
[18] I. Lee, S. Kannan, M. Kim, O. Sokolsky, and M. Viswanathan, “Runtime Assurance Based On Formal Specifications,” Proc. Int'l Conf. Parallel and Distributed Processing Techniques and Applications, June 1999.
[19] D. Luckham, F. von Henke, B. Krieg-Brückner, and O. Owe, ANNA A Language for Annotating Ada Programs Reference Manual. no. 260,Lecture Notes in Computer Science, 1987.
[20] M. Mansouri-Samani and M. Sloman, “Monitoring Distributed Systems (A Survey),” Research Report DOC92/23, Dept. of Computing, Imperial College, 180 Queen's Gate, London, UK, Apr. 1993.
[21] M. Mansouri-Samani and M. Sloman, “GEM: A Generalised Event Monitoring Language for Distributed Systems,” Technical Report DOC95/8, Dept. of Computing, Imperial College, 180 Queen's Gate, London, UK July 1995.
[22] E. Mikk, “Compilation of Z Specifications into C for Automatic Test Result Evaluation,” in Bowen and Hinchey [4], pp. 167–180, 1995.
[23] A.K. Mok and G. Liu, “Efficient Run-Time Monitoring of Timing Constraints,” Proc. IEEE Real-Time Technology and Applications Symp., 1997.
[24] D.L. Parnas and J. Madey, “Functional Documentation for Computer Systems,” Science of Computer Programming, vol. 25, no. 1, pp. 41–61, Oct. 1995.
[25] B.R. Pekilis and R.E. Seviora, “Detection of Response Time Failures fo Real-Time Software,” Proc. Int'l Symp. Software Reliability Eng. (ISSRE), Nov. 1997.
[26] D.K. Peters, “Generating a Test Oracle from Program Documentation,” MEng thesis, Dept. of Electrical and Computer Eng., McMaster Univ., Hamilton, Ontario, Apr. 1995.
[27] D.K. Peters, “Deriving Real-Time Monitors from System Requirements Documentation,” PhD thesis, McMaster University, Hamilton, Ontario, Jan. 2000.
[28] D.K. Peters and D.L. Parnas, “Using Test Oracles Generated from Program Documentation,” IEEE Trans. Software Eng., vol. 24, no. 3, pp. 161–173, Mar. 1998.
[29] J.G. Proakis and D.G. Manolakis, Digital Signal Processing Principles, Algorithms and Applications, second ed. Maxwell Macmillan, 1992.
[30] A.P. Ravn, H. Rischel, and K.M. Hansen, “Specifying and Verifying Requirements of Real-Time Systems,” IEEE Trans. Software Eng., vol. 19, no. 1, pp. 41–55, Jan. 1993.
[31] D.J. Richardson, S.L. Aha, and T.O. O'Malley, “Specification-Based Test Oracles for Reactive Systems,” Proc. Int'l Conf. Software Eng. (ICSE), pp. 105–118, May 1992.
[32] D.S. Rosenblum, “A Practical Approach to Programming With Assertions,” IEEE Trans. Software Eng., vol. 21, no. 1, pp. 19–31, Jan. 1995.
[33] Proc. IEEE Real-Time Technology and Applications Symp., June 1997.
[34] T. Savor and R.E. Seviora, “An Approach to Automatic Detection of Software Failures in Real-Time Systems,” Proc. IEEE Real-Time Technology and Applications Symp., June 1997.
[35] U. Schmid, “Monitoring Distributed Real-Time Systems,” Real-Time Systems, vol. 7, no. 1, pp. 33–56, July 1994.
[36] D. Simser and R.E. Seviora, “Supervision of Real-Time Systems Using Optimistic Path Prediction and Rollbacks,” Proc. Int'l Symp. Software Reliability Eng. (ISSRE), pp. 340–349, Oct. 1996.
[37] P. Stocks and D. Carrington, “Test Template Framework: A Specification-Based Testing Case Study,” Proc. Int'l Symp. Software Testing and Analysis (ISSTA '93), pp. 11–18, June 1993.
[38] Sun Microsystems Inc., ADL Language Reference Manual for ANSI C programmers, Release 1.1, document reference miti/0002/d/r1.1 ed., Dec. 1996.
[39] Distributed Real-Time Systems: Monitoring, Visualization, Debugging, and Analysis. J.J. Tsai, Y. Bi, S.J.H. Yang, and R.A.W. Smith, eds., ohn Wiley&Sons, 1996.
[40] Monitoring and Debugging of Distributed Real-Time Systems. J.J. Tsai and S.J. Yang, eds., IEEE CS Press, 1995.
[41] A.J. van Schouwen, “The A-7 Requirements Model: Re-Examination for Real-Time Systems and An Application to Monitoring Systems,” Technical Report TR 90-276, Queen's Univ., Kingston, Ontario, 1990.
[42] A.J. van Schouwen, D.L. Parnas, and J. Madey, “Documentation of Requirements for Computer Systems,” Proc. Int'l Symp. Requirements Eng. (RE'93), pp. 198–207, Jan. 1993.
[43] E.J. Weyuker, “On Testing Non-Testable Programs,” The Computer J., vol. 25, no. 4, pp. 465–470, 1982.
[44] P. Zave and M. Jackson, “Four Dark Corners of Requirements Engineering,” ACM Trans. Software Eng. and Methodology, vol. 6, no. 1, pp. 1–30, Jan. 1997.

Index Terms:
Automated testing, test oracle, real-time system, supervisor.
D.K. Peters, D.L. Parnas, "Requirements-Based Monitors for Real-Time Systems," IEEE Transactions on Software Engineering, vol. 28, no. 2, pp. 146-158, Feb. 2002, doi:10.1109/32.988496
Usage of this product signifies your acceptance of the Terms of Use.