This Article 
   
 Share 
   
 Bibliographic References 
   
 Add to: 
 
Digg
Furl
Spurl
Blink
Simpy
Google
Del.icio.us
Y!MyWeb
 
 Search 
   
A Taxonomy and Catalog of Runtime Software-Fault Monitoring Tools
December 2004 (vol. 30 no. 12)
pp. 859-872
ASCII Text
x 
 
Nelly Delgado, Ann Quiroz Gates, Steve Roach, "A Taxonomy and Catalog of Runtime Software-Fault Monitoring Tools," IEEE Transactions on Software Engineering, vol. 30, no. 12, pp. 859-872, December, 2004.
 
 
BibTex
x
 
@article{ 10.1109/TSE.2004.91,
author = {Nelly Delgado and Ann Quiroz Gates and Steve Roach},
title = {A Taxonomy and Catalog of Runtime Software-Fault Monitoring Tools},
journal ={IEEE Transactions on Software Engineering},
volume = {30},
number = {12},
issn = {0098-5589},
year = {2004},
pages = {859-872},
doi = {http://doi.ieeecomputersociety.org/10.1109/TSE.2004.91},
publisher = {IEEE Computer Society},
address = {Los Alamitos, CA, USA},
}
 
 
RefWorks Procite/RefMan/Endnote
x 
 
TY - JOUR
JO - IEEE Transactions on Software Engineering
TI - A Taxonomy and Catalog of Runtime Software-Fault Monitoring Tools
IS - 12
SN - 0098-5589
SP859
EP872
EPD - 859-872
A1 - Nelly Delgado,
A1 - Ann Quiroz Gates,
A1 - Steve Roach,
PY - 2004
KW - Assertion checkers
KW - runtime monitors
KW - specification
KW - specification language
KW - survey
KW - software/program verification.
VL - 30
JA - IEEE Transactions on Software Engineering
ER -
 
Ann Quiroz Gates, IEEE Computer Society
Steve Roach, IEEE Computer Society
A goal of runtime software-fault monitoring is to observe software behavior to determine whether it complies with its intended behavior. Monitoring allows one to analyze and recover from detected faults, providing additional defense against catastrophic failure. Although runtime monitoring has been in use for over 30 years, there is renewed interest in its application to fault detection and recovery, largely because of the increasing complexity and ubiquitous nature of software systems. This paper presents a taxonomy that developers and researchers can use to analyze and differentiate recent developments in runtime software fault-monitoring approaches. The taxonomy categorizes the various runtime monitoring research by classifying the elements that are considered essential for building a monitoring system, i.e., the specification language used to define properties; the monitoring mechanism that oversees the program's execution; and the event handler that captures and communicates monitoring results. After describing the taxonomy, the paper presents the classification of the software-fault monitoring systems described in the literature.

[1] B. Alpern and F. Schneider, “Verifying Temporal Properties without Temporal Logic,” ACM Trans. Programming Languages, vol. 11, no. 1, pp. 147-167, Jan. 1989.
[2] A. Avizienis and J. Laprie, “Dependable Computing: From Concepts to Design Diversity,” Proc. IEEE, vol. 74, no. 5, pp. 629-638, May 1986.
[3] M. Barbacci, “Quality Attributes,” Technical Report CMU/SEI-95-TR-021, Software Eng. Inst., Carnegie Mellon Univ., Dec. 1995.
[4] D. Bartetzko, “Jass— Java with Assertions,” Proc. First Workshop Runtime Verification (RV '01), July 2001.
[5] B. Berard and M. Bidoit, Systems and Software Verification: Model-Checking Techniques and Tools. Springer Verlag, 2001.
[6] G. Brand, “Java Pathfinder JPF2 Second Generation of Java Model Checker,” http://www.ist.tugraz.at/courses/akswt03 paper_ jpf2.pdf, July 2004.
[7] B. Bruegge, T. Gottschalk, and B. Luo, “A Framework for Dynamic Program Analyzers,” Proc. Eighth Ann. Conf. Object-Oriented Programming Systems, Languages, and Applications, pp. 65-82, 1993.
[8] L. Burdy, Y. Cheon, D.R. Cok, M. Ernst, J. Kiniry, G.T. Leavens, K. Rustan, M. Leino, and E. Poll, “An Overview of JML Tools and Applications,” Proc. Eighth Int'l Workshop Formal Methods for Industrial Critical Systems (FMICS '03), T. Arts and W. Fokkink, eds., June 2003.
[9] F. Chen and G. Roşu, “Towards Monitoring-Oriented Programming: A Paradigm Combining Specification and Implementation,” Electronic Notes in Theoretical Computer Science, vol. 89, no. 2, 2003.
[10] Y. Cheon, “A Runtime Assertion Checker for the Java Modeling Language,” Technical Report 03-09, Dept. of Computer Science, Iowa State Univ., Ames, 2003.
[11] S. Chodrow and M. Gouda, “Implementation of the Sentry System,” Software Practice and Experience, vol. 25, no. 4, pp. 73-387, 1995.
[12] M. Clavel, S. Eker, P. Lincoln, and J. Meseguer, “Principles of Maude,” Electronic Notes in Theoretical Computer Science, vol. 4, 1996.
[13] M. Clavel, F. Duran, S. Eker, P. Lincoln, N. Marti-Oliet, J. Meseguer, and J. Quesada, “Using Maude,” Proc. Third Int'l Conf. Fundamental Approaches to Software Eng., 2000.
[14] J. Cobleigh, L. Clarke, and L. Osterweil, “Flavers: A Finite State Verification Technique for Software Systems,” technical report, Univ. of Massachusetts Amherst, Apr. 2001.
[15] T. Colcombet and P. Fradet, “Enforcing Trace Properties by Program Transformation,” Proc. 27th ACM SIGPLAN-SIGACT Symp. Principles of Programming Languages, pp. 54-66, 2000.
[16] N. Delagdo, “A Taxonomy of Dynamic Software-Fault Monitoring Tools,” master's thesis, Univ. of Texas at El Paso, May 2002.
[17] J. deRaeve and S. McCarron, “Automated Test Generation Technology: Assertion Definition Language Project,” technical report, X/Open Company Ltd., 1997.
[18] M. Diaz, G. Juanole, and J. Courtiat, “Observer— A Concept for Formal On-Line Validation of Distributed Systems,” IEEE Trans. Software Eng., vol. 20, no. 12, pp. 900-913, Dec. 1994.
[19] D. Drusinsky, “The Temporal Rover and the ATG Rover,” Proc. SPIN 2000 Conf., 2000.
[20] D. Drusinksky, “Temporal Rule Checking,” An Oracle AppsWorld Online White Paper, http:/www.dbrover.com, July 2004.
[21] D. Drusinsky, Time-Rover, Cupertino, Calif., personal comm., 2002.
[22] S. Dudani, J. Geada, G. Jakacki, and D. Vainer, “Dynamic Assertions Using TXP,” Proc. First Workshop Runtime Verification (RV '01), July 2001.
[23] G. Eisenhauer, “The Echo Event Delivery System,” http://www. cc.gatech.edu/systems/projects ECho, July 2004.
[24] G. Eisenhauer, Georgia Inst. of Technology, Atlanta, Ga., personal comm., 2002.
[25] G. Eisenhauer and K. Schwan, “The Mirror Object Steering System,” http://www.cc.gatech.edu/systems/projects MOSS, July 2004.
[26] A. Gates and O. Mondragon, “A Constraint-Based Tracing Approach,” J. Systems and Software, vol. 63, pp. 219-239, Sept. 2002.
[27] A. Gates, S. Roach, O. Mondragon, and N. Delgado, “DynaMICs: Comprehensive Support for Run-Time Monitoring,” Proc. First Workshop Runtime Verification (RV '01), July 2001.
[28] A. Gates and P. Teller, “An Integrated Design of a Dynamic Software-Fault Monitoring System,” J. Integrated Design and Process Science, vol. 14, pp. 63-78, 2000.
[29] W. Gu, G. Eisenhauer, E. Kraemer, K. Schwan, J. Stasko, and J. Vetter, “Falcon: On-Line Monitoring and Steering of Large-Scale Parallel Programs,” Technical Report GIT-CC-94-21, College of Computing, Georgia Inst. of Tech nology, Apr. 1994.
[30] W. Gu, J. Vetter, and K. Schwan, “An Annotated Bibliography of Interactive Program Steering,” ACM SIGPLAN Notices, Sept. 1994.
[31] D. Haban and D. Wybranietz, “A Hybrid Monitor for Behavior and Performance Analysis of Distributed Systems,” IEEE Trans. Software Eng., vol. 16, no. 2, pp. 197-211, Feb. 1990.
[32] K. Havelund, “Using Runtime Analysis to Guide Model Checking of Java Programs,” Lecture Notes in Computer Science, vol. 1885, pp. 245-264, 2000.
[33] K. Havelund, NASA, Ames, Moffet Field, Calif., personal comm., 2002.
[34] K. Havelund and G. Roşu, “Monitoring Programs Using Rewriting,” Proc. 16th IEEE Int'l Conf. Automated Software Eng., pp. 135-143, 2001.
[35] K. Havelund and G. Roşu, “Java PathExplorer— A Runtime Verification Tool,” Proc. Symp. Artificial Intelligence, Robotics and Automation in Space, June 2001.
[36] K. Havelund and G. Roşu, “Monitoring Java Programs with Java PathExplorer,” Proc. First Workshop Runtime Verification (RV '01), July 2001.
[37] G. Holtzmann, “The Spin Model Checker,” IEEE Trans. Software Eng., vol. 23, no. 5, pp. 279-295, May 1997.
[38] C.L. Jeffery, Program Monitoring and Visualization: An Exploratory Approach. Springer-Verlag, 1999.
[39] C.L. Jeffrey, New Mexico State Univ., Las Cruces, N.M., personal comm., 2002.
[40] C.L. Jeffery, W. Zhou, K. Templer, and M. Brazell, “A Lightweight Architecture for Program Execution Monitoring,” Proc. ACM SIGPLAN/SIGSOFT Workshop Program Analysis for Software Tools and Eng., pp. 67-74, 1998.
[41] M. Kim, S. Kannan, I. Lee, O. Sokolsky, and M. Viswanathan, “Java-MaC: A Run-Time Assurance Tool for Java Programs,” Proc. Fourth IEEE Int'l High Assurance Systems Eng. Symp., pp. 115-132, 1999.
[42] M. Kim, I. Lee, and O. Sokolsky, Univ. of Pennsylvania, Philadelphia, personal comm., 2002.
[43] M. Kim, and M. Viswanathan, “MaC: A Framework for Run-Time Correctness Assurance of Real-Time Systems,” Technical Report MS-CIS-98-37, Dept. of Computer and Information Sciences, Univ. of Pennsylvania, Dec. 1998.
[44] M. Kim and M. Viswanathan, “Formally Specified Monitoring of Temporal Properties,” Proc. European Conf. Real-Time Systems, 1999.
[45] I. Lee and H. Ben-Abdallah, “A Monitoring and Checking Framework for Run-Time Correctness Assurance,” Proc. 1998 Korea-U.S. Technical Conf. Strategic Technologies, 1998.
[46] I. Lee and M. Kim, “Runtime Assurance Based on Formal Specifications,” Proc. 1999 Int'l Conf. Parallel and Distributed Processing Techniques and Applications, 1999.
[47] Y. Liao and D. Cohen, “A Specificational Approach to High Level Program Monitoring and Measuring,” IEEE Trans. Software Eng., vol. 18, no. 11, pp. 969-978, Nov. 1992.
[48] D. Luckham, W. Mann, S. Meldal, and D. Helmbold, “An Overview of Ann Specification Language for Ada,” IEEE Software, vol. 20, pp. 9-23, 1988.
[49] D. Luckham, S. Sankar, and S. Takahashi, “Two-Dimensional Pinpointing: Debugging with Formal Specifications,” IEEE Software, vol. 8, no. 1, pp. 74-84, Jan. 1991.
[50] B. Meyer, “Applying Design by Contract,” Computer, vol. 25, no. 10, pp. 40-51, Oct. 1992.
[51] A. Mok, Univ. of Texas at Austin, Austin, Tex., personal comm., 2002.
[52] A. Mok and G. Liu, “Efficient Run-Time Monitoring of Timing Constraints,” Proc. Third IEEE Real-Time Technology and Applications Symp., pp. 252-262, 1997.
[53] M. Möller, Univ. of Oldenburg, Oldenburg, Germany, personal comm., 2002.
[54] J. Nimmer and M. Ernst, “Static Verification of Dynamically Detected Program Invariants: Integrating Daikon and ESC/Java,” Proc. First Workshop Runtime Verification (RV '01), July 2001.
[55] D. Peters, “Automated Testing of Real-Time Systems,” technical report, Memorial Univ. of Newfoundland, Nov. 1999.
[56] D. Peters, “Deriving Real-Time Monitors from System Requirements Documentation,” PhD thesis, McMaster Univ., Jan. 2000.
[57] D. Peters, Memorial Univ. of Newfoundland, St. John's, Canada, personal comm., 2002.
[58] D. Peters and D. Parnas, “Requirements-Based Monitors for Real-Time Systems,” Proc. Int'l Symp. Software Testing and Analysis (ISSTA), pp. 77-85, 2000.
[59] B. Plattner and J. Nievergelt, “Monitoring Program Execution: A Survey,” Computer, vol. 14, pp. 76-93, 1981.
[60] W.N. Robinson, “Monitoring Web Service Requirements,” Proc. 12th IEEE Int'l Conf. on Requirements Eng., pp. 65-74, 2003.
[61] W.N. Robinson, “Monitoring Software Requirements Using Instrumented Code,” Proc. Hawaii Int'l Conf. System Sciences, vol. 9, pp. 276-287, Jan. 2002.
[62] D. Rosenblum, “A Practical Approach to Programming with Assertions,” IEEE Trans. on Software Eng., vol. 21, no. 1, pp. 19-31, Jan. 1995.
[63] G. Roşu and K. Havelund, “Synthesizing Dynamic Programming Algorithms from Linear Temporal Logic Formulae,” RIACS Technical Report TR 01-15, May 2001.
[64] RTI, “The Economic Impacts of Inadequate Infrastructure for Software Testing,” Health, Social, and Economic Research, Project No. 7007-001, Research Triangle Park, N.C., 2002.
[65] S. Rugaber, “DYNAMO Dynamic Assembly from Models,” http://www.cc.gatech.edu/dynamo/txtproposal.ps , July 2004.
[66] S. Rugaber and R. Stirewalt, “Position Paper: The Tradeoff between Dependability and Efficiency in Embedded Systems,” Proc. CASES 2001 Conf., Aug. 2001.
[67] S. Sankar and M. Mandal, “Concurrent Runtime Monitoring of Formally Specified Programs,” Computer, vol. 26, no. 3, pp. 32-41, Mar. 1993.
[68] B. Schroeder, “On-Line Monitoring: A Tutorial,” Computer, vol. 28, no. 6, pp. 72-78, June 1995.
[69] R. Smith and G. Avrunin, L. Clarke, and L. Osterweil, “PROPEL: An Approach Supporting Property Elucidation,” Proc. Int'l Conf. Software Eng., May 2002.
[70] O. Sokolsky, S. Kannan, and M. Viswanathan, “Steering of Real-Time Systems Based on Monitoring and Checking,” Proc. Fifth Int'l Workshop Object-Oriented Real-Time Dependable Systems, pp. 11-18, 1999.
[71] R. Sosic, “Dynascope: A Tool for Program Directing,” Proc. Fifth ACM SIGPLAN Conf. Programming Language Design and Implementation, pp. 12-21, 1992.
[72] A. Tanenbaum, Modern Operating Systems. Prentice Hall, 2001.
[73] P. Teller, M. Maxwell, and A. Gates, “Towards the Design of a Snoopy Coprocessor for Dynamic Software-Fault Detection,” Proc. 18th IEEE Int'l Performance, Computing, and Comm. Conf., pp. 310-317, 1999.
[74] K. Templer and C. Jeffery, “A Configurable Automatic Instrumentation Tool for ANSI C,” Proc. 13th IEEE Int'l Conf. Automated Software Eng., pp. 249-258, 1998.
[75] J. Tsai, Y. Bi, S. Yang, and R. Smith, Distributed Real-Time Systems: Monitoring, Visualization, Debugging and Analysis. John Wiley and Sons, 1996.
[76] J. Tsai and K. Fang, “A Noninterference Monitoring and Replay Mechanism for Real-Time Software Testing and Debugging,” IEEE Trans. Software Eng., vol. 16, no. 8, pp. 897-916, Aug. 1990.

Index Terms:
Assertion checkers, runtime monitors, specification, specification language, survey, software/program verification.
Citation:
Nelly Delgado, Ann Quiroz Gates, Steve Roach, "A Taxonomy and Catalog of Runtime Software-Fault Monitoring Tools," IEEE Transactions on Software Engineering, vol. 30, no. 12, pp. 859-872, Dec. 2004, doi:10.1109/TSE.2004.91
Usage of this product signifies your acceptance of the Terms of Use.