This Article 
 Bibliographic References 
 Add to: 
Testing Expert Systems in Process Control
June 1996 (vol. 8 no. 3)
pp. 403-415

Abstract—Special features of process control expert systems (PCX) make it both necessary and feasible to test them in a comprehensive manner. FAITH, an automated regression testing environment for PCX, integrates five specially adapted testing techniques, exploiting external specifications gained from the process control environment. FAITH also offers a set of metrics which can be used to predict testability during the early phases of PCX development. FAITH has been developed for a major German chemicals company where it is routinely used for certifying PCX, and for guiding design-for-testability.

[1] M. Ayel and J.-P. Laurent, "SACCOSYCOJET: Two Different Ways of Verifying Knowledge-Based Systems," Validation, Verification and Test of Knowledge Based Systems, M. Ayel and J.-P. Laurent, eds. Chichester, U.K.: John Wiley&Sons, 1991.
[2] M. Bruns, W. Eckelmann, U. Müller-Nehler, and K.-H. Wietschorke, "Wissensbasierte Systeme—Allheilmittel oder Flop," Chemische Industrie, vol. 2, pp. 38-40, 1992.
[3] H.-J. Bullinger, K.-P. Fähnrich, and G. v.Drabich-Wächter, "Expertensysteme: Wissensbasierte Systeme in der betrieblichen Anwendung," expert-Verlag, Böblingen 1988.
[4] N. Bouteldja, "A Reusable Environment for Verification and Validation of Knowledge-Based Systems," . Dependability of Artificial Intelligence Systems (DAISY-91), Schildt and Retti, eds., pp. 107-120, North-Holland.
[5] S. Bologna and H. Välisuo, "Deep Knowledge and Rigorous Engineering Practice: Two Ingredients to Build Dependable Knowledge Based Systems for Industrial Process Control," Dependability of Artificial Intelligence Systems (DAISY-91), Schildt and Retti, eds., pp. 73-90, North-Holland.
[6] E. Charles and O. Dubois, "MELODIA: Logical Methods for Checking Knowledge Bases," Validation, Verification and Test of Knowledge Based Systems, M. Ayel and J.-P. Laurent, eds. Chichester, U.K.: John Wiley, 1991.
[7] W.E. Deming, Out of the Crisis.Cambridge, Mass.: MIT Press, 1986.
[8] K. Finke, "Systematisches Testen von Expertensystemen zur Prozeßführung am Beispiel der Shell HEPROX," Diploma thesis, Informatik V, RWTH Aachen, 1993.
[9] A. Ginsberg, "Knowledge Base Reduction: A New Approach to Checking Knowledge Bases for Inconsistency and Redundancy," Proc. Seventh Nat'l Conf. Artificial Intelligence, 1988.
[10] C.J.R. Green and M.M. Keyes, "Verification and Validation of Expert Systems," Proc. Western Conf. Expert Systems, pp. 38-43, 1987.
[11] U.G. Gupta, Validating and Verifying Knowledge-Based Systems.Los Alamitos, Calif.: IEEE CS Press, 1991.
[12] P. Harmon and B. Sawyer, Creating Expert Systems for Business and Industry. John Wiley, 1990.
[13] J. Höhmann, K.H. Simmrock, and R. Soltysiak, "Expertensystem zur Prozeßführung einer Sprühtrocknungsanlage," Chemie-Ingenieur-Technik, vol. 64, no. 8, pp. 715-717, 1992.
[14] J. Höhmann, W. Röhl, R. Soltysiak, and K.H. Simmrock, "Wissenserwerb bei Experten—systemen zur Prozeßführung am Beispiel der Waschmittelproduktion," Automatisierungstechnische Praxis, vol. 32, no. 12, 1990.
[15] IEEE Standard for Software Test Documentation, ANSI/IEEE Standard 829-1983, 1983.
[16] ISO 3511, Int'l Organization for Standardization, "Process Measurement, Control Functions, and Instrumentation—Symbolic Representation, Part I-II," 1977.
[17] ISO 9000-3, Int'l Organization for Standardization, "Guidelines for the Application of ISO 9001 to the Development, Supply and Maintenance of Software," ISO, Geneva, 1991.
[18] A.V. Jones and M.G. Rodd, "Problems with Expert Systems in Real-Time Control," Eng. Applications in Artificial Intelligence, vol. 6, no. 6, pp. 499-506, 1993.
[19] Y. Kang and T. Bahill, "A Tool for Detecting Expert Systems Errors," AI Expert, vol. 5, no. 2, pp. 42-51, 1990.
[20] B. Kuipers, "Qualitative Reasoning—Modeling and Simulation with Incomplete Knowledge," Automatica, vol. 25, no. 4, 1989.
[21] N.E. Lane, "Global Issues in Evaluation of Expert Systems," Proc. IEEE Int'l Conf. Systems, Man, and Cybernetics, pp. 121-125,Atlanta, 1986.
[22] B. Marcor, "Testing Your Knowledge Base," AI Expert, vol. 2, no. 8, pp. 42-47, 1987.
[23] T.J. McCabe, "A Complexity Measure," IEEE Trans. Software Eng., vol. 2, no. 4, 1976.
[24] K.L. McGraw, "Guidelines for Producing Documentation for Expert Systems," IEEE Trans. Professional Comm., vol. 29, no. 4, 1986.
[25] S.M. McMenamin and J. Palmer, Essential Systems Analysis, Prentice Hall, Englewood Cliffs, N.J., 1984.
[26] G.J. Myers,The Art of Software Testing.New York: Wiley, 1979.
[27] T.A. Nguyen, W.A. Perkins, T.J. Laffey, and D. Pecora, “Knowledge Base Verification,” AI Magazine, pp. 69–75, Summer 1987.
[28] W.A. Perkins, T.J. Laffey, D. Pecora, and T.A. Nguyen, "Knowledge Base Verification," Topics in Expert Systems Design, G. Guida and C. Tasso, eds. Amsterdam and New York: North-Holland, 1989.
[29] A.D. Preece and R. Shinghal, "DARC: A Procedure for Verifying Rule-Based Systems," Proc. World Congress on Expert System, 1991.
[30] M.-C. Rousset, "On the Consistency of Knowledge Bases: The COVADIS System," Proc. ECAI-88 (European Conf. Artificial Intelligence), pp. 79-84.
[31] F. Saglietti, "Structural Testing Strategies Applied to Knowledge-Based Systems," Industrial and Engineering Applications of Artificial Intelligence and Expert Systems. Springer-Verlag, 1992.
[32] R. Soltysiak, "HEPROX, eine Expertensystemshell für Prozeßführungsaufgaben," Automatisierungstechnische Praxis, vol. 31, no. 2, pp. 74-80, 1989.
[33] M. Suwa, C.A. Scott, and E.H. Shortliffe, "An Approach to Verifying Completeness and Consistency in a Rule-Based Expert System," AI-Magazine, vol. 3, no. 3, pp. 16-21, 1982.
[34] P. Szczurko, "Qualitätssicherung von Expertensystemen zur Prozeßkontrolle," Proc. KAQS Workshop, Expertensysteme 95,Kaiserslautern, Germany, 1995 (in German).
[35] J. Tepandi, "Verification, Testing and Validation of Rule-Based Expert Systems," IFAC 11th Triennial World Congress,Tallin, Estonia, 1990.
[36] J.P. Tsai, A. Liu, and K. Nair, "Debugging Logic-Based Requirements Specifications—A FRORL Approach," Proc. Fifth Int'l Conf. Software Eng. and Knowledge Eng., pp. 372-379,San Francisco, 1993.
[37] R.W. Yeomans, A. Choudry, and P.J.W. Ten Hagen, Design Rules for a CIM System. North-Holland, 1987.

Index Terms:
Test methods, measurement environment, expert systems, process control, external specification, process control expert systems (PCX), test metrics.
Kai Finke, Matthias Jarke, Roland Soltysiak, Peter Szczurko, "Testing Expert Systems in Process Control," IEEE Transactions on Knowledge and Data Engineering, vol. 8, no. 3, pp. 403-415, June 1996, doi:10.1109/69.506708
Usage of this product signifies your acceptance of the Terms of Use.