This Article 
 Bibliographic References 
 Add to: 
Objective vs. Subjective Measures of Error-Proneness for Rule-Based Programs
November/December 1998 (vol. 10 no. 6)
pp. 1008-1012

Abstract—This paper relates four objective measures of program structure and three subjective ratings of program complexity to the number of postrelease documented errors contained within 80 commercially developed Prolog programs. All seven measures show a significant correlation with the number of errors. A factor analysis showed that the objective and subjective measures were indeed different measures, although a hierarchical analysis of oblique factors showed a strong common root. Finally, the Mann-Whitney U test was used to determine whether the measures could differentiate between those programs with errors and those with no documented errors. The results suggest that "complexity" can be grounded in terms of the difficulty to debug or test a program, while measures of "structure" require a detailed count of the number of predicates used within the program.

[1] A. Arora and J.E. Cooke, "Towards Effective Management of Expert System Projects," Proc. IEEE/ACM Int'l Conf. Developing and Managing Expert System Programs.Los Alamitos, Calif.: IEEE CS Press, pp. 339-345, 1991.
[2] B. Beizer, Software Testing Techniques, second ed. London: Thomson International Computer Press, 1990.
[3] I. Bratko, Prolog Programming for Artificial Intelligence, second ed. Wokingham, U.K.: Addison-Wesley, 1990.
[4] C.L. Chang, J.B. Combs, and R.A. Stachowitz, "A Report on the Expert Systems Validation Associate (EVA)," Expert Systems with Applications, vol. 1, pp. 217-230, 1990.
[5] Z. Chen and C.Y. Suen, "Complexity Metrics for Rule-Based Expert Systems," Proc. IEEE Int'l Conf. Software Maint., pp. 382-391.Los Alamitos, Calif.: IEEE CS Press, 1994.
[6] P. Doyle and R. Verbruggen, "Applying Metrics to Rule-Based Systems," Proc. Fourth IEEE Int'l Conf. Software Eng. and Knowledge Eng., pp. 123-130.Los Alamitos, Calif.: IEEE CS Press, 1992.
[7] N.E. Fenton and A.A. Kaposi, "An Engineering Theory of Structure and Measurement," Measurement for Software Control and Assurance, B.A. Kitchenham and B. Littlewood, eds. London: Elsevier, 1989.
[8] C. Grossner, A.D. Preece, P. Gokul Chander, T. Radhakrishnan, and C.Y. Suen, "Exploring the Structure of Rule Based Systems," Proc. AAAI, pp. 704-709, 1993.
[9] A. Gupta and C.L. Forgy, "Static and Run-Time Characteristics of OPS5 Production Systems," J. Parallel and Distributed Computing, vol. 7, pp. 64-95, 1989.
[10] S.H. Kaisler, "Expert System Metrics," Proc. Int'l Conf. Neural Networks, pp. 114-120, 1986.
[11] J.D. Kiper, "Structural Testing of Rule-Based Expert Systems," ACM Trans. Software Eng. Methodology, vol. 1, pp. 168-187, 1992.
[12] Z. Markusz and A.A. Kaposi, "Complexity Control in Logic-Based Programming," Computer J., vol. 28, pp. 487-495, 1985.
[13] M. Myers and A.A. Kaposi, "Modelling and Measurement of Prolog Data," Software Eng. J., vol. 6, pp. 413-434, 1991.
[14] D.L. Nazareth, "Issues in the Verification of Knowledge in Rule-Based Systems," Int'l J. Man-Machine Studies, vol. 30, pp. 255-271, 1989.
[15] D.E. O'Leary, "The Relationship Between Errors and Size in Knowledge-Based Systems," Int'l J. Human-Computer Studies, vol. 44, pp. 171-185, 1996.
[16] M.B. O'Neal and W.R. Edwards Jr., "Complexity Measures for Rule-Based Programs," IEEE Trans. Knowledge and Data Eng., vol. 6, pp. 669-680, 1994.
[17] A.D. Preece, "A New Approach to Detecting Missing Knowledge in Expert System Rule Bases," Int'l J. Man-Machine Studies, vol. 38, pp. 161-181, 1993.
[18] A.D. Preece, "Toward a Quality Assessment Framework for Knowledge-Based Systems," J. Systems and Software, vol. 29, pp. 219-234, 1995.
[19] J. Rushby, "Quality Measures and Assurance for AI Software," NASA Contractor Report CR-4187. Menlo Park, Calif.: SRI International, 1988.
[20] R.J. Wherry, Contributions to Correlational Analysis.New York: Academic Books, 1984.
[21] S.S. Yau and J.S. Collofello, "Some Stability Measures for Software Maintenance," IEEE Trans. Software Eng., vol. 6, pp. 545-552, 1980.

Index Terms:
Complexity measures, program structure, error-proneness, system quality, rule-based systems, empirical, Prolog.
Trevor T. Moores, "Objective vs. Subjective Measures of Error-Proneness for Rule-Based Programs," IEEE Transactions on Knowledge and Data Engineering, vol. 10, no. 6, pp. 1008-1012, Nov.-Dec. 1998, doi:10.1109/TKDE.1998.10002
Usage of this product signifies your acceptance of the Terms of Use.