Issue No. 01 - March (1989 vol. 1)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/69.43407
<p>The author suggests a new focus on benchmarks for knowledge systems, following the lines of similar benchmarks in other computing fields. It is noted that knowledge systems differ from conventional systems in a key way, namely their ability to interpret and apply knowledge. This gives rise to a distinction between intrinsic measures concerned with engineering qualities and extrinsic measures relating to task productivity, and both warrant improved measurement techniques. Primary concerns within the extrinsic realm include advice quality, reasoning correctness, robustness, and solution efficiency. Intrinsic concerns, on the other hand, center on elegance of knowledge base design, modularity, and architecture. The author suggests criteria for good measures and benchmarks, and ways to satisfy these through the design of knowledge and key knowledge engineering costs and performance parameters. It is suggest that the focus on measuring knowledge systems should help clarify the technical relationships between knowledge engineering and data engineering.</p>
benchmarks; knowledge systems; data engineering; interpret; apply; intrinsic measures; engineering qualities; extrinsic measures; task productivity; advice quality; reasoning correctness; robustness; solution efficiency; elegance; knowledge base design; modularity; architecture; knowledge engineering; knowledge based systems; knowledge engineering; performance evaluation; program testing
F. Hayes-Roth, "Towards Benchmarks for Knowledge Systems and Their Implications for Data Engineering," in IEEE Transactions on Knowledge & Data Engineering, vol. 1, no. , pp. 101-110, 1989.