Issue No. 06 - June (1997 vol. 23)
DOI Bookmark: http://doi.ieeecomputersociety.org/10.1109/32.601075
<p><b>Abstract</b>—Inability to identify weaknesses or to quantify advancements in software system robustness frequently hinders the development of robust software systems. Efforts have been made to develop benchmarks of software robustness to address this problem, but they all suffer from significant shortcomings. This paper presents the various features that are desirable in a benchmark of system robustness, and evaluates some existing benchmarks according to these features. A new hierarchically structured approach to building robustness benchmarks, which overcomes many deficiencies of past efforts, is also presented. This approach has been applied to building a hierarchically structured benchmark that tests part of the Unix file and virtual memory systems. The resultant benchmark has successfully been used to identify new response class structures that were not detected in a similar situation by other less organized techniques.</p>
System reliability, software dependability, robustness benchmarking, test suite organization, object-oriented benchmarks, software validation, extensible benchmarks.
Arup Mukherjee, Daniel P. Siewiorek, "Measuring Software Dependability by Robustness Benchmarking", IEEE Transactions on Software Engineering, vol. 23, no. , pp. 366-378, June 1997, doi:10.1109/32.601075