Transactions on Software Engineering

The IEEE Transactions on Software Engineering (TSE) is an archival journal published bimonthly. We are interested in well-defined theoretical results and empirical studies that have potential impact on the construction, analysis, or management of software. Read the full scope of TSE

Expand your horizons with Colloquium, a monthly survey of abstracts from all CS transactions! Replaces OnlinePlus in January 2017.

From the April 2017 Issue

Test Oracle Strategies for Model-Based Testing

By Nan Li and Jeff Offutt

Featured article thumbnail imageTesters use model-based testing to design abstract tests from models of the system’s behavior. Testers instantiate the abstract tests into concrete tests with test input values and test oracles that check the results. Given the same test inputs, more elaborate test oracles have the potential to reveal more failures, but may also be more costly. This research investigates the ability for test oracles to reveal failures. We define ten new test oracle strategies that vary in amount and frequency of program state checked. We empirically compared them with two baseline test oracle strategies. The paper presents several main findings. (1) Test oracles must check more than runtime exceptions because checking exceptions alone is not effective at revealing failures. (2) Test oracles do not need to check the entire output state because checking partial states reveals nearly as many failures as checking entire states. (3) Test oracles do not need to check program states multiple times because checking states less frequently is as effective as checking states more frequently. In general, when state machine diagrams are used to generate tests, checking state invariants is a reasonably effective low cost approach to creating test oracles.

download PDF View the PDF of this article      csdl View this issue in the digital library      TSE Facebook Link  TSE on Facebook

Editorials and Announcements



Guest Editorials

Reviewers List

Annual Index

Access All Recently Published TSE Articles

RSS Subscribe to the RSS feed of latest TSE content added to the digital library

Mail Sign up for the Transactions Connection Newsletter.

Access TSE PrePrints in the Computer Society digital library

A PrePrint is an article that has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication