An Experiment in Specification-based Testing
Ian MacColl
Department of Computer Science,
The University of Queensland,
Queensland 4072, Australia.
ianm@cs.uq.edu.au
David Carrington
Department of Computer Science,
The University of Queensland,
Queensland 4072, Australia.
davec@cs.uq.edu.au
Phil Stocks
Department of Computer Science,
Hill Center, Busch Campus,
Rutgers University,
Piscataway, NJ 08855, USA.
pstocks@cs.rutgers.edu
Abstract
The Test Template Framework (TTF) is a formal, abstract model of
tests used to derive a hierarchy of test information, including test
cases and oracles.
The hierarchy is derived from a formal model-based specification,
for example, one written in the Z notation.
A case study presents a Test Template Hierarchy (TTH) derived from
a specification for topological sorting.
Test suites were also prepared from the same specification by two classes
of testers: experts and non-experts.
Qualitative analysis of the test suites indicated a lack of
structured process in their development.
The suites were analysed quantitatively by executing all valid
test inputs on five implementations seeded with faults and
the TTF-derived suite was the only one to identify all faults.
The most structured of the non-TTF suites identified more faults
than any other non-TTF suite, and it was found to be covered
by the TTF-derived suite.
This informal comparison
demonstrates that a systematic, formal approach can raise the
quality of suites produced by non-expert testers.
Conference Home Page