testAdequacy.pdf (131.01 kB)
Assessing Test Adequacy for Black Box Systems without Specifications
conference contributionposted on 2012-07-03, 13:27 authored by Neil Walkinshaw
Testing a black-box system without recourse to a specification is difficult, because there is no basis for estimating how many tests will be required, or to assess how complete a given test set is. Several researchers have noted that there is a duality between these testing problems and the problem of inductive inference (learning a model of a hidden system from a given set of examples). It is impossible to tell how many examples will be required to infer an accurate model, and there is no basis for telling how complete a given set of examples is. These issues have been addressed in the domain of inductive inference by developing statistical techniques, where the accuracy of an inferred model is subject to a tolerable degree of error. This paper explores the application of these techniques to assess test sets of black-box systems. It shows how they can be used to reason in a statistically justified manner about the number of tests required to fully exercise a system without a specification, and how to provide a valid adequacy measure for black-box test sets in an applied context.
CitationProceedings of The 23rd IFIP WG 6.1 International Conference on Testing Software and Systems (ICTSS), Lecture Notes in Computer Science, 7019, pp. 209-224
Author affiliation/Organisation/COLLEGE OF SCIENCE AND ENGINEERING/Department of Computer Science
SourceThe 23rd International Conference on Testing Software and Systems (ICTSS), Paris, 7-10 November 2011
- AM (Accepted Manuscript)