Limited Aspects of Reality: Frames of reference in language assessment

Language testers and educational measurement practitioners operate within two frames of reference: norm-referenced (NRT) and criterion-referenced testing (CRT). The former underpins the world of large-scale standardized testing that prioritizes variability and comparison. The latter supports substantive score meaning in formative and domain specific assessment. It has recently been claimed that the criterion-referenced enterprise is all but dead; its one legacy being the way in which test results are communicated (Davidson, 2012, p. 198). In this article, we argue that the announcement of the demise of CRT is premature. But we acknowledge that what for the most part passes as CRT is in fact not criterion-referenced, and is based upon a corruption of the original meaning of “criterion” as domain-specific performance. This distortion took place when NRT co-opted the term “standard” to serve as a rationale for the measurement enterprise of establishing cut-scores to retrofit NR tests with meaning derived from external scales. We argue that this is not CRT, and the true heirs of the CRT movement in applied linguistics are researchers who base test design in the careful analysis of construct and content in domain specific communication.