Alternative vocabularies in the test validity literature

Justification of testing practice involves moving from one state of knowledge about the test to another. Theories of test validity can (a) focus on the beginning of the process, (b) focus on the end or (c) encompass the entire process. Analyses of four case studies test and illustrate three claims: (a) restrictions on validity entail a supplement required to obtain justification from validity. (b) Rationales for restrictions assume particular contexts. (c) Claims can be translated between contrasting vocabularies. Implications for consumers of test validity theory include encouragement to focus on content instead of form and to write and read mindfully of the multiplicity of validity vocabularies. Implications for producers of test validity theory include encouragement to consider multiple reconstructions of a particular theory of test validity, clearly distinguish validity theories from validity definitions, and focus on contributing arguments that constrain possible theories rather than contributing definitions or broad frameworks.