Alternative vocabularies in the test validity literature

Justification of testing practice involves moving from one state of knowledge about the test to another. Theories of test validity can (a) focus on the beginning of the process, (b) focus on the end or (c) encompass the entire process. Analyses of four case studies test and illustrate three claims: (a) restrictions on validity entail a supplement required to obtain justification from validity. (b) Rationales for restrictions assume particular contexts. (c) Claims can be translated between contrasting vocabularies. Implications for consumers of test validity theory include encouragement to focus on content instead of form and to write and read mindfully of the multiplicity of validity vocabularies. Implications for producers of test validity theory include encouragement to consider multiple reconstructions of a particular theory of test validity, clearly distinguish validity theories from validity definitions, and focus on contributing arguments that constrain possible theories rather than contributing definitions or broad frameworks.     

篮球从防守外线过渡到防守内线的技术及方法

阐述了篮球比赛内线防守的重要性,加强外线防守和内线防守能阻止对方将球传进或运进这个区域,使其减少投篮的机会。据此提出了从防守外线过渡到防守内线的技术及方法。     

Shifting the focus of validity for test use

The conventional focus of validity in educational measurement has been on intended interpretations and uses of test scores. Empirical studies of test use by teachers, administrators and policy-makers show that actual interpretations and uses of test scores in context are invariably shaped by local users’ questions, which frequently require attention to multiple sources of evidence about students’ learning and the factors that shape it, and depend on local capacity to use such information well. This requires a more complex theory of validity that can shift focus as needed from the intended interpretations and uses of test scores that guide test developers to local capacity to support the actual interpretations, decisions and actions that routinely serve local users’ purposes. I draw on the growing empirical literature on data use to illustrate the need for an expanded theory of validity, point to theoretical resources that might guide such an expansion, and suggest a research agenda towards these ends.     

Applying methods to evaluate construct validity in the context of A level assessment

Validity is a central principle of assessment relating to the appropriateness of the uses and interpretations of test results. Usually, one of the inferences that we wish to make is that the score reflects the extent of a student’s learning in a given domain. Thus, it is important to establish that the assessment tasks elicit performances that reflect the intended constructs. This research explored the use of three methods for evaluating whether there are threats to validity in relation to the constructs elicited in international A level geography examinations: (a) Rasch analysis; (b) analysis of processes expected and apparent when students answer questions; and (c) qualitative analysis of responses to items identified as potentially problematic. The results provided strong evidence to support validity with regard to the elicitation of constructs although one question part was identified as a threat to validity. Strengths and weaknesses of the methods can be identified.     

英语词汇量测试卷的编制及其信度与效度检验

本研究依据有关词汇测试的设计原理及模式,编制了词汇量测试卷,先后进行了两轮试测,运用SPSS18.0 ,对试卷项目进行筛选及修订,最终形成含104个题目的词汇量测试卷。信度、效度检验结果显示,试卷内在一致性信度Cronbach系数 ( 0 ． 918) 、 重测信度( 0 ． 644 ,p = 0 ． 000) 以及效标区分法效度( t = 6． 358 ,p= 0 ． 000) 、结构效度各level得分之间及总得分之间的相关性系数分别在 ( 0 ．068 ～ 0 ．496和 0 ．294 ～ 0 ．812)均达到测试学要求,本测试卷可作为新课改下非英语专业大学生的词汇量测评的有效工具。     

Examining the Predictive Validity of an Instrument: Technology Attitudes and Learning

This article reports the results of a study examining the predictive validity of a computer attitude instrument. The researchers attempted to determine the extent to which this instrument predicts student learning. Data from two universities were collected using this instrument over a nine-year period and were sorted into three sets with a random n of 400 in each. Three procedures were performed. First, one set of data was used to develop a base model of prediction. Second, this model was used to calculate the predicted learning achievement scores for the other two sets of data. Finally, in those two sets of data, the means of the predicted and observed learning achievement scores were compared using inferential statistics. The predictive validity of the instrument was confirmed, as no significant differences were found between the mean predicted and observed learning outcome scores.     

Argumentation Surrounding Argument-Based Validation: A Systematic Review of Validation Methodology in Peer-Reviewed Articles

Since it was formalized by Kane, the argument-based approach to validation has been promoted as the preferred method for validating interpretations and uses of test scores. Because validation is discussed in terms of arguments, and arguments are both interactive and social, the present review systematically examines the scholarly arguments which appear in 83 papers on argument-based validation methods published in peer-reviewed journals. Findings suggest that scholars generally agree on the nature and importance of argument-based validation but disagree on whether validation should be structured or unstructured, formal or informal. Implications are discussed, including promotion of the Standards for Educational and Psychological Testing (AERA, APA, and NCME) as a foundation for consensus in the field.     

A framework for evidencing assessment validity in large-scale,high-stakes international examinations

It is important for educational assessment bodies to demonstrate how they are seeking to meet the demands of validity. The approach to validity taken here assumes a ‘consequentialist’ view where the appropriacy of the inferences made on the basis of assessment results is seen as central. This paper describes the development of a systematic approach to the collection of evidence that can support claims about validity for general qualifications. An operational framework was developed drawing on Kane (2006 Kane, M.T. 2006. “Validation”. In Educational measurement, 4th ed., Edited by: Brennan, R.L. 17–64. Westport, CT: Praeger.  [Google Scholar]). The framework involves a list of inferences to be justified as indicated by a number of linked validation questions. For each question various data would be gathered to provide ‘evidence for validity’ and to identify any ‘threats to validity’. The structure is designed to be accessible for operational users. This paper describes the development of the proposed framework and the types of methods to be used to gather relevant evidence.     

怀疑论、认知资格与实用辩护

当代彻底怀疑论宣称关于外部世界的知识是不可能的。面对怀疑论的攻击,怀特发展了一条内在主义式的反怀疑论方案。基于传递失败和认知资格两个关键概念,该方案指出认知者接受类似我没有在做梦一类的反怀疑论命题是符合认知理性的,因此怀疑论攻击所造成的破坏是有限度的。有一派反对意见指出,该方案只是为接受反怀疑论命题给出了实用辩护,而怀疑论者则是要求认知辩护。但经过分析可以发现,批评者对于认知辩护与实用辩护的理解过于狭隘,而怀特的方案同时给出了实用辩护与认知辩护,也对怀疑论产生的根源做出了哲学治疗。     

Actual Interpretations and Use of Scores as Aspects of Validity

Validity is the most fundamental consideration in test development. Understandably, much time, effort, and money is spent in its pursuit. Central to the modern conception of validity are the interpretations made, and uses planned, on the basis of test scores. There is, unfortunately, however, evidence that test users have difficulty understanding scores as intended. That is, although the proposed interpretations and use of test scores might be theoretically valid they might never come to be because the meaning of the message is lost in translation. This necessitates pause. It is almost absurd to think that the intended interpretations and uses of test scores might fail because there is a lack of alignment with the actual interpretations made and uses enacted by the audience. Despite this, there has only recently been contributions to the literature regarding the interpretability of score reports, the mechanisms by which scores are communicated to their audience, and their relevance to validity. These contributions have focused upon linking, through evidence, the intended interpretation and use with the actual interpretations being made and actions being planned by score users. This article reviews the current conception of validity, validation, and validity evidence with the goal of positioning the emerging notion of validity of usage within the current paradigm.     

Same mindset,different goals and motivational frameworks: Profiles of mindset-based meaning systems

Growth and fixed mindsets have been linked to distinct effort beliefs, goals, and behaviours, creating a seemingly dichotomous pattern of motivation. Yet, students holding the same mindset are unlikely a homogenous group and may further differ in their motivational patterns. The current study employed a person-centred approach to investigate how mindsets and associated constructs naturally cohered and functioned together to influence student achievement. Data were collected from 535 English students (aged 14–16 years) on mindsets, effort beliefs, achievement goals, perseverance, and self-handicapping, along with their English and maths performance at the end of secondary school. Latent profile analyses revealed four distinct profiles. Across the profiles, students’ mindset co-varied with effort beliefs, mastery goals, perseverance, and self-handicapping, but the relationship between mindsets and performance goals was less straightforward. Two profiles supported the classic growth mindset–mastery goal (Growth-Focused) and fixed mindset–performance goal pairings (Ability-Focused). The other two profiles, however, displayed alternative combinations of mindsets and goals that had not been acknowledged in the past. Specifically, some growth mindset students embraced performance goals alongside mastery goals (Growth-Competitive), and some fixed mindset students did not endorse performance goals (Disengaged). The two growth-oriented profiles consistently performed well, and Growth-Competitive students even outperformed Growth-Focused students in maths. Compared to girls, boys were more often found in Ability-Focused and Disengaged profiles. The results indicate a nuanced set of relations between mindsets and achievement goals, highlighting the dynamic integration of motivational beliefs and goals within individuals.     

Taylorism,Tylerism and Performance Indicators: defending the indefensible?

This paper explores some of the antecedents to the recent growing interest in the United Kingdom in the use of educational performance indicators, and links it in particular to aspects of both Taylorist and Tylerist philosophies. It attempts to distinguish between different constructed meanings of performance indicators evident in both policy statements and practice. Whilst acknowledging the many potential problems inherent in the adoption of this approach to evaluation, the paper argues that both the nature and use of educational performance indicators are crucially shaped by factors such as purpose, authorship, focus and audience. The paper outlines an approach to performance indicators which tacitly addresses a critique of Taylorist and Tylerist influences, adopting an optimistic view that indicators can be made professionally relevant and useful. At the heart of the paper is an account of a collaborative evaluation project which, over the last 5 years, has supported the development of teacher‐generated indicators across some 80 schools and colleges in a consortium of six local education authorities. Despite the difficulties which have been encountered in this approach, the paper concludes that performance indicators can be made professionally relevant and useful.     

Reading and spelling acquisition in two different teaching methods: A test of the independence hypothesis

The development of word reading and word spelling was examined in French speaking children initially instructed either by a phonic or a whole-word method. Second, fourth and sixth graders were administered to reading and spelling tests in which grapho-phonological regularity, frequency, length and lexicality were manipulated. The results showed that in both curricula, reading and spelling acquisition can be characterized by a parallel increase in the use of sub-lexical correspondences and in the reliance on word-specific information. Contrary to a simple view of lexical development according to which the use of analytical knowledge and the use of word-specific knowledge correspond to two different cognitive processes that develop independently from each other, whole-word children did not appear to rely more on whole-word knowledge. On the contrary, and paradoxically, grade 2 whole-word children tended to use analytical correspondences to a greater extent than their peers. In later development, reading matched phonic and whole-word groups did not differ from each other. It is argued that the results support the hypothesis that the acquisition of sub-lexical correspondences constitutes a necessary step in the acquisition of reading and spelling. We conclude that the analytic comparison of different curricula provides a naturalistic tool for the study of the dynamics of development.     

Keeping your audience in mind: applying audience analysis to the design of interactive score reports

Score reports have one or more intended audiences: the people who use the reports to make decisions about test takers, including teachers, administrators, parents and test takers. Attention to audience when designing a score report supports assessment validity by increasing the likelihood that score users will interpret and use assessment results appropriately. Although most design guidelines focus on making score reports understandable to people who are not testing professionals, audiences should be defined by more than just their lack of statistical knowledge. This paper introduces an approach to identifying important audience characteristics for designing computer-based, interactive score reports. Through three examples, we demonstrate how an audience analysis suggests a design pattern, which guides the overall design of a report, as well as design details, such as data representations and scaffolding. We conclude with a research agenda for furthering the use of audience analysis in the design of interactive score reports.     

影像文化：特征、功能与意义

影像化是伴随着现代高科技的发展而诞生的全新的化形态，它的最大贡献是打破了亘古以来传统化的生产、流通及消费方式，科技与市场的介入而有力地推动了影像化的发展，影像化又以它特有的兼容性、共时性、开放性等优势，对现代社会的政治、经济、化、教育等产生了广泛而深远的影响。