Validity on Trial: Psychometric and Legal Conceptualizations of Validity

The psychometric literature is replete with comprehensive discussions of test validity, test validation, and the characteristics of quality assessment programs. The most authoritative source for guidance regarding sound test development and evaluation practices is the Standards for Educational and Psychological Testing. However, the Standards are not legally binding. In this article, we review the way in which validity is conceptualized in the Standards and compare this conceptualization with validity evidence presented in specific court cases involving legal challenges to tests. Our review indicates that, in general, there is strong congruence between the Standards and how validity is viewed in the courts, and that testing agencies that conform to these guidelines are likely to withstand legal scrutiny. However, the courts have taken a more practical, less theoretical view on validity and tend to emphasize evidence based on test content and testing consequences.     

A Historical Perspective and Current Views on the Standards for Educational and Psychological Testing

The Standards for Educational and Psychological Testing have evolved in the breadth and depth of coverage of issues in educational testing and measurement since their first publication in 1954. There were a number of substantive changes in the 1999 revision that addressed validity, fairness, accommodations, and compliance with the Standards. In addition, there was nearly a 50% increase in the number of standards contained in the last revision. The next revision of the Standards may be initiated in 2007 and there are remaining concerns about access and awareness by non-measurement professionals, compliance by test publishers and users, relevance in addressing mandates for accountability, and substantive areas of educational assessment. This review of major changes to the Standards and discussion of future topics is designed to inform the next revision.     

Social Studies Education in the Age of Testing and Accountability

With 3 years and counting since its inception, the scope and impact of No Child Left Behind is now being felt in classrooms across the nation. Although some successes have been identified, concerns about the implementation and expectations of the legislation are emerging. As a result of the legislation's emphasis on the development of standards and assessment systems in reading, mathematics, and science, educators of other core and elective subjects are weighing the impact of being left out of the No Child Left Behind formula of testing and accountability. In an attempt to ascertain the perceptions and sentiments of social studies teachers concerning the impact of No Child Left Behind and the concomitant issue of state-level testing and accountability policy, a tristate study was conducted. The findings of this research indicate that although elementary, middle, and high school social studies educators recognize the negative consequences of being left out of No Child Left Behind, they are not convinced that being included would be better for them, their students, or social studies education in general.     

ROM效应与歧义优势和劣势

歧义词在词汇判断任务中表现出的加工优势以及在语义任务中表现出的加工劣势常被研究者们统称为歧义效应。歧义效应的产生究竟是因为歧义词正字法与其语义表征之间联系的本质不同于非歧义控制词正字法与其语义表征之间联系的本质,还是因为词汇判断任务和语义任务所强调的加工不同,亦或是因为实验材料引起的决策冲突以及决策类型的差异等问题引起了研究者的极大兴趣。语义关联度（relatedness of meaning,简称ROM）效应的发现为澄清歧义效应研究中的一些不一致现象,深入探讨歧义效应产生的根源,了解词汇心理表征的本质以及更为严密的操控试验等提供了崭新的视角。     

指导教学的究竟是什么--兼及"论"与"思"的思考

指导教学的究竟是什么,不是“论”也不是“思”,而是个人化的教学实践知识。它是从内涵上对前两者的统摄与超越,是一种协调着科学化追求与人文关怀的实践理性。教学实践知识的提出对于促进广大教师在当前新课程改革背景下自身教学行为的积极转变,将具有明显的现实指导意义。     

Mental testing and educational streaming in Ontario and Denmark in the early twentieth century: a comparative and transnational perspective

ABSTRACT

This article compares and contrasts the use of mental testing and the formation of educational streaming in Denmark and Ontario during the interwar years. In this sense, the article adds nuances to the meaning of internationalism as well as contributing to our knowledge about how ideas of testing practices circulated among countries and continents. One way ideas and practices circulated was via informal networks promoted by the education traveller. Key proponents of mental testing in both Denmark and Ontario travelled to continental Europe, England, and the United States studying and observing the practices and institutional arrangements associated with educational streaming. Our main findings are that the processes used to implement mental testing in the two countries differed significantly. Mental testing was implemented much later in Denmark than in Ontario. This was due to different contextual, cultural, and historical factors that promoted changes to the existing system, or, alternatively, represented a barrier or even obstructed changes to it. Nevertheless, mental testing was implemented in both education systems as a relatively coherent technology rooted in transnational movements and exchange, but was attended by highly different practices and local meaning-making.     

Standardisation and diversity in international assessments: barking up the wrong tree?

ABSTRACT

This article organises potential areas of criticism or challenges embedded in the design and administration of standardised assessments of learning levels in order to promote dialogue and research on educational assessments. The article begins by addressing debates around epistemological claims: issues that pertain to testing in general and issues that are particular to standardised testing. Then, it addresses some political attributes of international tests so as to situate the debates beyond feasibility, attributes and scope-related issues. The article claims that the field of education testing has identified a number of issues and challenges stemming from diversity, and has developed methods and procedures to address many of them. From this viewpoint, testing is just like any other domain of scientific enquiry. However, international assessments of learning outcomes are not necessarily, or primarily, scientific endeavours; they are political devices and therefore should be scrutinised considering scientific attributes as well as some political features that, even if intertwined with technicalities, go well beyond them. Thus, critiques of international assessments would be better framed if their political attributes are taken as organising principles of the criticism, alongside those elements that pertain to their technical attributes, since these are not incidental but deeply interlinked.     

Standardized Testing + High-Stakes Decisions = Educational Inequity

Changes in assessment policy have increased standardized testing at provincial, national, and international levels, introduced testing at more grade levels, increased the reporting of test results, and attached more significance to those results. Advocates claim that testing will result in greater accountability in education. The research demonstrates that standardized testing has a negative impact on students, perpetuating and intensifying educational inequity through test bias and the misuse of test scores. Test results are increasingly being used to analyse policy, program, school, and teacher success, and they are being inappropriately used as "educational gatekeepers" to make important decisions about students, teachers, schools, and school systems. This paper focuses on how standardized testing is becoming the mechanism that facilitates many questionable education practices that contribute to educational inequity.     

High stakes testing and teacher access to professional opportunities: lessons from Indonesia

Abstract

High-stakes testing regimes, in which schools are judged on their capacity to attain high student results in national tests, are becoming common in both developed and developing nations, including the United States, Britain and Australia. However, while there has been substantial investigation around the impact of high-stakes testing on curriculum and pedagogy, there has been very little research looking at the impact on teachers’ professional opportunities. The current project used a case study approach to examine the impact a high-stakes national testing programme had on teachers’ access to professional learning and their teaching allocations in four Indonesian public schools. It found that better qualified teachers were allocated to classes that would be sitting for the national examinations, and that these teachers were given much more access to professional learning opportunities than those teaching non-examined year levels. This in turn impacted negatively on the staff morale of less qualified teaching staff and potentially on their career trajectories. Findings suggest that school leaders should be wary of targeting better qualified and/or more experienced staff to year levels sitting for high-stakes tests, as this may lead to staff stratification within schools, limiting opportunities for staff to learn from one another and reducing the morale of less qualified and less experienced staff. They also add support to a substantial body of research that suggests policy-makers should be wary of the flow-on effects of using performance in high-stakes tests as the key means of judging school effectiveness.     

Using the Standards to Evaluate the Redesign of the U.S. Naturalization Tests: Lessons for the Measurement Community

In 2001, the U.S. Citizenship and Immigration Services (USCIS) began the process of redesigning the U.S. naturalization tests due to concerns that the current testing procedure may not be sufficiently uniform, and that the test content may not be appropriate. A National Research Council committee issued recommendations, based largely on the Standards , to ensure that the redesigned tests would be scientifically and politically defensible; however, USCIS chose not to follow much of the committee's advice. This article summarizes some lessons for the measurement community and possible strategies for getting policymakers to pay more attention to the Standards .     

Common Core State Standards,Writing, and Students with LD: Recommendations

This article examines the Common Core State Standards as they apply to writing and students with learning disabilities (LD). We first consider why the implementation of these standards is advantageous to writing instruction for students with LD as well as the challenges in implementing them. Next, we make the following four recommendations in terms of their implementation: (1) increase general and special education teachers’ knowledge about writing development; (2) create a writing environment in which students with LD can thrive; (3) employ evidence‐based writing practices in general education classes (where most students with LD are taught); and (4) use evidence‐based writing practices effective with students with LD. We conclude by considering research that still needs to be undertaken to help educators maximize the probability that students with and without LD meet the writing benchmarks proposed in these Standards.     

高考改革的目标模式与推进策略

高校招生和人的发展的现实需要与现行高考"大一统"之间存在的巨大反差,是当前高考存在的最大的矛盾。解决这一矛盾的高考改革的目标模式为建立统一的综合测试平台,按高校专业大类设定科目组合,统一测试与自主测试相结合选拔特殊人才,高职院校招生采用灵活多样的考试方式,加强艺术类、体育类专业测试,确立综合评价的原则,形成多元录取机制。此目标模式可以优化人才选拔功能,使高校拥有更多的招生自主权;提高高考与普通高中新课程的吻合度,对实施素质教育发挥良好的导向作用;提供高考的个性选择空间,为学生个性化发展创造条件;理顺共性考查与个性考查的关系,兼顾统一性与多样性。高考改革目标模式的推进策略为继续推进高水平大学自主招生试点,逐步扩大自主招生的范围和规模;把高职院校考试招生改革作为大步推进高考改革的突破口;加强普通本科院校分类考试招生的研究和探索;逐步增加招生考试通道,推进"综合测试平台"建设;明确改革方向,形成高考改革的合力。     

标准化控制对CSY系列传感器与检测技术实验的成效影响

现实的实验教学效果结合务实的实验技艺改进,将以"稳妥环节、优化全局"的标准化控制模式,逐步深入并推进到具有典型"实验特色、标准特性"的传感器测试技术实验教学领域中,取得"质量把脉、基准全程"的标准化良好实验行为效果。以用"CSY产品系列传感器与检测技术"实验台为实践平台和简述背景,就实验过程中标准化排序控制和标准化理念控制对实验全过程产生的影响简要诠释,以及对实验器材的标准化定性问题进行归纳。     

Implications of cognitive psychology for educational testing

The purpose of this review is to examine views about learning and the learner grounded in cognitive psychology and to discuss their implications for educational testing. The new perspectives about learning and the learner suggested by cognitive psychology imply changes in educational testing. The first part of this paper compares perspectives about learning and the learner in cognitive psychology with traditional perspectives. The second part describes its implications for educational testing. The goals of educational testing should be the improvement of learning and instruction, achieving these goals in the real society, and a symbiotic relationship between cognitive psychologists and traditionalists in educational testing.     

Interpretations of graphs by university biology students and practicing scientists: Toward a social practice view of scientific representation practices

Using graphs is a key social practice of professional science. As part of a research program that investigates the development of graphing practices from elementary school to professional science activities, this study was designed to investigate similarities and differences in graph‐related interpretations between scientists and college students engaged in collective graph interpretation. Forty‐five students in a second‐year university ecology course and four scientists participated in the study. Guided by domain‐ specific concerns, scientists' graph‐related activities were characterized by a large number of experience‐based, domain‐specific interpretive resources and practices. Students' group based activities were characterized by the lack of linguistic distinctions (between scientific terms) which led to ambiguities in group negotiations; there was also a lack of knowledge about specific organism populations which helped field ecologists construct meaning. Many students learned to provide correct answers to specific graphing questions but did not come to make linguistic distinctions or increase their knowledge of specific populations. In the absence of concerns other than to do well in the course, students did not appear to develop any general interpretive skills for graphs, but learned instead to apply the professor's interpretation. This is problematic because, as we have demonstrated, there are widely differing viable interpretations of the graph. Suggestions for changes in learning environments for graphing that should alleviate this problem are made. © 1999 John Wiley & Sons, Inc. J Res Sci Teach 36: 1020–1043, 1999     

The Impact of Reform: Challenges for Mathematics Teacher Preparation

The mathematics education community has been widely recognized as a leader in the standards-based reform movement. Despite the widespread interest and attention that reform documents such as the NCTM Standards have generated, what has yet to be fully understood is the impact of these types of reform documents on the preservice teacher preparation process. This paper examines the state of the standards-based reform effort in mathematics teacher education by presenting the results of a three-year study of six cohorts of secondary mathematics student teachers (n = 63). Examined in particular are the ways in which these student teachers' professed beliefs about and knowledge of the NCTM Standards contrast with their teaching practices. Possible explanations for the mismatch between belief and knowledge statements and teaching practices are examined, along with other implications for mathematics teacher preparation.     

Poetry education research as an anchorage of thought: using poetry as interview stimulus material

Interviews in qualitative research may sometimes employ stimulus material as a means of eliciting richer data. However, scant consideration has been given to the use of poetry for this purpose, especially within the field of poetry education research. This article seeks to address the gap in the literature by illustrating how the use of poetry as interview stimulus material can help poetry education research to clarify the connection between teachers’ and students’ beliefs and practices in relation to poetry and pedagogy. Depending on the nature of their inquiry and context, poetry education researchers can choose relevant poems that have the potential to stimulate a discussion about specific beliefs and practices.     

Test Preparation: Examining Teacher Perceptions and Practices

This study analyzed questionnaire and interview data on teachers' practices and perceptions with respect to test preparation. Questionnaire respondents were asked to rate the ethicality of various test-preparation practices and indicate the extent to which they utilize these practices in their instruction. On the basis of questionnaire results, interviews were conducted with a smaller sample of teachers to determine their views on the appropriateness of particular test-preparation practices, and to determine the factors affecting teacher perceptions about a given activity. Contrary to previous empirical work, questionnaire results indicated that neither use of a given practice nor teacher perceptions of the ethicality of the practice vary across levels of student achievement. On the other hand, consistent with previous empirical work, both use and perceptions varied across grade-level configuration. Estimates of the prevalence of particular teacher practices and perceptions were obtained and compared with those from the literature. In addition, dimensions of teacher reasoning were explored, indicating that when considering the appropriateness of a given practice, teachers consider the following factors: score meaning, learning, the potential for raising student scores, professional ethics, equity, and external perceptions.     

朱熹的知识论诠释学和陆九渊的实践论诠释学

朱熹理学把理解的目标规定为"穷理",而"格物"和"读书"这两种外求的方式,"从个别到一般"从特殊到普遍"的理解方法成为达成这一目标的重要保证,"知先行后"说更是把理解限定在知识论范围.朱熹理解理论的视野是属于知识论诠释学的;陆九渊心学则转向了"发明本心"以"穷理"的思路,内求成为"明理"的关键,"从一般到个别"从普遍到特殊"的理解方法成为达成"明理"这一理解目标的重要保证,"知行合一"说凸显了理解中的实践智慧.陆九渊意义上的"明理"具有了实践论的意味,其理解理论的视野是实践论诠释学的.     

Disagreement over the best way to use the word ‘validity’ and options for reaching consensus

The ability to convey shared meaning with minimal ambiguity is highly desirable for technical terms within disciplines and professions. Unfortunately, there is no widespread professional consensus over the meaning of the word ‘validity’ as it pertains to educational and psychological testing. After illustrating the nature and extent of disagreement, we consider three options for reaching consensus: to eliminate its ambiguity by agreeing a precise technical definition; to embrace its ambiguity by agreeing a catchall lay usage; and to retire ‘validity’ from the testing lexicon.