Impact of Changing Difficulty on Inferences From the National Assessment of Educational Progress

The U.S. Department of Education measures student achievement through the National Assessment of Educational Progress (NAEP). NAEP estimates of population proficiency quantiles are based on a Bayesian multiple-imputation procedure. This article shows (a) that the resulting estimates depend directly on the mix of item difficulties on the test, and (b) the difficulty of items on the NAEP mathematics exam has increased over time. Does the increasing difficulty of the exam lead to observable changes in student performance over time? This study compared the simulated performance of 1990 examinees on the easier 1990 exam and the more difficult 1996 exam. No significant differences were found. While our results instill confidence that these changes have not impacted the NAEP trend line, our findings are both data-specific and limited in scope, and NAEP should carefully evaluate future adjustments to the test in this manner.     

Effects of Motivational Interventions on the National Assessment of Educational Progress Mathematics Performance

One of the reasons often cited h r the low average level of proficiency demonstrated by U.S. students on national and international assessments is that there are no consequences or stakes attached to performance on the tests and, therefore, students are not motivated to invest their best effort. In this study, money was chosen as an incentive, but we hoped that short written instructions would be almost as powerful as money and easier and more desirable to implement in the National Assessment of Educational Progress (NAEP). Our results indicate that, at least for Grade 8 participants, student effort can be increased by financial rewards offered at the time of test taking, and that such effort can result in an increase in NAEP math test scores. Thus, from a policy perspective, scores from low-stakes tests may not represent what the student knows. Rather, such scores represent what students will demonstrate with minimal effort     

美国全国教育进步评价(NAEP)中的科学探究

1NAEP简介美国全国教育进步评价(NAEP)是由美国国会授权并赞助,具有全国代表性的,连续评价美国学生在数学、科学、阅读、写作、艺术、美国历史、世界地理以及其他一些学科的成绩的机构。NAEP开展全国性评价工作已有二十多年的历史。目前,NAEP已成为美国国家及其各州政府衡量其教育发展状况的一个重要指标。1996年NAEP推出了新的1996-2000科学学科评价框架,该框架由科学领域与认知要素构     

教育投入对学生发展的影响——对西发项目影响力评价研究设计的反思

促进学生发展是大多数教育投资项目的终极目标,但由于学生发展受其自身特征以及外部社会环境因素的影响,往往很难确定教育项目干预对学生发展所产生的真正影响.影响力评价是许多国际组织以及发达国家在评价公共政策时通常选用的一种较为科学、严密的评价方法,该方法通过有效剥离其他因素对结果产生的影响,获得政策"净影响效应"."西部地区基础教育发展"项目影响力评价,是在较为完备数据库支持下,中国教育领域第一次运用比较严密的研究设计,利用影响力评价方法就教育投资项目对学生发展所产生影响进行的评价.针对当前中国教育投资项目多以简单描述性评价为主的现状,为了保证促进学生发展这一教育投资项目目标最大程度的实现,项目管理部门必须重视引入和使用影响力评价的理念和方法,提高评价结果的科学性和有效性.     

美全国教育进展评估述评

为了了解全美中小学生在核心课程中的学习成绩,美国建立了全国教育进展评估.本文将就全国教育进展评估建立的背景及发展历程、全国教育进展评估的数学和阅读评估框架、2007年最新评估结果以及这一评估项目的成绩和问题展开论述.     

美国国家教育进展评估

在美国,国家教育进展评估(NAEP)也被称为"国家成绩单"。2001年《一个孩子都不能掉队》(NCLB)法案通过之后,国家教育进展评估成为衡量全国、各州和越来越多的大城市教育进展的一个重要工具。《一个孩子都不能掉队》法案规定四年级和八年级学生应该参加国家和州级的阅读和数学测试,同时,州级的科学和写作测试也享有很高的优先权。此外,国家教育进展评估也对其他科目,包括公民学、历史和经济学进行不定期测试。然而,在测试艺术和外语方面,国家教育进展评估并未取得成功。本文介绍了美国国家教育进展评估的主要内容,同时探讨了这一评估目前所面临的挑战。     

Describing Performance Standards: Validity of the 1992 National Assessment of Educational Progress Achievement Level Descriptors as Characterizations of Mathematics Performance

In this article, I examined the proposed role of "authentic" assessment as a policy tool of educational reform from two broad, closely related perspectives-technological and histo1,ical. I argue that one needs to first view testing as a technology. Next, when evaluating proposed policies to create a national examination system, one should examine their feasibility by considering the history of technological developments in testing and the reasons for those developments. Too often we have overlooked the lessons of history in formulating high-stakes testing, policies. In evaluating such testing policies, we need to look back while looking forward. When examined from the technological and historical angle, the practicality of current proposals for the large-scale use of performance measures for certifying an individual's successful completion of a give^ level of education, or for making decisions about an individual's entrance to college or the workplace, is certainly questionable.     

美国2010年全国教育进展评估述评

全国教育进展评估是美国最具有代表性的、持续性的教育质量评估,它对中小学生各学科知识和技能掌握情况进行测评。2010年美国全国教育进展评估的学科包括历史、地理和公民学,评估的结果为我们了解这三个学科的学习状况提供了重要的依据。而对此次评估的具体分析阐述也为我们评析美国全国教育进展评估的优点和不足提供了依据。     

Contention and Consensus: The Development of the 1992 National Assessment of Educational Progress in Reading

Because of plans for state-by-state reporting of 1992 reading data from the National Assessment of Educational Progress (NAEP), we investigated the adequacy of the process used to develop the assessment, the degree to which it represents a consensus among professionals in the reading field, and its content and curricular validity. To carry out this investigation, we analyzed documents produced by NAEP, convened a 2-day panel of experts, held two public colloquia, conducted 50 interviews, and analyzed responses to a questionnaire completed by 627 leading educators. We found that the planning process did not include enough time to address some major concerns of the field. Despite this, there was widespread agreement that the 1992 NAEP in Reading represents important advances in reading assessment, including more open-ended responses, more authentic texts, and student choice about passages. But these very advances raise problems for test design and the interpretation and scoring of student responses.     

国家教育进展评估的效度研究

作为一项得到广泛认可的教育绩效指标,国家教育进展评估(NAEP)是美国数十年来用于跟踪和了解教育进展的重要工具,也是全美初等教育与中等教育状况的晴雨表。它是美国当前唯一一项定期对小学、初中和高中学生的教育成就进行的全国性调查,在新测试技术的发展过程中发挥着重要的作用。本文对NAEP的发展历史进行综述和总结,同时对当前NAEP所面临的效度问题,如跨年级(纵向)量表以及在分数报告中使用表现水平的做法等进行评论。     

The Impact of Registration Timing on Student Performance

This study compares the impact of timing of registration on the student learning outcomes of students taking courses at three rural community colleges in the southeastern U.S. during the school years 2001–2003. Findings from this study indicate that early registration has a positive influence on students' grades and course completion rates. Also contributing to differences in student outcomes were student race, Pell Grant status, gender, program of study, and age.     

美国国家教育进展评估（NAEP）体系的产生与发展

美国国家教育进展评估是目前美国国内惟一连续、长期的中小学生学业成绩测量体系。文章从历史的角度出发，对该评估体系的产生、发展与现状进行综述。     

NAEP2010公民教育评估框架述评

全国教育进步评估（NAEP）是由美国国会授权的一项调查,以便收集和报告学生各门学科的成绩信息。自1969年以来,公民教育就是其评估项目之一,为了进一步了解4、8、12年级学生公民教育情况,NAEP2009年制订了2010年公民教育评估框架,计划于2010年对全美学生进行公民教育评估。文章主要介绍了2010年公民教育评估框架产生的背景、评估框架设计中考虑的问题以及评估框架的组成部分,以便为我国公民教育评估框架的研制提供借鉴。     

Using Assessment Metadata to Quantify the Impact of Test Disengagement on Estimates of Educational Effectiveness

Abstract

Educational stakeholders have long known that students might not be fully engaged when taking an achievement test and that such disengagement could undermine the inferences drawn from observed scores. Thanks to the growing prevalence of computer-based tests and the new forms of metadata they produce, researchers have developed and validated procedures for using item response times to identify responses to items that are likely disengaged. In this study, we examine the impact of two techniques to account for test disengagement—(a) removing unengaged test takers from the sample and (b) adjusting test scores to remove rapidly guessed items—on estimates of school contributions to student growth, achievement gaps, and summer learning loss. Our results indicate that removing disengaged examinees from the sample will likely induce bias in the estimates, although as a whole accounting for disengagement had minimal impact on the metrics we examined. Last, we provide guidance for policy makers and evaluators on how to account for disengagement in their own work and consider the promise and limitations of using achievement test metadata for related purposes.     

美国《2017年NAEP写作评估框架》述评

《2017年NAEP写作评估框架》由美国国家评估监管委员会设计,它将写作视为与他人进行沟通交流的方式,强调写作的交际目的与读者的重要性。框架的评估标准是观点的发展、观点的组织以及语言能力和使用规范,结果主要通过等级分数、成就水平和学生写作概况这三种方式呈现。总体来说,《2017年NAEP写作评估框架》具有基于认知过程理论和社会文化理论、强调读者意识和指向大学与就业等特点。