Evaluating Content Alignment in Computerized Adaptive Testing

The alignment between a test and the content domain it measures represents key evidence for the validation of test score inferences. Although procedures have been developed for evaluating the content alignment of linear tests, these procedures are not readily applicable to computerized adaptive tests (CATs), which require large item pools and do not use fixed test forms. This article describes the decisions made in the development of CATs that influence and might threaten content alignment. It outlines a process for evaluating alignment that is sensitive to these threats and gives an empirical example of the process.     

Methodological Choices in the Content Analysis of Textbooks for Measuring Alignment With Standards

With the recent adoption of the Common Core standards in many states, there is a need for quality information about textbook alignment to standards. While there are many existing content analysis procedures, these generally have little, if any, validity or reliability evidence. One exception is the Surveys of Enacted Curriculum (SEC), which has been widely used to analyze the alignment among standards, assessments, and teachers’ instruction. However, the SEC can be time‐consuming and expensive when used for this purpose. This study extends the SEC to the analysis of entire mathematics textbooks and investigates whether the results of SEC alignment analyses are affected if the content analysis procedure is simplified. The results indicate that analyzing only every fifth item produces nearly identical alignment results with no effect on the reliability of content analyses.     

First Language of Test Takers and Fairness Assessment Procedures

Over the past few decades, those who take tests in the United States have exhibited increasing diversity with respect to native language. Standard psychometric procedures for ensuring item and test fairness that have existed for some time were developed when test‐taking groups were predominantly native English speakers. A better understanding of the potential influence that insufficient language proficiency may have on the efficacy of these procedures is needed. This paper represents a first step in arriving at this better understanding. We begin by addressing some of the issues that arise in a context in which assessments in a language such as English are taken increasingly by groups that may not possess the language proficiency needed to take the test. For illustrative purposes, we use the first‐language status of a test taker as a surrogate for language proficiency and describe an approach to examining how the results of fairness procedures are affected by inclusion or exclusion of those who report that English is not their first language in the fairness analyses. Furthermore, we explore the sensitivity of the results of these procedures, differential item functioning (DIF) and score equating, to potential shifts in population composition. We employ data from a large‐volume testing program for this illustrative purpose. The equating results were not affected by either inclusion or exclusion of such test takers in the analysis sample, or by shifts in population composition. The effect on DIF results, however, varied across focal groups.     

Validity Considerations in Designing an English Reading Test

Validity is one of the essential factors in evaluating test quality. In order to design a high-quality reading test paper, such aspects should be taken much more considerations as test specification, theoretical construct, test content and test tasks & item types.     

英语专业八级口译测试内容效度研究

专八口译测试是一项针对于英语专业学生的水平考试,测试英语专业学生的口译能力.为了更加有效地评估英语学习者的口译能力和口译水平,必须确保该项考试的内容效度.该文将对历年来专入口试的考题进行全面分析,基于效度理论中的相关性、代表性、真实性研究理论,对专八口试的内容效度进行研究,并给出一些自己的观点与建议,希望对大学英语口译课程的教与学有所帮助.     

Inclusive Achievement Testing for Linguistically and Culturally Diverse Test Takers: Essential Considerations for Test Developers and Decision Makers

Substantial growth in the numbers of English language learners (ELLs) in the United States and Canada in recent years has significantly affected the educational systems of both countries. This article focuses on critical issues and concerns related to the assessment of ELLs in U.S. and Canadian schools and emphasizes assessment approaches for test developers and decision makers that will facilitate increased equity, meaningfulness, and accuracy in assessment and accountability efforts. It begins by examining the crucial issue of defining ELLs as a group. Next, it examines the impact of testing originating from the No Child Left Behind Act of 2001 (NCLB) in the U.S. and government‐mandated standards‐driven testing in Canada by briefly describing each country's respective legislated testing requirements and outlining their consequences at several levels. Finally, the authors identify key points that test developers and decision makers in both contexts should consider in testing this ever‐increasing group of students.     

大学英语B受试者情感状况和学习行为调查与分析

本文采用文献调研法、问卷法和访谈法,以2010年12月全国大学英语B网考某电大本部及各分考场966名考生为研究对象,从语言测试反拨效应角度,研究远程英语学习者的情感状况和学习行为。研究结果表明:1)大学英语B网考给受试者带来了负面影响,造成一定程度上的情感冲击和学习行为偏差;2)考试失败次数越多,产生的负面效应越大;3)电大和网院两类不同学校的学习者在自我英语水平评价、学习困难和对面授课评价方面存在显著性差异,电大学生的自我英语水平评价更高、遭遇的学习困难较少、对面授课评价更高;4)男女学习者之间仅在对学习困难评价上存在显著性差异,男生认为遭遇的学习困难更多;5)文理两类专业学习者之间不存在显著性差异。文章最后建议相关教育部门和机构应采取切实措施减少大规模高风险考试给远程英语学习者带来的负面效应。     

激光准直测量系统的运用和常见故障处理

基准线法是观测直线型建筑物水平位移的重要方法,文章介绍了捣固车激光准直测量系统的组成、工作原理和作用,重点介绍了激光准直测量系统的操作方法,总结激光准直测量系统常见故障的产生原因、处理方法以及激光准直测量系统的维护保养。     

Rater Cognition: Implications for Validity

The scoring process is critical in the validation of tests that rely on constructed responses. Documenting that readers carry out the scoring in ways consistent with the construct and measurement goals is an important aspect of score validity. In this article, rater cognition is approached as a source of support for a validity argument for scores based on constructed responses, whether such scores are to be used on their own or as the basis for other scoring processes, for example, automated scoring.     

Flexibly Using the Surveys of Enacted Curriculum to Study Alignment

As the standards movement continues into its third decade, there remains a need for alignment methodologies that can be broadly applied to study instruction and policy. This article reports on a series of development efforts meant to revise the Surveys of Enacted Curriculum (SEC) surveys and methods to study the implementation of new college- and career-readiness standards. The work included a meeting of content experts, a series of cognitive interviews, two validation studies, and a small pilot. We discuss both the results of the specific studies and the implications of the work for other potential users of the SEC or SEC-like tools.     

Maximizing Learning Through Course Alignment and Experience with Different Types of Knowledge

Consistency among the objectives, learning activities, and assessment exercises results in aligned courses, which give students direction and clarity and yield increased learning. However, instructors may not check for course alignment. This article describes a concrete way to determine course alignment by plotting the course components on a table using the cognitive process levels from a revised taxonomy of learning objectives. Once instructors realize that courses are misaligned, they can make adjustments. By giving students experience with varied types of knowledge, which is the other part of this taxonomy, they also learn more. The types of knowledge include factual, conceptual, procedural, and meta-cognitive knowledge. Phyllis Blumberg received her A.B. in Psychology from Washington College (MD), her M.A. and Ph.D. both in educational and developmental psychology from the University of Pittsburgh. She is a Professor and Director of the Teaching and Learning Center at the University of the Sciences in Philadelphia. Her research interests include learning-centered teaching, self-directed learning and problem-based learning.     

基于价值链的中小物流企业联盟研究

随着社会生产分工不断深化,企业之间建立动态联盟顺应了产品空间的分化与市场空间价值链一体化的需要。我国物流业缺少物流巨头,除了屈指可数的几家企业初具规模且有稳定的客户群外,大多数物流企业规模小、利润低、客户少、人才短缺、资金匮乏、信息封闭。中小物流企业自身资源有限,品牌可信度不高,在日益激烈的竞争环境中应采取各种联盟方式共求发展。     

CET-SET测试效度研究

本研究就CET-SET(大学英语四、六级口语考试)测试效度作了相应的实证研究,研究结果表明CET-SET测试任务类型的结构效度还不完善,不能完成测试目的与测试结果的拟合(Hughes,1989),证明了CET-SET结构效度偏低的事实。针对研究结果,研究者提出了提升测试效度相关的建议和措施。     

TEM 8成段改错试题的效度分析

文章结合2001-2006年的TEM 8成段改错试题,采用定性研究验证成段改错的效度。通过试题分析及内容效度和结构效度分析,认为TEM 8成段改错试题的内容效度不完善,没有体现当前新的交际语言能力观。最后提出建议认为应该在设计试题时,把重点放在意义的理解和辨别上,并应增加对于文本的得体性的考察。     

以“同盟”范畴谈师生课堂互动

文章基于国内外学者对于“同盟”范畴的研究,运用于解释课堂场景的过程和基础教育和高校教育的倾向性。文章通过分析传统的“同盟”范畴理念,提出了“同盟”范畴下的功能性概念:认识功能、态度功能和风格功能,并将其运用于对课堂师生互动行为的识解,提出了师生课堂活动中的“功能-课堂互动”对照结构,分析基础教育和高校教育的倾向性问题。     

Alignment of Mathematics State-Level Standards and Assessments: The Role of Reviewer Agreement

This article examines the role of reviewer agreement in judgments about alignment between tests and standards. We used case data from three state alignment studies to explore how different approaches to incorporating reviewer agreement changes alignment conclusions. The three case studies showed varying degrees of reviewer agreement about correspondences between objectives and test items. Moreover, taking into account reviewer agreement in the analyses sometimes had a marked effect on alignment conclusions. We discuss reasons for differences across case studies and alignment approaches, as well as implications for future alignment efforts.     

美国“SEC”一致性分析范式的诠秤与启示——基础教育中评价与课程标准一致性的视角

美国在基于课程标准的教育改革以及学校教学要求偏低的现实背景下,致力于研制“SEC”（Surveys of Enacted Curriculum）等一致性分析范式,即分析评价与课程标准一致性程度的理念、程序和方法,推动了美国学校基于课程标准的评价实践。该范式对我国课程评价的启示在于：要认识到评价与课程标准一致性策略是调适课程运行偏差的重要手段;要立足本土化,研制评价与课程标准一致性分析的程序和方法。