Benchmarking Academic Standards in the UK

The Dearing Report in the UK supported the use of benchmarking in higher education, taking as its stimulus the use of benchmarking in industrial and commercial contexts. The UK Government's concern regarding academic standards has raised the question of whether benchmarking could be applied to them.This article reports on a pilot study of the benchmarking of academic standards in the UK, and demonstrates -- with reference to empirical data -- that benchmarking in this context needs to be approached differently from benchmarking in industrial/commercial milieux, since it requires a combination of finely-drawn comparisons and professional judgements if valid conclusions are to be drawn. It is argued that the complexity which underpins academic standards is inimical to the production of statements about standards that will be applicable across the span of a diverse national system of higher education. The argument presented can be applied, mutatis mutandis, to systems of higher education beyond the UK.     

Measurement Issues in the Alignment of Standards and Assessments

Abstract

This study examined the impact of rater agreement on decisions concerning the alignment between the Golden State Examination in High School Mathematics (California Department of Education, 2001a Academic Senate of the California State University. 1997. Statement on competencies in mathematics expected of entering college students Sacramento, CA: Author.  [Google Scholar]) and the University of California (UC) Statement on Competencies in Mathematics Expected of Entering College Students (Academic Senate of the California State University, 1997 Academic Senate of the California State University. 1997. Statement on competencies in mathematics expected of entering college students Sacramento, CA: Author.  [Google Scholar]). The UC faculty and high school mathematics teachers (N = 20) rated the mathematics items of the Golden State Examination relative to the expectations identified in the UC competency statement, identifying item features related to content and dimensionality. Raters assigned values for a primary topic, a secondary topic, item/topic centrality, and depth of knowledge. Agreement within these criteria was the basis of the assessment of alignment. Results showed considerable variability in judgments across raters and different pictures of alignment depending on the particular subset of raters providing the ratings. A few differences emerged between rater types.     

判断评价与课程标准一致性的若干问题

Webb一致性程序是判断评价与课程标准一致性的重工具,它有4个标准:类别一致性、知识深度一致性、知识范围一致性和知识分布平衡性,分别对应着多级可接受水平。一致性研究的挑战在于判断"什么样的一致性才是最好的",这一判断过程虽然可以基于众多实证研究,但也包含某种程度的主观性。     

基于课程标准的学业成就评价:韦伯模式之研究

学业成就评价必须基于课程标准,这是国家对基础教育整体质量卓越的诉求。韦伯博士是美国"基于标准的评价"运动的重要代表人物,他提出了学业成就评价与课程标准保持一致性的分析维度、分析程序、分析方法等一整套一致性框架,并开发了基于网络的评价与标准一致性工具。全面深入研究韦伯模式,对我国基础教育阶段学生学业成就评价改革具有重要意义。     

Issues Related to Judging the Alignment of Curriculum Standards and Assessments

Abstract

A process for judging the alignment between curriculum standards and assessments developed by the author is presented. This process produces information on the relationship of standards and assessments on four alignment criteria: Categorical Concurrence, Depth of Knowledge Consistency, Range of Knowledge Correspondence, and Balance of Representation. Five issues are identified—but not resolved—that have arisen from conducting alignment studies. All of these issues relate to making a decision about what alignment is good enough. Pragmatic decisions have been made to specify acceptable levels for each of the alignment criteria. The assumptions are described. The issues discussed arise from a change in the underlying assumptions and from considering variations in the purpose for an assessment. The existence of such issues reinforces that alignment judgments have an element of subjectivity.     

Evaluating Content Alignment in Computerized Adaptive Testing

The alignment between a test and the content domain it measures represents key evidence for the validation of test score inferences. Although procedures have been developed for evaluating the content alignment of linear tests, these procedures are not readily applicable to computerized adaptive tests (CATs), which require large item pools and do not use fixed test forms. This article describes the decisions made in the development of CATs that influence and might threaten content alignment. It outlines a process for evaluating alignment that is sensitive to these threats and gives an empirical example of the process.     

Alignment of Mathematics State-Level Standards and Assessments: The Role of Reviewer Agreement

This article examines the role of reviewer agreement in judgments about alignment between tests and standards. We used case data from three state alignment studies to explore how different approaches to incorporating reviewer agreement changes alignment conclusions. The three case studies showed varying degrees of reviewer agreement about correspondences between objectives and test items. Moreover, taking into account reviewer agreement in the analyses sometimes had a marked effect on alignment conclusions. We discuss reasons for differences across case studies and alignment approaches, as well as implications for future alignment efforts.     

一种具有帧同步测试功能的数字信源研制

针对现有数字信源输出信号类型少,特别是不能满足帧同步实验的需要的缺点,采用CPLD技术设计了一种新型的数字信源电路,并介绍了系统的组成及工作过程。该数字信源电路可以演示数字复接过程,输出数据逐帧重复、逐帧变化的时分复用数字信源信号,输出漏同步和假同步测试信号。若将漏同步和假同步测试信号输入到帧同步电路,可以让学生更直观地了解到帧同步电路中,漏同步和假同步保护电路的功能和作用。     

理解和运用《教育与心理测试标准》——ETS的案例研究

《教育与心理测试标准》由美国心理协会(APA)、美国教育研究协会(AERA)和美国国家教育测量协会(NCME)联合制定,美国教育考试服务中心(ETS)仿效《教育与心理测试标准》,制定了《ETS质量和公平标准》。本文介绍这两套标准的制定过程以及历史发展,《ETS质量和公平标准》的理念以及在ETS考试产品中的应用。     

中国英语能力等级量表与英语测评

中国英语能力等级量表是对我国英语使用者运用英语语言能力的系列描述,分3个阶段9个级别,规定了中国英语学习者和使用者的英语能力等级,描述了各等级的能力表现特征。量表以交际语言能力为基础,以语言使用为出发点,以能……为描述语的基本结构。量表为我国英语教学、学习和测评提供了统一标准,为我国建立终结性和形成性相结合的综合评价体系奠定了基础。     

Methodological Choices in the Content Analysis of Textbooks for Measuring Alignment With Standards

With the recent adoption of the Common Core standards in many states, there is a need for quality information about textbook alignment to standards. While there are many existing content analysis procedures, these generally have little, if any, validity or reliability evidence. One exception is the Surveys of Enacted Curriculum (SEC), which has been widely used to analyze the alignment among standards, assessments, and teachers’ instruction. However, the SEC can be time‐consuming and expensive when used for this purpose. This study extends the SEC to the analysis of entire mathematics textbooks and investigates whether the results of SEC alignment analyses are affected if the content analysis procedure is simplified. The results indicate that analyzing only every fifth item produces nearly identical alignment results with no effect on the reliability of content analyses.     

高中生数学学科核心素养测评与课程标准一致性研究

综合世界各国和地区的经验,基于核心素养的课程标准研制基本遵循学生核心素养-学科核心素养-内容标准的思路.在这种思路中,学科核心素养起着桥梁作用,是研制课程标准的关键.[1]随着《普通高中数学课程标准(2017年版)》(以下简称《课程标准(2017年版)》)的发布,数学学科核心素养测评成为数学教育研究的热点问题,高中数学...     

Benchmarking Art and Design

This paper describes the evolution of the Benchmark Statement for undergraduate provision in Art and Design in the UK. The development of subject‐specific benchmark standards was a key recommendation of the Dearing Report. The National Committee of Inquiry into Higher Education had recommended that the Quality Assurance Agency (QAA) should ‘work with institutions to establish small, expert teams to provide benchmark information on standards, in particular threshold standards, operating within the framework of qualifications.’ Subject Benchmarking is one of a number of linked initiatives emanating from QAA and, as such, will complement the National Qualifications Framework, the Codes of Practice, and Programme Specifications, as well as future rounds of Subject Review. The benchmark statements for 22 subjects were published in April 2000, and the statements for a further 20 subjects, including those for Art and Design, were published in March 2002. The full statements for all 42 subjects can be accessed on‐line at the QAA website All quoted statements not referenced are extracted from the benchmark statements for art and design.     

软件测试技术与应用课程标准的研究

课程建设是工学结合人才培养模式的载体,课程标准对课程建设起着重要的指导作用。以软件测试技术与应用这门课程为例,通过对该课程的调研,进行了学习领域、学习情境、学习资源、学习环境和考核方式等方面标准化的教学设计,探讨了以工作过程为导向,以真实的项目案例为驱动的新教学模式课程标准的制定过程。     

公务员招警体能测试标准研究

通过对公务员招警体能考试的监考,对人民警察的考试程序、方法、项目、标准进行探究,并对人民警察的体能测试标准提出分析和建议,以供职能部门参考。     

Steps Toward More Effective Implementation of the Standards for Educational and Psychological Testing

The goal of the Standards for Educational and Psychological Testing is to improve testing practices, but their impact on practice appears spotty. Self-regulation clearly fails in some instances. The establishment of an external agency to oversee testing practices and adherence to the Standards would face substantial hurdles, and the ambiguity of many of the Standards would hobble such an organization if one were created. Many of the Standards are general statements of principle, and past controversies make clear that we in the field often disagree about the reasons for them, their applicability to specific cases, and their practical meaning in specific contexts. This paper argues that the field should follow two approaches to lessen this ambiguity. First, using journals, conferences, and other vehicles, we should foster more frequent and more protracted debate about the practical meaning of key Standards, such as 13.6 and 13.7, which mandate that a decision that will have a "major impact" on a student should not be based on a single score. Second, future revisions of the Standards should use concrete examples of testing practices to clarify the meaning of the Standards, much as the legal system uses case law to clarify the meaning of the general principles embodied in statutes .     

汉语测试语言能力标准研究述评

介绍了国外语言能力标准(量表)研究的历史背景,对几个国际上有重要影响力的语言能力量表进行了简要分析。在前人研究的基础上,对汉语语言能力量表的研究现状进行整合,通过与国外语言能力量表的对比,指出两者各自的优缺点以及汉语语言能力量表的发展趋势,以期推动汉语作为母语以及第二语言的语言教学、语言学习及语言测试的发展。