Using Diagnostic Profiles to Describe Borderline Performance in Standard Setting

In test-centered standard-setting methods, borderline performance can be represented by many different profiles of strengths and weaknesses. As a result, asking panelists to estimate item or test performance for a hypothetical group study of borderline examinees, or a typical borderline examinee, may be an extremely difficult task and one that can lead to questionable results in setting cut scores. In this study, data collected from a previous standard-setting study are used to deduce panelists’ conceptions of profiles of borderline performance. These profiles are then used to predict cut scores on a test of algebra readiness. The results indicate that these profiles can predict a very wide range of cut scores both within and between panelists. Modifications are proposed to existing training procedures for test-centered methods that can account for the variation in borderline profiles.     

Governance structure and standard setting in educational assessment

ABSTRACT

Seldom have comparative studies of educational assessment systems been undertaken, especially regarding their standard setting procedures. This study examines the effects of governance structures on the power relations in standard setting in the dominant school-leaving or university-entrance examinations. We present acritical analysis of the published research and policy documents, including sense-checking with senior assessment practitioners from 22 jurisdictions. The nature of standard setting systems in three cases of Ireland, the USA and India are described in detail to showcase the differences between the following three models of governance systems: nationalised, commercial market and quasi-market. The contribution of this article, then, is to describe the three models of governance systems, to classify the 22 jurisdictions using the three models, and to generate propositions inductively. Thus, the article provides aconceptual basis for extension of this work to other cases to advance the literature cumulatively by theory-building.     

高职院校专业设置评估指标体系及评估标准研究

目前我国省一级的专业设置审核管理制度尚不健全,高职院校还普遍存在着专业设置的随意性、盲目性现象。构建一套操作性强的专业设置评估指标体系,把专业设置条件定性评价与定量评价结合起来,对专业设置进行科学化管理十分必要。根据教育部专业设置管理文件基本要求,文章构建了高职院校专业设置的评估指标体系,该体系由7个一级指标、19个二级指标构成。     

The Bookmark Standard-Setting Method: A Literature Review

The Bookmark method for setting standards on educational tests is currently one of the most popular standard-setting methods. However, research to support the method is scarce. In this report, we review the published and unpublished literature on this method as well as some seminal work in the area of evaluating standard-setting studies. Our review highlights both strengths and limitations of the method. Strengths include its wide acceptance and panelist confidence in the method. Limitations include a potential bias to produce lower-than-intended standards and problems in selecting the most appropriate response probability value for ordering the items presented to panelists. It is clear that more research on this method is needed to support its wide use. Several areas for future research to better understand the validity of the Bookmark method for setting standards on educational tests are presented.     

数控加工中常见对刀的方法及其精确对刀技巧研究

对刀是数控加工中的重要操作,零件的加工精度在很大程度上取决于对刀的精确性,本文介绍了对刀原理及对刀思路、对刀的方法以及探讨了精确对刀的技巧。     

电火花加工中一种新的找正方法

电火花成型机床(EDM)在模具行业中起着很大的作用,它可以完成其它机床无法完成的工作。由于电极是事先加工好的,所以在实际加工中找正遇到了很大困难。本文介绍了一种新的找正方法,提高了模具加工精度,并方便了找正过程。     

International language proficiency standards in the local context: Interpreting the CEFR in standard setting for exam reform in Luxembourg

ABSTRACT

In the field of second and foreign language learning, the Common European Framework of Reference for languages (CEFR) is widely-used for setting language proficiency standards within European, and increasingly global, contexts. Few studies, however, have investigated the ways in which systemic, macro-level factors within national educational contexts may influence standard setting practices using the CEFR. In this paper, we explore this issue through an analysis of recorded discussions within standard setting sessions for the Épreuve Commune for English, a national English language examination in Luxembourg. The data reveals four key sources of influence on standard setting decision-making: Luxembourg’s unique language ecology, streamed schooling, the national curriculum, and an ongoing exam reform project. Through this analysis, we argue that Luxembourg functions as a critical case illustrating the tension between international standards of language proficiency and local realities.     

The Choice of Response Probability in Bookmark Standard Setting: An Experimental Study

Evidence of the internal consistency of standard-setting judgments is a critical part of the validity argument for tests used to make classification decisions. The bookmark standard-setting procedure is a popular approach to establishing performance standards, but there is relatively little research that reflects on the internal consistency of the resulting judgments. This article presents the results of an experiment in which content experts were randomly assigned to one of two response probability conditions: .67 and .80. If the standard-setting judgments collected with the bookmark procedure are internally consistent, both conditions should produce highly similar cut scores. The results showed substantially different cut scores for the two conditions; this calls into question whether content experts can produce the type of internally consistent judgments that are required using the bookmark procedure.     

The application of pairwise comparisons to form scaled exemplars as a basis for setting and exemplifying standards in teacher education

ABSTRACT

This paper details a two-stage process for standard setting using an extended pairwise comparison method. It describes the application of a new process of establishing a set of scaled and ordered performances as a basis for exemplifying standards. In stage one, the method of pairwise comparison was used to generate data from pairwise judgements made at the criterion level and overall grade. In stage two, judges identified where the pass/fail boundary lay in relation to a selected subset of ordered performances generated from stage one. The combined evidence from the two-stage process demonstrates the method’s application to establish exemplified standards using on-balance judgements across multiple criteria. The approach combines expert judgement and statistical data to provide a basis for comparable cross-institution assessments against standards in the case of a summative assessment of professional competence in teacher education. It may be relevant to any discipline with professional standards and complex performance assessments.     

法学本科教育的培养目标与教学方法

近年来，我国法学本科教育得到了很大的发展，但仍存在不足，亟待完善。章以商法实务课程为例，探讨了如何在新形势下对法学本科教育的培养目标重新定位，如何在新培养目标下选择适当的教学方法，并初步构想了模拟实务教学法。     

Condensed Mastery Profile Method for Setting Standards for Diagnostic Assessment Systems

The purpose of this study was to develop a standard‐setting method appropriate for use with a diagnostic assessment that produces profiles of student mastery rather than a single raw or scale score value. The condensed mastery profile method draws from established holistic standard‐setting methods to use rounds of range finding and pinpointing to specify cut points between performance levels. Panelists are convened to review profiles of mastery and specify cut points between performance levels based on the total number of skills mastered. Following panelist specification of cut points, a statistical method is implemented to smooth cut points over grades to decrease between‐grade variability. Procedural evidence, including convergence plots, standard errors of pinpointing ratings, and panelist feedback, suggest the condensed mastery profile method is a useful and technically sound approach for setting performance standards for diagnostic assessment systems.     

Standard Setters: Stand Up and Take a Stand!

This 2006 NCME Career Award Address presents observations made over a career in the field of standard setting, summarizes the status of current research, and makes recommendations for future research.     

教师专业发展的客观要求:教学诊断能力及其提高

教师专业发展无论是理论研究还是实践层面仍然在其外围打转,还没有触及到实质问题。文章通过对教师专业发展的分析,着重从教师职业的功能性特征和不可替代性来把握教师专业发展的内涵。认为教学诊断能力及其提高是教师专业发展的客观要求。提出了教学诊断对教师专业发展的重要意义以及提高教师教学诊断能力的方式和途径。     

The symbolic violence of setting: A Bourdieusian analysis of mixed methods data on secondary students’ views about setting

‘Setting’ is a widespread practice in the UK, despite little evidence of its efficacy and substantial evidence of its detrimental impact on those allocated to the lowest sets. Taking a Bourdieusian approach, we propose that setting can be understood as a practice through which the social and cultural reproduction of dominant power relations is enacted within schools. Drawing on survey data from 12,178 Year 7 (age 11/12) students and discussion groups and individual interviews with 33 students, conducted as part of a wider project on secondary school grouping practices, we examine the views of students who experience setting, exploring the extent to which the legitimacy of the practice is accepted or challenged, focusing on students’ negative views about setting. Analyses show that privileged students (White, middle class) were most likely to be in top sets whereas working‐class and Black students were more likely to be in bottom sets. Students in the lowest sets (and boys, Black students and those in receipt of free school meals) were the most likely to express negative views of setting and to question the legitimacy and ‘fairness’ of setting as a practice, whereas top‐set students defended the legitimacy of setting and set allocations as ‘natural’ and ‘deserved’. This paper argues that setting is incompatible with social justice approaches to education and calls for the foregrounding of the views of those who are disadvantaged by the practice as a tool for challenging the doxa of setting.     

The National Literacy Strategy and setting: an investigation in one school

This article investigates the policy of setting that is commonly being seen as an organizational tool for effective delivery of the National Literacy Strategy (NLS). It is suggested that teachers are finding the increased amount of whole-class teaching to a diverse range of abilities, as prescribed by the NLS, problematic. This case-study researches both teachers' and children's perspectives on the Literacy Hour as taught in mixed-ability classes and in sets. Reflective commentaries and pupil questionnaires were used to collect data over two years. The results indicate a complex range of issues, both pedagogical and organizational, which need be considered before setting can be promoted as a strategy for literacy delivery. I conclude that there is a need for urgent research into the effectiveness of setting as a means of teaching the Literacy Hour.     

榜样教育中的误区:榜样选取的高标性

榜样教育法作为思想政治教育的主要方法之一,其效果并不尽如人意,主要原因之一就是榜样选取的高标性。首先,从客观特点来看,榜样应具有可接近性,使受教育者感到有相似条件可以学;从主观状况来说,受教育者要具备把榜样看成某种信念的体现者,并认为符合自己的选择条件。其次,榜样教育更要遵循实事求是的原则。再次,要注重榜样行为的社会背景以及遵循因材施教的原则。     

新课程标准下的初中数学分层次教学法初探

在新一轮课程改革中现有的中学数学教育模式已不能适应当前的新课程标准 ,显然探索新的教学模式势在必行。中学数学分层次教学法就是结合新课程标准的内容以及改变以往的传统教学模式的新模式。文章着重论述了课前设计 ,课上问题情景建立 ,课上如何分层次教学几方面内容 ,同时重视例题 ,优化效益和课后反思     

Commentary: A Response to Reckase's Conceptual Framework and Examples for Evaluating Standard Setting Methods

A look at real data shows that Reckase's psychometric theory for standard setting is not applicable to bookmark and that his simulations cannot explain actual differences between methods. It is suggested that exclusively test-centered, criterion-referenced approaches are too idealized and that a psychophysics paradigm and a theory of group behavior could be more useful in thinking about the standard setting process. In this view, item mapping methods such as bookmark are reasonable adaptations to fundamental limitations in human judgments of item difficulty. They make item ratings unnecessary and have unique potential for integrating external validity data and student performance data more fully into the standard setting process.     

卓越工程师培养目标与标准的制定

石油工程专业本科层次“卓越工程师教育培养计划”的培养目标和标准的制定是实施本计划的前提和基础。本文依据教育部制定的“卓越工程师教育培养计划”通用标准,结合中国地质大学（北京）石油工程专业现状和特点,研究并制定了石油工程专业本科层次卓越工程师培养目标和标准。     

杨万里序跋文的"定法"与"活法"

杨万里的序跋文讲求立意．重视文章的结构形式,这是他散文理论“定法”的很好实践。同时序跋文又有求真、求活的特点,注重诗、文、画的会通．风趣有味,是他“活法”理论在散文领域的实践。