测验等值中等值公式的研究

通过对测验等值中线性等值公式进行研究,得到改进后的线性等值公式,该公式不仅与两测验的相关系数有关,而且同两测验的信度有密切联系。目前常用的线性等值公式是其当两份测验信度相等时的一个特例。     

锚测验难度参数方差特征对测验等值的影响研究

本文使用R-2.15.2软件模拟研究锚测验难度参数方差特征对测验等值误差的影响,采用三种等值方法(链百分位等值法、Levine等值法和Tucker等值法)对锚测验不同类型的难度方差进行比较研究。结果显示,当锚测验难度方差小于全测验难度方差时,其等值的随机误差和系统误差与锚测验难度方差和全测验难度方差一致时(即锚测验为全测验的平行缩减版minitest时)的表现基本相同。因此,对锚测验而言,要求其与全测验具有相同的统计规格可能过于严格。     

HSK和MHK测验等值分析

本研究通过随机选取2382名考生,采用共同组等值设计和线性等值法,对MHK三级与HSK三级、四级、五级、六级的考生成绩进行了等值,等值结果包括听力、阅读、书面表达各分测验分数及测验总分。     

基于IRT的测验等值应用研究

项目反应理论(IRT)在当前教育测量中应用日益普遍。由于不同原因其入门推广有一定难度,该文从与经典测量理论的衔接延伸入手,深入浅出的阐明项目反应理论的基础。基于项目反应理论所依据的数学模型,从联合概率分布的角度,对项目反应理论进行了剖析,对应用广泛的等值技术进行阐释,并推荐易用软件进行实现。     

测验等值是开发中考评价功能之必需

中考是各地区规模较大和有影响力的高利害性考试,只有建立科学完善的考试评价系统才能充分发挥中考对地区初中教学多方面的服务作用,而建立完善考试评价系统的必备程序是等值。IRT等值的步骤包括估计项目参数、进行IRT量表转换以及制作分数转换表。     

测验等值:新一轮高考改革的技术问题

随着新一轮高考改革的深入,考生在一些科目中将有两次考试机会。这两次考试分数间的相互转换可以通过测验等值来解决。然而测验等值实践涉及诸多环节,每个环节都对最终的等值效果有重要的影响。本文从等值设计的选择、等值必要性判断、等值方法的选择、评价标准的选择以及等值过程的质量控制等方面说明在高考改革中测验等值应注意的问题,以期显著提高等值质量。     

Excel97在测验等值中的应用

利用 Excel97进行单组设计试卷分半法线性等值及传统的单组设计线性等值处理 ,和利用 Excel97的模板功能建立模板 ,输入待处理数据 ,即可得到等值结果。     

测验等值设计的一种新方法──单组设计试卷分半法

测验等值的单组设计,是将需等值的测验X、Y都由同一考生组施测,然后对测验分数进行等值。其优点是考生组只有一个X、Y成绩的差异将归因于测验的不同而不会混杂考生组不同的因素。缺点是同一考生要测验两次,练习效应和疲劳将会干扰等值结果。本文提出一种新的设计方法──单组设计试卷分半法,是在单组设计中,把测验X、Y各分成平行的两半卷,各取X、Y的半卷组成新的测验Z,将Z对同一考生组施测,根据施测结果导出等值转换公式进行分数等值转换。这种方法每个考生只施测一次,既保持了单组设计的优点,又克服了它的缺点。     

全国英语等级考试锚测验非等组设计中样本量对等值结果的影响

本研究采用锚测验非等组设计,探究了锚测验样本量的变化对等值结果的影响。数据来自全国英语等级考试(PETS),使用以Bigsteps为核心的自主改进软件,基于Rasch模型估计题目参数。为了探究等值结果对合格分数线的影响,本研究对比了不同样本量锚测验参数估计值与锚题参数给定值的差异,并对不同样本量锚测验得出的实考试卷等值结果与最大样本量锚测验得出的实考试卷等值结果进行了差异分析。结果表明,当锚测验样本量达到150时,等值结果比较稳定。这一结果表明,PETS设定的300人左右的锚测验样本量是合理的。     

测验等值在中小学新课程教学管理中的应用

中小学新课程改革遇到了一些新问题,这些问题对中小学教学管理产生了一定的影响.教育测量理论中的测验等值技术,是解决中小学新课程改革中出现问题的途径.测验等值为实施新课程改革提供了保障,也方便了管理者.掌握测验等值技术相对困难,造成了测验等值技术在教学管理中的应用相对滞后.     

On the Choice of Anchor Tests in Equating

The choice of anchor tests is crucial in applications of the nonequivalent groups with anchor test design of equating. Sinharay and Holland (2006, 2007) suggested “miditests,” which are anchor tests that are content‐representative and have the same mean item difficulty as the total test but have a smaller spread of item difficulties. Sinharay and Holland (2006, 2007), Cho, Wall, Lee, and Harris (2010), Fitzpatrick and Skorupski (2016), Liu, Sinharay, Holland, Curley, and Feigenbaum (2011a), Liu, Sinharay, Holland, Feigenbaum, and Curley (2011b), and Yi (2009) found the miditests to lead to better equating than minitests, which are representative of the total test with respect to content and difficulty. However, these findings recently came into question as Trierweiler, Lewis, and Smith (2016) concluded, based on a comparison of correlation coefficients of miditests and minitests with the total test, that making an anchor test a miditest does not generally increase the anchor to total score correlation and recommended the continuation of the practice of using minitests over miditests. Their recommendation raises the question, “Should miditests continue to be considered in practice?” This note defends the miditests by citing literature that favors miditests and then by showing that miditests perform as well as the minitests in most realistic situations considered in Trierweiler et al. (2016), which implies that miditests should continue to be seriously considered by equating practitioners.     

Measurement,Sampling, and Equating Errors in Large-Scale Assessments

In large-scale assessments, such as state-wide testing programs, national sample-based assessments, and international comparative studies, there are many steps involved in the measurement and reporting of student achievement. There are always sources of inaccuracies in each of the steps. It is of interest to identify the source and magnitude of the errors in the measurement process that may threaten the validity of the final results. Assessment designers can then improve the assessment quality by focusing on areas that pose the highest threats to the results. This paper discusses the relative magnitudes of three main sources of error with reference to the objectives of assessment programs: measurement error, sampling error, and equating error. A number of examples from large-scale assessments are used to illustrate these errors and their impact on the results. The paper concludes by making a number of recommendations that could lead to an improvement of the accuracies of large-scale assessment results.     

The Examination of the Classification of Students into Performance Categories by Two Different Equating Methods

This study investigates the comparability of two item response theory based equating methods: true score equating (TSE), and estimated true equating (ETE). Additionally, six scaling methods were implemented within each equating method: mean-sigma, mean-mean, two versions of fixed common item parameter, Stocking and Lord, and Haebara. Empirical test data were examined to investigate the consistency of scores resulting from the two equating methods, as well as the consistency of the scaling methods both within equating methods and across equating methods. Results indicate that although the degree of correlation among the equated scores was quite high, regardless of equating method/scaling method combination, non-trivial differences in equated scores existed in several cases. These differences would likely accumulate across examinees making group-level differences greater. Systematic differences in the classification of examinees into performance categories were observed across the various conditions: ETE tended to place lower ability examinees into higher performance categories than TSE, while the opposite was observed for high ability examinees. Because the study was based on one set of operational data, the generalizability of the findings is limited and further study is warranted.     

评分、等值和分数报告过程中的质量监控

从评分、等值到成绩报告的过程中,各环节相互依赖和影响,其评价结果极易出现错误。为了监控这一评价过程并尽可能减少犯错数量,需要制定一套质量监控程序。所谓质量监控即指用来确保评分、等值和分数报告过程中达到预期质量标准的一个正规的系统化过程。评分-等值-分数报告过程可分为11个环节,在很多情况下,质量检查都可以在最终产品上进行。     

Quality Control Procedures in the Scoring, Equating, and Reporting of Test Scores

There is significant potential for error in long production processes that consist of sequential stages, each of which is heavily dependent on the previous stage, such as the SER (Scoring, Equating, and Reporting) process. Quality control procedures are required in order to monitor this process and to reduce the number of mistakes to a minimum. In the context of this module, quality control is a formal systematic process designed to ensure that expected quality standards are achieved during scoring, equating, and reporting of test scores. The module divides the SER process into 11 steps. For each step, possible mistakes that might occur are listed, followed by examples and quality control procedures for avoiding, detecting, or dealing with these mistakes. Most of the listed quality control procedures are also relevant for Internet-delivered and scored testing. Lessons from other industries are also discussed. The motto of this module is: There is a reason for every mistake. If you can identify the mistake, you can identify the reason it happened and prevent it from recurring.     

基于IRT的共同题非等组设计中五种项目参数等值方法的比较研究

本文采用共同题非等组设计,对五种基于IRT的项目参数等值方法进行比较研究。研究数据来自湖北某试点地区课改实验区和非课改区考生在标准化中考数学科目的考试数据,兼用大样本标准和其他标准作为各等值方法比较的检验标准,以RMSD指标作为操作性检验标准,利用STUIRT程序进行等值分析。研究结果表明,针对本研究所设置的等值情境,MS方法稳健性最差,对于项目难度参数的等值,同时校准方法最好,其次是SL特征曲线法,对于项目区分度参数的等值,MM方法精确性最好。     

NCME 2008 Presidential Address: The Impact of Anchor Test Configuration on Student Proficiency Rates

Examined in this study were the effects of reducing anchor test length on student proficiency rates for 12 multiple‐choice tests administered in an annual, large‐scale, high‐stakes assessment. The anchor tests contained 15 items, 10 items, or five items. Five content representative samples of items were drawn at each anchor test length from a small universe of items in order to investigate the stability of equating results over anchor test samples. The operational tests were calibrated using the one‐parameter model and equated using the mean b‐value method. The findings indicated that student proficiency rates could display important variability over anchor test samples when 15 anchor items were used. Notable increases in this variability were found for some tests when shorter anchor tests were used. For these tests, some of the anchor items had parameters that changed somewhat in relative difficulty from one year to the next. It is recommended that anchor sets with more than 15 items be used to mitigate the instability in equating results due to anchor item sampling. Also, the optimal allocation method of stratified sampling should be evaluated as one means of improving the stability and precision of equating results.     

误差因素对教学质量评估工作的影响及对策

误差是影响评估质量的关键,教学评估误差按其产生的原因可分为:抽样误差、系统误差和随机误差等。要有效控制和减少评估误差,需建立科学合理的评估指标体系,评估过程要规范,数据处理应尽可能运用现代技术手段等。