试题组卷算法的研究与应用

试题组卷问题是一个多目标约束优化问题,拟运用遗传算法的全局寻优对考试中的自动化组卷进行研究,实现自适应遗传算法的智能组卷方案,并基于实际系统进行了算法优化,同时得到了一个解决适合学校教务要求的试题模型的算法。     

试题命制的理论和技术(二)

大规模教育考试试题命制以心理学的某些理论假设为基础。与这些理论假设一致的试题定义要求试题应该具备三个要素,即测量目标、刺激情境和设问,这三个要素缺失了任何一个,都不能构成完整的试题。根据这些理论假设以及试题定义和要素,本文讨论了命制客观题和主观题的基本要求,客观题包括题干的要求、选项设置的要求以及选项数的问题;主观题包括情境材料的选择、设问、赋分和评分标准制定。     

基于高考英语难题的试题命制技术探讨

难度不是试题的固有属性,而是考生因素与试题特征之间互动的结果。很多试题分析者倾向于将试题难度偏高的原因仅仅归结于学生未掌握相关知识或技能,而忽视试题本身的特征。通过分析60道难度在0.6以下的高考英语试题,探究其难度来源。结果显示,除考生因素外,难题或偏难题的难度来源也与命题技术有关,比如答案的唯一性与可接受性、考查内容超纲、考点设置与评分标准欠妥等方面的问题。为此,提出考试机构应提高命题水平,加强试题质量监控,确保大规模考试科学选拔人才。     

选择题的测试功能及其适用性分析

选择题作为客观性试题中有代表性的试题类型,目前在各类考试中最为通用,鉴于它在标准化考试中发挥的重要作用,我们有必要正确认识和科学把握这类题型的测试功能以及适用范围,选乃是现代考试优化试卷结构和实现高信度与高效度的基本前提。本文以单项选择题、多项选择题、双项选择题和判断题为研究对象,试图探讨其在考试实践中的测试功能、适用性以及亟待改进的问题,目的在于使广大教育工作者和考试命题管理人员对此类题型有所了解和掌握,为今后更为有效地发挥选择题的作用做好理论准备,这对于搞好考试命题工作、提高试题质量至关重要。     

高考改革更关注对教学的积极导向

考试可能有负面作用,也可以产生积极的影响,关键在于考试设计的指导思想。考试的形式和命题的取向主宰着教与学的模式。我国著名的考试问题研究专家桂诗春教授就曾指出,题型不变化,考试这把"刀"就"钝化"了,而要使之变得锋利,就要注意经常变换题型。新课程改革实验区高考英语试卷的变化,凸现了对学生分析、推理、概括和表达等较高层次能力的考查;题型的变化为广大教师深入领悟和实践高中英语新课程标准做出了正面引领的作用。     

关于日语古文今译用于古典语法测试的探讨

对日语古典语法测试项目使用的古文今译多项选择题进行探讨,阐述以下内容：（1）在译文多项选择题的命题中,常常出现双重评判标准和选择项中有多个答案的问题,这些问题源于评判译文标准的不确定性。（2）我们可以应用功能翻译理论的目的原则建立教学翻译的评判标准;教学翻译有区分合不合适的标准,合适的教学翻译又有区分一般和高质量之差的标准。前者可应用于语法测试,而后者应用于语法测试时会引起诸多问题,应尽量避免使用。     

A Mixture IRTree Model for Extreme Response Style: Accounting for Response Process Uncertainty

This paper presents a mixture item response tree (IRTree) model for extreme response style. Unlike traditional applications of single IRTree models, a mixture approach provides a way of representing the mixture of respondents following different underlying response processes (between individuals), as well as the uncertainty present at the individual level (within an individual). Simulation analyses reveal the potential of the mixture approach in identifying subgroups of respondents exhibiting response behavior reflective of different underlying response processes. Application to real data from the Students Like Learning Mathematics (SLM) scale of Trends in International Mathematics and Science Study (TIMSS) 2015 demonstrates the superior comparative fit of the mixture representation, as well as the consequences of applying the mixture on the estimation of content and response style traits. We argue that methodology applied to investigate response styles should attend to the inherent uncertainty of response style influence due to the likely influence of both response styles and the content trait on the selection of extreme response categories.     

Latent Variable Modeling and Adaptive Testing for Experimental Cognitive Psychopathology Research

The adaptation of experimental cognitive tasks into measures that can be used to quantify neurocognitive outcomes in translational studies and clinical trials has become a key component of the strategy to address psychiatric and neurological disorders. Unfortunately, while most experimental cognitive tests have strong theoretical bases, they can have poor psychometric properties, leaving them vulnerable to measurement challenges that undermine their use in applied settings. Item response theory–based computerized adaptive testing has been proposed as a solution but has been limited in experimental and translational research due to its large sample requirements. We present a generalized latent variable model that, when combined with strong parametric assumptions based on mathematical cognitive models, permits the use of adaptive testing without large samples or the need to precalibrate item parameters. The approach is demonstrated using data from a common measure of working memory—the N-back task—collected across a diverse sample of participants. After evaluating dimensionality and model fit, we conducted a simulation study to compare adaptive versus nonadaptive testing. Computerized adaptive testing either made the task 36% more efficient or score estimates 23% more precise, when compared to nonadaptive testing. This proof-of-concept study demonstrates that latent variable modeling and adaptive testing can be used in experimental cognitive testing even with relatively small samples. Adaptive testing has the potential to improve the impact and replicability of findings from translational studies and clinical trials that use experimental cognitive tasks as outcome measures.     

Parameter Estimation Accuracy of the Effort-Moderated Item Response Theory Model Under Multiple Assumption Violations

As low-stakes testing contexts increase, low test-taking effort may serve as a serious validity threat. One common solution to this problem is to identify noneffortful responses and treat them as missing during parameter estimation via the effort-moderated item response theory (EM-IRT) model. Although this model has been shown to outperform traditional IRT models (e.g., two-parameter logistic [2PL]) in parameter estimation under simulated conditions, prior research has failed to examine its performance under violations to the model’s assumptions. Therefore, the objective of this simulation study was to examine item and mean ability parameter recovery when violating the assumptions that noneffortful responding occurs randomly (Assumption 1) and is unrelated to the underlying ability of examinees (Assumption 2). Results demonstrated that, across conditions, the EM-IRT model provided robust item parameter estimates to violations of Assumption 1. However, bias values greater than 0.20 SDs were observed for the EM-IRT model when violating Assumption 2; nonetheless, these values were still lower than the 2PL model. In terms of mean ability estimates, model results indicated equal performance between the EM-IRT and 2PL models across conditions. Across both models, mean ability estimates were found to be biased by more than 0.25 SDs when violating Assumption 2. However, our accompanying empirical study suggested that this biasing occurred under extreme conditions that may not be present in some operational settings. Overall, these results suggest that the EM-IRT model provides superior item and equal mean ability parameter estimates in the presence of model violations under realistic conditions when compared with the 2PL model.     

Computer-based knowledge of results feedback in different delivery modes: Effects on performance,motivation, and achievement emotions

Immediate Knowledge of Results (KR) feedback at the task level may motivate test takers by showing that their answers matter. Appealing feedback cues may help to prevent negative emotions in lower performers who receive a higher amount of negative feedback. In this experiment, we varied the presence of KR feedback and the feedback delivery mode in a 1 × 5 between-subjects design (i.e., no feedback vs. text, color, sound, or animation feedback) to investigate effects on learning outcomes and affective-motivational measures. Our sample included 661 fifth and sixth graders who solved two computer-based multiple-choice science tests. First, students worked on an 18-item treatment test (with experimental feedback manipulation). Students repeatedly rated their effort, enjoyment, pride, and boredom during the test, as well as their expectancy of success and attainment value after the test. Subsequently, they worked on a posttest (without feedback) that assessed recall and near-transfer learning. All KR feedback conditions significantly increased recall, but there was no evidence for near-transfer learning. Feedback had a significant, negative effect on attainment value, whereas significant interactions between the feedback conditions and students’ treatment performance revealed that feedback effects on several affective-motivational dimensions (i.e., expectancy of success, enjoyment, pride, and boredom) were performance-dependent. Feedback benefited higher performers’ motivation and affect but showed negative effects on some affective-motivational measures for lower performers. The pattern of results indicated that color/sound/animation feedback may have reduced the effect of performance on emotional feedback perception to some extent. However, none of the feedback conditions improved affective-motivational outcomes independent of students’ performance.