General and Specific Factors in Reading Comprehension

One explanation for why many students routinely choose to do their homework with a radio or TV operating is that this distraction facilitates studying. To investigate the validity of this explanation, the authors carried out two experiments with sixth-grade students who worked on two difficulty levels of individualized mathematics assignments (N = 12) and reading assignments (N = 9) under three distraction conditions: quiet; self-selected, self-regulated radio; and self-selected, self-regulated TV. The data yielded no evidence of sizable distractor effects on students' time spent studying, computational accuracy, reading comprehension, or reading rate. These results generally are consistent with the theoretical conceptualization of attention as flexible and adaptable.     

Application of Think Aloud Protocols for Examining and Confirming Sources of Differential Item Functioning Identified by Expert Reviews

This paper demonstrates and discusses the use of think aloud protocols (TAPs) as an approach for examining and confirming sources of differential item functioning (DIF). The TAPs are used to investigate to what extent surface characteristics of the items that are identified by expert reviews as sources of DIF are supported by empirical evidence from examinee thinking processes in the English and French versions of a Canadian national assessment. In this research, the TAPs confirmed sources of DIF identified by expert reviews for 10 out of 20 DIF items. The moderate agreement between TAPs and expert reviews indicates that evidence from expert reviews cannot be considered sufficient in deciding whether DIF items are biased and such judgments need to include evidence from examinee thinking processes.     

Is academic tracking related to gains in learning competence? Using propensity score matching and differential item change functioning analysis for better understanding of tracking implications

This study analyzes gains in cognitive components of learning competence with respect to cohorts based on ability tracking in a Czech longitudinal study. Propensity score matching is used to form parallelized samples of academic and non-academic track students and to eliminate the effect of selective school intake. We applied regression models on the total scores to test for the overall track effect. Furthermore, we analyze scores and gains on the subscores and check for differential item functioning in Grade 6 and in change to Grade 9. While after 3 years, no significant difference between the two tracks was apparent in the total learning competence score, we did, however, find significant differences in some subscores and in the functioning of some items. We argue that item-level analysis is important for deeper understanding of the tracking implications and may provide the basis for more precise evidence-based decisions regarding the tracking policy.     

基于RCMLM模型的数学试卷性别DIF研究

RCMLM模型是基于Rasch测量理论的通用拓展模型。利用RCMLM模型对一份普通高中数学试卷进行不同性别的DIF分析。结果表明:该模型可对具有二分计分和多分计分的试题同时进行DIF分析,避免了以往分别对两种计分方式试题进行DIF分析的弊端,保持了试卷的完整性,使DIF分析结果更加有效。     

Examining type I error and power for detection of differential item and testlet functioning

In this study, the effectiveness of detection of differential item functioning (DIF) and testlet DIF using SIBTEST and Poly-SIBTEST were examined in tests composed of testlets. An example using data from a reading comprehension test showed that results from SIBTEST and Poly-SIBTEST were not completely consistent in the detection of DIF and testlet DIF. Results from a simulation study indicated that SIBTEST appeared to maintain type I error control for most conditions, except in some instances in which the magnitude of simulated DIF tended to increase. This same pattern was present for the Poly-SIBTEST results, although Poly-SIBTEST demonstrated markedly less control of type I errors. Type I error control with Poly-SIBTEST was lower for those conditions for which the ability was unmatched to test difficulty. The power results for SIBTEST were not adversely affected, when the size and percent of simulated DIF increased. Although Poly-SIBTEST failed to control type I errors in over 85% of the conditions simulated, in those conditions for which type I error control was maintained, Poly-SIBTEST demonstrated higher power than SIBTEST.     

语言测试中的性别偏差与公平性

要保证语言测试的效度和公平性,必须采取有效方法侦测及消除测试中存在的偏差。本文从性别偏差的角度对测试偏差作一简要介绍,探讨了性别偏差和公平性的概念,并介绍了用来侦测性别偏差的方法。     

DIF分析实际应用中的常见问题及其研究新进展

多等级计分题、小样本、匹配变量不纯以及DIF检验后的原因分析是DIF检验面临的常见问题,对多等级计分题目进行DSF分析,小样本情况下DIF检测的平滑方法,匹配变量不纯情况下采用MIMIC法,以及运用Logistic模型进行DIF检验后的原因分析是DIF研究中的一些新进展。对这些进展的分析使我们相信,多种检验方法的配合使用、运用DIF研究进行多维IRT框架下的潜在变量探究等,都有可能使DIF研究成为测量学未来的基础研究领域之一。     

新汉语水平考试HSK(六级)的性别公平性评估

本文通过对2011年新汉语水平考试HSK(六级)8次考试的试题进行项目功能差异(DIF)分析,以评估其性别公平性。结果显示,800个试题中存在DIF的题目占总数的3.3%;800个试题的MH值平均数为0.02,其95%置信区间包含0,即试卷总体上不存在DIF。因此,HSK(六级)具有较理想的性别公平性。     

Reasons for Gender-Related Differential Item Functioning in a College Admissions Test

Gender fairness in testing can be impeded by the presence of differential item functioning (DIF), which potentially causes test bias. In this study, the presence and causes of gender-related DIF were investigated with real data from 800 items answered by 250,000 test takers. DIF was examined using the Mantel–Haenszel and logistic regression procedures. Little DIF was found in the quantitative items and a moderate amount was found in the verbal items. Vocabulary items favored women if sampled from traditionally female domains but generally not vice versa if sampled from male domains. The sentence completion item format in the English reading comprehension subtest favored men regardless of content. The findings, if supported in a cross-validation study, can potentially lead to changes in how vocabulary items are sampled and in the use of the sentence completion format in English reading comprehension, thereby increasing gender fairness in the examined test.     

语言测试的社会公平性问题的实证研究——汉语水平考试的DIF检验

任何一种测试都要公平、公正,否则就失去了它存在的价值和意义。对语言测试的公平性问题的研究是测验开发者不可推卸的责任和义务。汉语水平考试(HSK)是专门为汉语作为第二语言的学习者而设计的语言测试。经过二十多年的发展,HSK在公平性问题研究方面已经取得了长足进展。针对HSK特有的考生构成特点,本文将考生数量较少的非亚裔考生当作研究对象,将其设为目标组,考察HSK是否会对这个亚群体考生不公平。本文运用3种传统的DIF检验方法——MH方法、SIBTEST方法和Logistic regression方法,对HSK【初中等】一套试卷的听力理解测验进行DIF检验,比较目标组(非亚裔考生)和参照组(亚裔考生)在同一组项目上的表现。