Identifying differential item functioning in multi-stage computer adaptive testing |
| |
Authors: | Mark J. Gierl Hollis Lai Johnson Li |
| |
Affiliation: | 1. Centre for Research in Applied Measurement and Evaluation , University of Alberta , Edmonton , Alberta , Canada mark.gierl@ualberta.ca;3. Centre for Research in Applied Measurement and Evaluation , University of Alberta , Edmonton , Alberta , Canada |
| |
Abstract: | The purpose of this study is to evaluate the performance of CATSIB (Computer Adaptive Testing-Simultaneous Item Bias Test) for detecting differential item functioning (DIF) when items in the matching and studied subtest are administered adaptively in the context of a realistic multi-stage adaptive test (MST). MST was simulated using a 4-item module in a 7-panel administration. Three independent variables, expected to affect DIF detection rates, were manipulated: item difficulty, sample size, and balanced/unbalanced design. CATSIB met the acceptable criteria, meaning that the Type I error and power rates met 5% and 80%, respectively, for the large reference/moderate focal sample and the large reference/large focal sample conditions. These results indicate that CATSIB can be used to consistently and accurately detect DIF on an MST, but only with moderate to large samples. |
| |
Keywords: | computer adaptive testing SIBTEST differential item functioning |
|
|