Application of Think Aloud Protocols for Examining and Confirming Sources of Differential Item Functioning Identified by Expert Reviews |
| |
Authors: | Kadriye Ercikan Rubab Arim Danielle Law Jose Domene France Gagnon Serge Lacroix |
| |
Institution: | 1. University of British Columbia;2. University of New Brunswick |
| |
Abstract: | This paper demonstrates and discusses the use of think aloud protocols (TAPs) as an approach for examining and confirming sources of differential item functioning (DIF). The TAPs are used to investigate to what extent surface characteristics of the items that are identified by expert reviews as sources of DIF are supported by empirical evidence from examinee thinking processes in the English and French versions of a Canadian national assessment. In this research, the TAPs confirmed sources of DIF identified by expert reviews for 10 out of 20 DIF items. The moderate agreement between TAPs and expert reviews indicates that evidence from expert reviews cannot be considered sufficient in deciding whether DIF items are biased and such judgments need to include evidence from examinee thinking processes. |
| |
Keywords: | differential item functioning English and French test comparability expert reviews sources of DIF think aloud protocols |
|