Investigating Uniform and Non-Uniform Gender DIF in Computer-Based ESL Writing Assessment

The objective of the present investigation was to examine the comparability of writing prompts for different gender groups in the context of the computer-based Test of English as a Foreign Language? (TOEFL®-CBT). A total of 87 prompts administered from July 1998 through March 2000 were analyzed. An extended version of logistic regression for polytomous items was used to investigate both uniform and non-uniform gender effects. An English Language Ability variable was developed from the multiple-choice components of the TOEFL®-CBT examination and used as a matching variable. Initially, most of the prompts were flagged because of statistically significant uniform gender effects, with some prompts displaying non-uniform effects as well. Nevertheless, the effect sizes were too small for any of those flagged prompts to be classified as having an important group effect. These findings are discussed in relation to prompt content review, gender format differences, and second language learning theories.