The Consistency of Detecting Item Bias Across Different Test Administrations: Implications of Another Failure

Several item bias detection methods were applied to the analysis of bias among males and females for items from a curriculum-based mathematics test. The focus of this analysis was the consistency of the methods across different test administrations of the same items. The results indicated that, of the methods studied, the Mantel–Haenszel (M–H) and IRT-based sum-of-squares methods were the most consistent. However, the degree of reliability and agreement for these methods was modest at best. As with most prior research, no reasonable explanation could be found for the most consistently flagged items. A likely reason for this lies in the confusion of visible genetic group characteristics with their instructional backgrounds. A multidimensional perspective of item bias is proposed for future research that will take such confounding into account.