Methodology for Examining the Reliability of Group Mean Difference Scores |
| |
| Authors: | Robert L Brennan Ping Yin Michael T Kane |
| |
| Institution: | Chair in Measurement and Testing, and Director, Center for Advanced Studies in Measurement and Assessment, University of Iowa, 297 Lindquist Center North, Iowa City, IA 52242-1529;. His research interests include classical test theory, generalizability theory, equating, and scaling.;Postdoctoral Research Scholar, Center for Advanced Studies in Measurement and Assessment, University of Iowa, 297 Lindquist Center North, Iowa City, IA 52242-1529;. Her interests include reliability, generalizability theory, and difference scores.;Director of Research, National Conference of Bar Examiners, 402 West Wilson Street, Madison, WI 53703-3614;. His research interests include validity and generalizability theory. |
| |
| Abstract: | This article treats various procedures for examining the reliability of group mean difference scores, with particular emphasis on procedures from univariate and multivariate generalizability theory. Attention is given to both traditional norm-referenced perspectives on reliability as well as criterion-referenced perspectives that focus on error-tolerance ratios and functions of them. The procedures discussed are illustrated using three cohorts of data for third- and fourth-grade students in Iowa who took the Iowa Tests of Basic Skills in recent years. For these data, estimates of reliability for norm-referenced decisions tend to be relatively low. By contrast, for criterion-referenced decisions, estimates of reliability-like coefficients based on error-tolerance ratios tend to be noticeably larger. |
| |
| Keywords: | |
|
|
