Quantifying Error in Survey Measures of School and Classroom Environments

Developing indicators that reflect important aspects of school and classroom environments has become central in a nationwide effort to develop comprehensive programs that measure teacher quality and effectiveness. Formulating teacher evaluation policy necessitates accurate and reliable methods for measuring these environmental variables. This article investigates different approaches to quantify measurement error in school- and classroom-level indicators constructed from survey data collected at a lower level of aggregation (i.e., teachers or students). Within a generalizability theory framework the article first compares four widely used approaches for accounting for measurement error in school- and classroom-level aggregate indicators. Then, it uses two empirical examples to demonstrate how each of these approaches can lead to different conclusions about the precision of aggregate indicators, and can influence inferences about the relationships between environmental variables and policy-relevant outcomes. Finally, the article discusses the degree to which these commonly used models accurately represent the structure of the data found in common survey administration scenarios.