Teachers' Assessments of Students' Achievements: The Ecological Validity of Studies Using Case Vignettes

In two studies, we investigated the extent to which research using case vignettes of fictitious students is able to yield research results that are ecologically valid representations of teachers' assessments of students' educational achievement in their real classrooms. The type of assessment was teachers' tracking decisions from primary to secondary school. In Study 1, which was conducted in the German educational system, teachers decided which secondary school track their own students would attend and were subsequently confronted with the same decision for fictitious students described in vignettes. Study 2 was a replication in the Luxembourgish educational system. Ordinal and logistic regression models were used to compare decision making in the two conditions of actual students versus case vignettes. No major differences were found between the two conditions. Results are discussed with respect to the ecological validity of research designs that use case vignettes.