Measuring value added effects across schools: Should schools be compared in performance?

This article traces the evolution of a quest for solving the problems involved in the analysis of multilevel data and the estimation of the value added effects of schools in influencing educational outcomes. The authors report the findings of two studies that followed several cohorts of students that were tested at two grade levels (Grade 3 and Grade 5) and at three grade levels (Grade 3, 5, and 7) respectively with basic skills tests of literacy and numeracy in a single school system. The tests were calibrated using Rasch scaling and equated using concurrent equating procedures for approximately 8000 students in 440 schools. Hierarchical linear modelling was employed in the analysis with different multilevel models used in the two studies to assess relative and absolute change in performance respectively. The findings of these studies show that with different regression models and different variables taken into consideration there are very different estimates of the variance between schools and under some circumstances the residual variance between schools is very small. Research is clearly needed into the procedures of analysis and the different value added effects that could and should be employed.