Achievement Measures of School Effectiveness: Comparison of Model Stability Across Years

The purpose of this study was to determine the feasibility of combining different test types (criterion-referenced and norm-referenced) in a composite school achievement score to be used in a model for school effectiveness classification. The cross-year stability and within-model consistency of the composite was compared to models using subcomposite, overall scores for both the criterion- referenced and norm-referenced tests, subject-area scores (across grades), grade-level scores, and component scores for each grade. Stability of the different models across 2 years was determined by using the agreement ratio, kappa coefficient, and correlation of residuals (N = 361). The same statistical procedures were used to compute consistency across subsamples (N = 264). Results indicated that transforming and combining student-level scores of different test types, grade levels, and subject areas allows for a broader basis for judging schools and provides a school effectiveness model that is both consistent across subsamples and stable across years.