The effects of POWERSOURCE© assessments on middle-school students’ math performance

This paper describes results from field testing of middle-school math formative assessments alongside professional development and instructional resources. We employed a randomised, controlled design to address the question: Does using our formative assessment strategies improve student performance on assessments of key mathematical ideas relative to a comparison group? This study also provided data on the instructional sensitivity of the assessments, which is part of the validation needed for formative assessments. Teachers were recruited from two districts and seven middle schools. Nineteen treatment and 17 comparison group teachers and their students were included in study analyses. Scores on extended response and short-answer questions indicated that students in the treatment group performed better than students in the comparison group who received the formative assessments alone. These findings demonstrate both the feasibility and value of including performance task-types in a brief assessment context.     

Teacher effect change model: latent variable regression 5-level hierarchical model

Asia Pacific Education Review - This paper proposes a teacher effect change model in the form of a latent variable regression 5-level hierarchical model (LVR-HM5). Using multiple years of student...     

Using growth models to monitor school performance: comparing the effect of the metric and the assessment

This paper investigates whether inferences about school performance based on longitudinal models are consistent when different assessments and metrics are used as the basis for analysis. Using norm-referenced (NRT) and standards-based (SBT) assessment results from panel data of a large heterogeneous school district, we examine inferences based on vertically equated scale scores, normal curve equivalents (NCEs), and nonvertically equated scale scores. The results indicate that the effect of the metric depends upon the evaluation objective. NCEs significantly underestimate absolute individual growth, but NCEs and scale scores yield highly correlated (r >.90) school-level results based on mean initial status and growth estimates. SBT and NRT results are highly correlated for status but only moderately correlated for growth. We also find that as few as 30 students per school provide consistent results and that mobility tends to affect inferences based on status but not growth – irrespective of the assessment or metric used.     

Latent variable modeling in the hierarchical modeling framework in longitudinal studies: a fully bayesian approach

This paper presents a strategy for specifying latent variable regressions in the hierarchical modeling framework (LVR-HM). This model takes advantage of the Structural Equation Modeling (SEM) approach in terms of modeling flexibility—regression among latent variables—and of the HM approach in terms of allowing for more general data structures. A fully Bayesian approach via Markov Chain Monte Carlo (MCMC) techniques is applied to the LVR-HM. Through analyzing the data from a longitudinal study of educational achievement, gender difference are explored in the growth of mathematical achievement across grade 7 through grade 10. Allowing for the fact that initial status effect to rates of change may differ for girls and boys, the LVR-HM is specified in a way that rates of change parameters are modeled as a function of initial status parameters and the interaction between initial status and gender.     

A multilevel latent growth curve approach to predicting student proficiency

Value-added models and growth-based accountability aim to evaluate school??s performance based on student growth in learning. The current focus is on linking the results from value-added models to the ones from growth-based accountability systems including Adequate Yearly Progress decisions mandated by No Child Left Behind. We present a new statistical approach that extends the current value-added modeling possibilities and focuses on using latent longitudinal growth curves to estimate the probabilities of students reaching proficiency. The aim is to utilize time-series measures of student achievement scores to estimate latent growth curves and use them as predictors of a dichotomous outcome, such as proficiency or passing a high-stakes exam, within a single multilevel longitudinal model. We illustrated this method through analyzing a three-year data set of longitudinal achievement scores and California High School Exit Exam scores from a large urban school district. This latent variable growth logistic model is useful for (1) early identification of students at risk of failing or of those who are most in need; (2) a validation or/and adequacy of student growth over years with relation to distal outcome criteria; (3) evaluation of a longitudinal intervention study.     

Children Left Behind in AYP and Non-AYP Schools: Using Student Progress and the Distribution of Student Gains to Validate AYP

The No Child Left Behind Act ( NCLB, 2002 ) establishes ambitious goals for increasing student learning and attaining equity in the distribution of student performance. Schools must assure that all students, including all significant subgroups, show adequate yearly progress (AYP) toward the goal of 100% proficiency by the year 2014. In this paper, we illustrate an alternative way of evaluating AYP that both emphasizes individual student growth over time and focuses on the distribution of student growth between performance subgroups. We do so through analyses of a longitudinal data set from an urban school district in the state of Washington. We also examine what these patterns tell us about schools that have been designated as meeting their AYP targets and those that have not. This alternative way of measuring AYP helps bring to light potentially important aspects of school performance that might be masked if we limit our focus to classifying schools based only on current AYP criteria. In particular, we are able to identify some schools meeting Washington state's AYP criteria in which above-average students are making substantial progress but below-average students making little to no progress. In contrast, other schools making AYP have below-average students making adequate progress but above-average students showing little gains. These contrasts raise questions about the meaning of "adequate" progress and to whom the notion of progress refers. We believe that closely examining the distribution of student progress may provide an important supplementary or alternative measure of AYP.     

Closing the Gap: Modeling within-school variance heterogeneity in school effect studies

Effective schools should be superior in both enhancing students’ achievement levels and reducing the gap between high- and low-achieving students in the school. However, the focus has been placed mainly on schools’ achievement levels in most school effect studies. In this article, we focused our attention upon the school-specific achievement dispersion as well as achievement level in determining effective schools. The achievement dispersion in a particular school can be captured by within-school variance in achievement (σ²). Assuming heterogeneous within-school variance across schools in hierarchical modeling, it is possible to identify school factors related to high achievement levels and a small gap between high- and low-achieving students. By analyzing data from the TIMMS-R, we illustrated how to detect variance heterogeneity and how to find a systematic relationship between within-school variance and school practice. In terms of our results, we found that schools with a high achievement level tended to be more homogeneous in achievement dispersion, but even among schools with the same achievement level, schools varied in their achievement dispersion, depending on classroom practices.