A comparison between value-added school estimates and currently used metrics of school accountability in California

This study compared a value-added approach to school accountability to the currently used metrics of accountability in California of Adequate Yearly Progress (AYP) and Academic Performance Index (API). Five-year student panel data (N?=?53,733) from 29 elementary schools in a large California school district were used to address the research questions. Results show the strong relationship between AYP and API to student background measures. Schools with a majority of students from low socioeconomic background lagged far below schools from more affluent context. Results from the value-added approach however, showed a strongly diminished relationship to student background. Under this model, several schools from a low socioeconomic context can be seen as high achieving. Additionally, little evidence was found that high levels of student achievement negatively affect school value-added scores. Schools that enroll large proportions of advanced students, which often do not show positive growth across years are not penalized under a value-added approach. Recommendations for policy and future research are discussed.     

Conceptual and empirical differences among various value-added models for accountability

Accountability systems in education generally include indicators of student performance. However, these indicators often differ considerably among the various systems. More and more countries try to include value-added measures, mainly because they do not want to hold schools accountable for differences in their initial intake of students. This study presents a conceptual framework of these value-added measures, resulting in an overview of 5 different types. Using data from Dutch secondary schools, we empirically provide estimates of these different measures. Our analyses show that the correlation between the different types of school effects estimated is rather high, but that the different models implicate different results for individual schools. Based on theoretical considerations, arguments are given to use the following indicators in the value-added accountability models: prior achievement, student-level background characteristics, and compositional characteristics of the student population.     

Creativity and performativity policies in primary school cultures

Cultures of performativity in English primary schools refer to systems and relationships of: target‐setting; Ofsted inspections; school league tables constructed from pupil test scores; performance management; performance related pay; threshold assessment; and advanced skills teachers. Systems which demand that teachers ‘perform’ and in which individuals are made accountable. These policy measures, introduced to improve levels of achievement and increased international economic competitiveness, have, potentially, profound implications for the meaning and experience of primary teachers’ work; their identities; their commitment to teaching; and how they view their careers. At the same time as policies of performativity are being implemented there is now increasing advocacy for the adoption and advancement of ‘creativity’ policies within primary education. These major developments are being introduced in the context of a wide range of social/educational policies also aimed at the introduction of creativity initiatives into schools and teaching. This complex policy context has major implications for the implementation process and also primary teachers’ work and how they experience it. The ethnographic research reported in this article has been conducted over a school year in six English primary schools in order to analyse the effects of creativity and performativity policy initiatives at the implementation stage. The article concludes by arguing that in the schools of our research the drive to raise pupil test scores involves both performative and creative strategies and that this critical mediation goes beyond amelioration toward a more complex view of professional practice. Implementing creativity and performativity policies provided important contextual influencing factors on teacher commitment. These were: curriculum coverage and task completion; and providing psychic rewards of teaching.     

Do trainee teachers harm pupil attainment? Isolating the effect of pre-service teachers on contemporaneous pupil performance in ‘high-stakes’ tests

The prevalence of ‘pre-service’ or ‘trainee’ teachers in schools is rising in England, driven by the expansion of school-led routes to qualified teacher status and increasing demand for newly qualified teachers. This may have important implications for schools, which have historically been concerned with the impact of trainee teachers on their pupils’ attainment. There are, however, confounding factors which affect both the decision to host a trainee teacher and pupil attainment. We empirically model the impact of trainee teachers on contemporaneous pupil attainment in ‘high-stakes’ exams, exploiting unique data combining national administrative data on pupil test scores with a survey of schools’ involvement with initial teacher training over multiple academic years. We use school fixed effects to account for time-invariant school factors which may determine both schools’ teacher training decisions and pupil attainment. Counter to schools’ concerns, we find that pupil attainment in high-stakes assessments, on average, is not significantly affected by the number of trainee teachers. This is an important empirical finding, as it suggests that the rapid expansion of school-led teacher training is not likely to have a detrimental effect on pupil attainment in England, conditional on the set of schools that choose to engage with initial teacher training remaining similar: trainee teachers may still affect pupil attainment in schools that do not currently participate in initial teacher training, as these schools are typically more constrained.     

Diversity and progress in school accountability systems in Australia

School education in Australia is a complex interplay between federal and state governments, and between government and non-government schools. This article explores the supervision of schools in Australia through school accountability systems. Utilising publicly available documents a systematic analysis of the state and territory systems for government schools is provided. It is a paper that attempts to document rather than critique school accountability, although a conceptual framework utilising contractual, moral and professional accountability is used to analyse the different accountability processes reported upon. Contractual and moral accountability is supported by most systems, whilst there is potential to foster professional accountability in only two systems. Fostering professional accountability is important because this is where the internal motivation of teachers helps to drive school improvement. When compared to leading-edge systems, Australian accountability systems are lacking in judgements on teaching practice in individual classrooms, and the use of sophisticated measures of learning and value-added analysis.     

Failed regulations and school composition: selective admission practices in Chilean primary schools

This is an exploratory study about pupil selection. Admission regulations are central to understanding issues of school mix, segregation and educational justice. In Chile, student selection has been intensively discussed but scarcely studied. Using a questionnaire for headteachers (N = 581), we explore how school admission processes are organised and implemented within a market-based educational system with a high-stakes testing regime where schools are pushed to use selection practices. Despite the existence of a national (although weak) legal prohibition at the time the survey was conducted, half of the headteachers state that they use some mechanism to select students (play sessions, student testing, or interviews with parents) even when they do not face demand pressures. Exploratory data suggest higher levels of selectivity in subsidised private schools. Regression analysis shows a strong association between ‘selectivity’ and homogeneous academic intake and social composition. Presumably, the widespread use of selection procedures responds to the structuring force of a double system of educational accountability; strong pressure on performance within a testing regime and where funding depends on enrolments makes selection a fundamental strategy for schools, but which is seriously affecting equality goals.     

Value-added indicators for a fairer Chilean school accountability system: a pending subject

ABSTRACT

Although schools’ relative contribution to pupils’ progress is increasingly used in accountability systems around the world, momentum for value-added models (VAM) has not been reached in Chile. This small-scale study explores qualitatively the policy context in which this omission takes place, by analyzing policy documents and interviewing local policymakers and researchers about their views on VAM. These agents and official documents not only point to political, methodological and pragmatic reasons for and against value-added indicators, but also highlighted ethical and legal reasons pro (but not against) VAM. Overall, the most prominent reason for including VAM into the new accountability system was the ethical consideration. The notion that a fairer indicator could make justice, especially for those schools working in disadvantaged contexts, was a recurrent idea. In contrast, the most recurrent reasons against VAM were methodological. Whilst research on VAM for the Chilean school system has been conducted over the last decade, it has impacted very little the policy arena. Given that the creation of frameworks for assessing value-added indicators takes time, needs policy and research support and needs to consider potential unintended consequences, the road towards including them into the Chilean school accountability system is just starting.     

Pupil mobility,attainment and progress in primary school

This article presents an analysis of the association between pupil mobility and educational attainment in the 2002 national end of Key Stage 2 (KS2) tests for 11‐year‐old pupils in an inner London education authority. The results show that pupil mobility is strongly associated with low attainment in the end of key stage tests. However, the negative association with pupil mobility is reduced by half when account is taken of other pupil background factors known to be related to educational attainment (such as special educational need and socio‐economic disadvantage), and is eliminated entirely when account is also taken of pupils' prior attainment as indicated by end of KS1 test scores at age 7. Thus there is no indication that changing school has a negative impact on educational progress during primary school. Pupils who join their school during KS2 from other schools in England are more likely to be ‘at risk’ of low attainment due to higher levels of socio‐economic disadvantage, a greater need for support in relation to English as an additional language, a higher incidence and greater severity of special educational needs and pre‐existing low attainment at the end of KS1. A key factor in understanding the relationship between mobility and attainment is the reason for mobility. One‐third of mobile pupils had arrived from schools outside of England, often as refugees, asylum seekers or economic migrants, and these pupils accounted for the major part of the effect ascribed to ‘pupil mobility’. The low attainment of these pupils is the result not of ‘changing school’ but of a broad range of factors including substantial cultural, educational and social adjustment.     

Parent preferences and parent choices: the public–private decision about school choice

School choice survey data from the Metropolitan Nashville Public Schools, a large county‐wide school district, is analysed to examine the characteristics of parents who consider choosing private schools for their children and those who do not. We examine differences in background, including race, educational attainment and socioeconomic status, as well as differences in parent satisfaction with their child’s previous school, parent involvement in school, parents’ priorities in school choice, as well as parents’ social networks. After controlling for background characteristics, we find that parent satisfaction with their child’s previous school was not a predictor of considering a private school. Rather, parent involvement seems to be a more important indicator of whether or not a parent would consider sending their child to a private school. In this case, parents are not ‘pushed’ away from public schools, contrary to much public rhetoric that suggests private schools are somehow inherently ‘better’ than public schools and parents who are dissatisfied with their public schools will opt for private schools. Instead, these findings suggest a ‘pull’ towards private schools. Parents may perceive that parent involvement and parent communication are more easily facilitated and valued in private schools.     

Contextualising Catholic school performance in England

Despite there being significant numbers of state maintained Catholic schools in England, they have, until recently, proved to be of interest only to a minority of researchers. Government initiatives to promote greater scrutiny and accountability through the publication of school test and examination results have generated interest in their academic performance from proponents and critics. Early attempts by government to compare school performance using ‘raw’ examination and test scores were severely criticised. Newly introduced ‘Contextualised Value Added’ measures, when correctly applied, have largely overcome the deficiencies of earlier datasets. This paper presents new CVA performance data provided by Ofsted that may help the debate about Catholic school effectiveness to progress. Possible reasons for the CVA findings are explored and further areas for research are suggested.     

The Effect of Ignoring Classroom‐Level Variance in Estimating the Generalizability of School Mean Scores

Contemporary educational accountability systems, including state‐level systems prescribed under No Child Left Behind as well as those envisioned under the “Race to the Top” comprehensive assessment competition, rely on school‐level summaries of student test scores. The precision of these score summaries is almost always evaluated using models that ignore the classroom‐level clustering of students within schools. This paper reports balanced and unbalanced generalizability analyses investigating the consequences of ignoring variation at the level of classrooms within schools when analyzing the reliability of such school‐level accountability measures. Results show that the reliability of school means cannot be determined accurately when classroom‐level effects are ignored. Failure to take between‐classroom variance into account biases generalizability (G) coefficient estimates downward and standard errors (SEs) upward if classroom‐level effects are regarded as fixed, and biases G‐coefficient estimates upward and SEs downward if they are regarded as random. These biases become more severe as the difference between the school‐level intraclass correlation (ICC) and the class‐level ICC increases. School‐accountability systems should be designed so that classroom (or teacher) level variation can be taken into consideration when quantifying the precision of school rankings, and statistical models for school mean score reliability should incorporate this information.     

Test Score or Student Progress? A Value-Added Evaluation of School Effectiveness in Urban China

Outcome-oriented evaluation of school effectiveness is often based on student test scores in certain critical examinations. This study provides another method of evaluation—value-added—which is based on student achievement progress. This paper introduces the method of estimating the value-added score of schools in multi-level models. Based on longitudinal student achievement data, two measures of school effectiveness in one local education authority in China are compared. It is found that the between-school difference in both test-score and value-added is large comparable with that of Western countries. The results of the two measures of school effectiveness are highly different. The value-added measures lack consistency across different subject areas within schools while the test score measures are highly correlated between subjects. Teachers show their preference for value-added measures over test-score measures of education quality. It is suggested that value-added measures of school effectiveness should be used as a complement to rather than a substitute for test-score measures. The shortcomings of value-added approach are also discussed.     

English language acquisition and educational attainment at the end of primary school

This paper analyses the national key stage 2 test results for 2300 11‐year‐old pupils in an inner London LEA. A range of concurrent pupil background data was also collected, including whether pupils spoke English as an additional language (EAL), and if so, their stage of fluency in English. EAL pupils at the early stages (1–3) of developing fluency had significantly lower KS2 test scores in all subjects than their monolingual peers. However, EAL pupils who were fully fluent in English achieved significantly higher scores in all KS2 tests than their monolingual peers. The negative association with attainment for the early stages of fluency remained significant after controls for a range of other pupil characteristics, including age, gender, free school meal entitlement, stage of special educational need and ethnic group, although these factors effectively explained the higher attainment of the ‘fully fluent’ group. We conclude that EAL is not itself a good guide to levels of attainment, and a measure of stage of English fluency is necessary to interpret associations with test performance. Alternative measures which focus only on the very early stages of English proficiency, such as the QCA ‘language in common’ steps, are inadequate to assess the impact of bilingualism for all but the very earliest learners of English. Given the uneven distribution of EAL pupils across the country, those schools and local education authorities with high concentrations of pupils in the early stages of learning English are likely to be adversely affected in school achievement and attainment tables. The policy implications for national data collection and for the use of such data are considered.     

Mediated diffusion: Translating professional practice across schools in a high-stakes system

Educational systems across the globe are attempting to reshape the vertical and horizontal dimensions of school accountability. The vertical dimension involves devolution of responsibility to individual schools while the horizontal typically entails promoting the professionalism of leaders and teachers through school networks and school-to-school pairings. In many systems, these shifts occur within an environment of high-stakes accountability. Little is known about the ways that high-stakes consequences for individual schools shape the translation of professional practice across schools. The study reported here traces the diffusion of data-informed practices in a formal pairing of two primary schools in England—a school ranked as “outstanding” and a school in “special measures,” a persistently low-achieving school. Study findings trace the ways in which the threat of consequences under which the low-achieving school operates shapes the diffusion of professional practice across schools. The study details how the lead school brokered, or mediated, state requirements, co-constructing the supported school’s narrative of improvement. Findings from this study illuminate the ways that vertical relationships within the system of schooling influence horizontal relationships across schools. Clarifying the shaping of professional practice across schools is increasingly critical as patterns of provision of schooling proliferate and the links among the state and schools become increasingly complex.     

The ethics of competition: accountability policy enactment in Chilean schools’ everyday life

ABSTRACT

Public policies have a moral order, an ethical horizon. They offer a vocabulary of imagined micro-policies. Using the case of Chile, this paper examines the ways in which accountability policies are reworked within schools and how they affect actors’ subjectivities. It adds new findings to the existing body of research on school accountability policies, offering in-depth evidence based on the case of Chile, which has a high-stakes testing model and a widespread competitive voucher system. The research is based on case studies of ten public and private subsidised schools, framed by a sociological perspective of policy enactment theory. The research findings show the ways that accountability policies are recreated, expanded, and intensified at the local level, permeating an ethic of competition. The analysis focuses on three qualitative trends: school actors’ sense-making of test scores and labels; zones of safety and risk for teachers under an accountability regime; and the emergence of a sticky web of persuasion, surveillance, and coercion among school members in order to improve performance. The practices examined are not understood as ‘secondary effects’ or an ‘implementation problem’, as if they occur unconnected from the policy rationale. The outcomes are consistent with the policy itself in interaction with school life.     

Exploring how the school context mediates intern learning in underserved rural border schools

This research used poststructural theories to examine a crucial issue of teacher-learning in rural border schools that are under pressure from high-stakes school accountability, fewer resources, and significant numbers of English language learners (ELLs). The methodology was based on a multiple case study of four intern teachers who participated in the research for 16 weeks. The sources of data included classroom observation, interview, reflective essay, and field note. The findings showed that the interns deconstructed teacher-learning and constructed ways of teaching within contradictory and conflicting school policies and practices.     

A multilevel latent growth curve approach to predicting student proficiency

Value-added models and growth-based accountability aim to evaluate school??s performance based on student growth in learning. The current focus is on linking the results from value-added models to the ones from growth-based accountability systems including Adequate Yearly Progress decisions mandated by No Child Left Behind. We present a new statistical approach that extends the current value-added modeling possibilities and focuses on using latent longitudinal growth curves to estimate the probabilities of students reaching proficiency. The aim is to utilize time-series measures of student achievement scores to estimate latent growth curves and use them as predictors of a dichotomous outcome, such as proficiency or passing a high-stakes exam, within a single multilevel longitudinal model. We illustrated this method through analyzing a three-year data set of longitudinal achievement scores and California High School Exit Exam scores from a large urban school district. This latent variable growth logistic model is useful for (1) early identification of students at risk of failing or of those who are most in need; (2) a validation or/and adequacy of student growth over years with relation to distal outcome criteria; (3) evaluation of a longitudinal intervention study.     

Have State Accountability and High-Stakes Tests Influenced Student Progression Rates in High School?

The question addressed in this paper is whether there are signs that students' progression through high school to graduation has been affected by the efforts to increase school accountability, including high-stakes exit exams. If such accountability systems were able to raise student academic performance significantly in elementary and middle school, these same students should be more likely to succeed academically in high school and graduate with their cohort. If state accountability systems also attempt to raise minimum standards for high school graduation by requiring exit tests, as many do, this could have the opposite effect of lowering high school graduation rates by putting yet one more barrier in the way of lower-performing students completing requirements.     

Languages,cultural capital and school choice: distinction and second-language immersion programmes

This paper argues that languages, increasingly marginalised in schools in English-speaking countries, are gaining ‘elitist’ ground as part of the ‘value-added’ marketisation of schools and parents’ desire for their children to gain ‘positional goods’ through schooling. In arguing our case, the paper draws on survey and other data derived from second-language immersion programmes in two Queensland secondary schools, where key learning areas such as mathematics and science are taught through the medium of another language. As a corollary, we also argue that some schools – in our case, government schools – are using their immersion programmes as markers of distinction in a period of post-comprehensive schooling and emerging school markets, which includes both government and non-government schools. There is also a global policy context to such programmes in respect of countries such as Spain, China and Germany supporting the teaching of their respective languages in nations around the world.     

Reassessing school effectiveness: Multi-objective value-added measures (MOVAM) of academic and socioemotional learning

Under the Every Student Succeeds Act (ESSA), it is imperative that American public schools assess and improve not only academic but also non-academic measures of student learning such as socioemotional skills. The policy shift towards broad-based school accountability calls for reassessing school effectiveness from whole child development perspectives and addressing potential biases and limitations of conventional value-added measures (VAM). Through multivariate multilevel analyses of the Early Childhood Longitudinal Study-Kintergarten (ECLS-K): 2011 data, this study applies multi-objective value-added measures (MOVAM) approach to assess and improve school effectiveness for academic and socioemotional learning. The study results show different patterns of academic vs. socioemotional learning gains, and also weak correlations between school effects on the two types of learning outcomes. Nevertheless, the comparisons of academically and socioemotionally effective vs. ineffective schools imply that schools can and should improve both academic and socioemotional learning outcomes through synergistic improvement of key organizational and instructional conditions.