The importance of adjusting for pupil background in school value-added models: A study of Progress 8 and school accountability in England

In the UK, USA and elsewhere, school accountability systems increasingly compare schools using value-added measures of school performance derived from pupil scores in high-stakes standardised tests. Rather than naïvely comparing school average scores, which largely reflect school intake differences in prior attainment, these measures attempt to compare the average progress or improvement pupils make during a year or phase of schooling. Schools, however, also differ in terms of their pupil demographic and socioeconomic characteristics and these factors also predict why some schools subsequently score higher than others. Many therefore argue that value-added measures unadjusted for pupil background are biased in favour of schools with more ‘educationally advantaged’ intakes. But others worry that adjusting for pupil background entrenches socioeconomic inequities and excuses low-performing schools. In this article we explore these theoretical arguments and their practical importance in the context of the ‘Progress 8’ secondary school accountability system in England, which has chosen to ignore pupil background. We reveal how the reported low or high performance of many schools changes dramatically once adjustments are made for pupil background, and these changes also affect the reported differential performances of regions and of different school types. We conclude that accountability systems which choose to ignore pupil background are likely to reward and punish the wrong schools and this will likely have detrimental effects on pupil learning. These findings, especially when coupled with more general concerns surrounding high-stakes testing and school value-added models, raise serious doubts about their use in school accountability systems.     

The impact of school SES composition on science achievement and achievement growth: mediating role of teachers’ teachability culture

ABSTRACT

Previous studies have shown that schools’ socioeconomic-status (SES) composition has an impact on the academic performance of pupils. Less attention has been given to the explanation of this effect. This study examined whether the teachability culture among the school staff (teachers’ collective beliefs about how teachable their pupils are) mediated the school SES effect on science achievement and achievement growth. Multilevel analyses were conducted with data from 1,761 pupils and 1,255 teachers across 66 primary schools in Flanders. First, the analyses indicated that there was a positive association between school SES composition and teachability culture: Even after controlling for cognitive ability and performance of pupils, there was a more pessimist culture in socioeconomically disadvantaged schools. Second, the association between school SES and academic performance was explained/mediated by the teachability culture. However, no school effects or mediation effects were found for achievement growth as the covered period of academic growth was too short.     

Schools, school quality and achievement growth: Evidence from the Philippines

A broad literature seeks to assess the importance of schools, proxies for school quality, and family background on children's achievement growth using the education production function. Using rich data from the Philippines, we introduce and estimate a model that imposes little structure on the relationship between intake achievement and follow-up achievement and evaluate school performance based on this estimated relationship. Our methods nest typical value-added specifications that use test score gains as the outcome variable and models assuming linearity in the relationship between intake and follow-up scores. We find evidence against the use of value-added models for our data and show that such models give very different assessments of school performance in the Philippines. Using a variety of tests we find that schools matter in the production of student achievement, though variation in performance across schools only explains about 4.4–5.3% of the total (conditional) variation in follow-up achievement. Schools providing basic facilities—in particular schools providing electricity—are found to perform much better in the production of achievement growth.     

Self-concept in children receiving special support at school

The aim of this study is to investigate the variations of self-concept in pupils attending basic compulsory schools and special units. It also examines self-concept in pupils receiving support according to different models. The study involved 183 pupils aged nine to 13 years, attending basic compulsory schools or special programme units in Sweden. Seventy-seven pupils were receiving support from special educators, who have completed a supplementary programme in special education. A comparison was made between pupils receiving support and pupils receiving no support and between pupils in different organization models of support. Global self-concept at school does not seem to be related to the model of special support. Pupils at compulsory schools who are receiving support differ from pupils who receive no support, in that peer relations appear to be more important for their self-concept at school. This could be interpreted as being a compensatory strategy to maintain a good self-concept in spite of difficulties at school. Some implications for educational strategies are discussed.     

Do Testing Conditions Explain Cyber Charter Schools’ Failing Grades?

Recent Center for Research on Education Outcomes (CREDO) analyses find that cyber charter schools in seventeen states show consistently low reading and mathematics value-added test scores compared to traditional public schools serving comparable students. This generally accords with prior research. We hypothesize that the relatively poor measured academic value-added of cyber charters reflects artificial testing conditions for students in those schools. Accordingly, we have collected testing information from the seventeen CREDO states. State-level analyses find that cyber student persistence, which likely indicates school quality, correlates moderately and significantly with the cyber student academic value-added as measured by CREDO. Further, we find evidence of lower cyber school value-added in states which permit cybers to use narrow testing windows, perhaps reflecting testing fatigue on the part of test-takers. We discuss implications, and suggest next steps for research exploring whether testing conditions affect measured cyber charter performance.     

Social capital in the classroom: a study of in-class social capital and school adjustment

Social capital is generally considered beneficial for students’ school adjustment. This paper argues that social relationships among pupils generate social capital at both the individual and the class levels, and that each has its unique effect on pupils’ performance and well-being. The sample in this study consists of 1036 children in 60 first-grade classes in 46 Dutch elementary schools. Multilevel regression results show that a substantial proportion of the variance in school adjustment can be attributed to the class level and that both individual-level and classroom-level social capital have substantial effects on school adjustment. At the individual level, the size of one’s network is more important than its structure. At the collective level, social capital also has a ‘dark side’ because it can have negative effects on adjustment, lowering the academic performance in a class.     

Reassessing school effectiveness: Multi-objective value-added measures (MOVAM) of academic and socioemotional learning

Under the Every Student Succeeds Act (ESSA), it is imperative that American public schools assess and improve not only academic but also non-academic measures of student learning such as socioemotional skills. The policy shift towards broad-based school accountability calls for reassessing school effectiveness from whole child development perspectives and addressing potential biases and limitations of conventional value-added measures (VAM). Through multivariate multilevel analyses of the Early Childhood Longitudinal Study-Kintergarten (ECLS-K): 2011 data, this study applies multi-objective value-added measures (MOVAM) approach to assess and improve school effectiveness for academic and socioemotional learning. The study results show different patterns of academic vs. socioemotional learning gains, and also weak correlations between school effects on the two types of learning outcomes. Nevertheless, the comparisons of academically and socioemotionally effective vs. ineffective schools imply that schools can and should improve both academic and socioemotional learning outcomes through synergistic improvement of key organizational and instructional conditions.     

Real estate values,school quality,and the pattern of urban development in Charlotte,North Carolina

This study presents estimates of a model that includes the racial composition and academic quality of public schools among the determinants of housing prices. Results suggest that academic quality is a more important factor in the determination of housing values than the racial ratio of pupils in area schools. Drawing on estimates of the model, a school preference index (SPI) is developed. The SPI is found to be positively correlated to the level of the academic achievement and negatively related to the level of nonwhite school enrollment. The SPI also is shown to be positively correlated to the level of new residential construction.     

Lower secondary school teachers' judgements of pupils' problems

Lower secondary school teachers from six different schools in one Norwegian municipality reported on the prevalence of problems among pupils entering lower secondary school and types of problems among pupils in Grades 8–10. Teachers report that the transition from Elementary to Lower secondary school is problematic for approximately 30% of the pupils. About 70% of the teachers report that 25% or more of pupils transitioning to Grade 8 lack academic experiences and skills and have problems following directions. There is also a tendency for teachers who report higher values on professional (un)certainty about pupils' learning and on two efficacy variables included in the study, to be more favourably inclined towards inclusion of children who are perceived as having learning or behaviour problems. Additionally, teachers who report higher values on efficacy scales tend to report lower values on problems among pupils, and teachers who are less favourable towards inclusion report higher numbers of pupils having problems. The results are discussed in relation to schools as workplaces and professional development for pre‐service and in‐service teachers.     

Track choice,school engagement and feelings of perceived control at the transition from primary to secondary school

The transition from primary to secondary education is regarded as a crucial phase in pupils’ school careers. Changes in the school environment have a negative influence on pupils’ perceived control and engagement. However, until now little attention has been devoted to the role of the onset of ability grouping therein, which often coincides with the start of secondary education. Research has shown that students in non-academic tracks display lower levels of perceived control and engagement. In this study we examine the relation between pupils’ prospective track choice and feelings of perceived control and behavioural and cognitive engagement before the transition to secondary education. Stepwise multilevel regression models were run on data collected from pupils in their last year of primary education in 36 schools in the cities of Antwerp and Ghent (Flanders, Belgium) in May–June 2016. The results show that pupils who indicated that they would start secondary education in non-academic tracks displayed lower levels of perceived control and behavioural and cognitive engagement than pupils who indicated that they would start in an academic track. Further analyses suggest that teacher assessments of pupils’ competence play an important role in explaining these differences according to prospective track choice—with regard to perceived control and behavioural disengagement, this effect is established net of students’ actual competence. This study demonstrates that differences in perceived control and engagement according to track originate in primary education, and that primary school teachers play a vital part in labelling students according to their future careers.     

Education reform and the academic performance of public and private secondary school students in Thailand

Taking advantage of newly available data on a standardized national examination for secondary school students in Thailand, this study attempts to explain the variation in the average performance of students using school and local characteristics. Special emphasis is placed on trying to understand the asymmetric success factors of public and private secondary schools, and the possible effects of their coexistence. Results from a propensity score matching model suggest that private schools may create positive spillovers on the academic performance of students attending public schools located in the same district. The main implication from the research is that a more careful discussion about the impact of the ongoing education reforms on private schools is needed.     

Collaboration and integration of services in Greek special schools: two different models of delivering school services

Multidisciplinary collaboration is considered to be very important for the education of pupils with special educational needs and particularly those pupils with the most severe disabilities. This research adopts a multiple case‐study design in order to understand collaboration and the integration of services and the effectiveness of these among pupils with profound and multiple learning difficulties (PMLD) in Greece. The teams, choosing case studies in five special schools, considered 10 pupils and their parents. Reviews and analyses of the legislation; the use of diaries; participant observations and semi‐structured interviews were used for the collection of data. Two different models of service integration which provide different opportunities to the inclusion of pupils with PMLD were revealed since schools have different methods of integrating health and social professionals. In the first model, the school works with outside services, whereas in the second, services are within the school. In addition, roles and responsibilities differ within the different models. Service integration and collaboration were found to be more effective in schools where teachers and health and social professionals work under the same management. The study suggests the expansion of the second model of multidisciplinary collaboration with the integration in schools of health therapists, educational psychologists and social workers.     

The Catholic school advantage and common school effect examined: a comparison between Muslim immigrant and native pupils in Flanders

This study investigates the impact of Catholic schooling on academic achievement of native Belgian and Muslim immigrant pupils. The distinctive characteristics of Catholic schools in Belgium (Flanders) form an exceptionally suitable context to study this. Multilevel latent growth curve analyses are conducted with data from approximately 5,000 pupils across 200 primary schools. No support was found for the Catholic school advantage hypothesis as the overall achievement growth for math and reading was not significantly better in Catholic schools than in public schools. Likewise, no evidence was found for the so-called “common school effect” hypothesis: The learning growth of Muslim pupils was not significantly better in Catholic schools. In fact, the initial achievement gap was found to be higher in Catholic schools than in public schools. Implications of these findings are discussed.     

Pupil Performance,Absenteeism and School Drop-out: A Multi-dimensional Analysis

This article assesses whether second-level schools in Ireland, typically covering pupils 12 to 18 years of age, are equally effective in relation to three different outcomes: examination performance, absenteeism and potential drop-out among pupils. The article uses data from a large-scale survey of second-level pupils in 116 schools in Ireland. Analysis is restricted to one cohort: pupils aged 15-16 years who took a nationally standardised examination, the Junior Certificate, in 1994. Multivariate multi-level modelling techniques are used to assess the impact of pupil background and schooling factors on overall examination performance, on absenteeism levels and on intentions to leave school after the exam. Some consistency is found among these different dimensions of school effectiveness: pupil absenteeism and potential drop-out rates are lower in schools which enhance academic progress among pupils. These outcomes are associated with more positive teacher-pupil relations and a more positive academic climate within the school.     

THE ROLE OF SCHOOL LEVEL PROTECTIVE FACTORS IN OVERCOMING CUMULATIVE RISK FOR BEHAVIOR DIFFICULTIES IN CHILDREN WITH SPECIAL EDUCATIONAL NEEDS AND DISABILITIES

The study investigated whether school‐level protective factors could moderate the effects cumulative risk has upon behavior difficulties in children with special educational needs and disabilities (SEND). The sample comprised 4,288 children identified with SEND: 2,660 pupils within 248 primary schools, and 1,628 pupils within 57 secondary schools. Risk factors associated with increases in behavior difficulties over an 18‐month period were summed to a cumulative risk score. Various school‐level factors were added to multilevel models, with interaction terms computed between cumulative risk and these variables to assess their potential protective effects. The primary school model revealed a significant interaction between cumulative risk and school academic achievement in predicting behavior difficulties. Higher levels of achievement in primary schools help reduce behavior difficulties for children most at risk. The secondary school model evidenced a significant interaction between cumulative risk and school percentage of students eligible for free school meals (FSM). Lower proportions within a school of children eligible for FSM were associated with reductions in behavior difficulties for children at high levels of risk. Interventions aimed at improving school‐level academic achievement and targeting high‐risk students attending schools with large proportions of children eligible for FSM would be beneficial.     

The effect of pre-school education on academic performance in primary school: a case study of grade one pupils in Botswana

Evidence abounds in the literature of a direct link between pre-primary education and academic performance in the primary school. The salutary effect of the ‘Head start’ programme inaugurated in the United States of America in the early 1960s on the academic performances of its beneficiaries in the lower primary is such a piece of evidence. Premised on this commonality between pre-primary education and academic achievement in the primary school, the study aimed at finding out whether there were significant differences in the performances of Botswana grade one pupils with pre-school education experience and their counterparts without such an experience on selected tasks in English language, mathematics and science. Using purposive sampling technique for school selection, a total of 120 grade one pupils were randomly selected for the study from four selected primary schools in Botswana. In addition, 20 grade one teachers from the study schools participated in the study. For data collection purpose, each pupil-subject was individually interviewed for about 20 min on the study tasks; and the opinions of the twenty teachers on the subject of the study was sampled by the use of a questionnaire. The results of the study indicated that pupils with pre-school education experience significantly out-performed their counterparts without such experience in all the three school subject areas surveyed by the study. This trend of the impact of pre-school education on academic achievement at the early primary school level was corroborated by the opinions of the primary school teachers. The paper concluded by observing that pre-school education equips children with pre-requisite skills which make learning in grade one easier and faster for children so exposed.     

School effectiveness research findings in the Portuguese speaking countries: Brazil and Portugal

This paper provides findings of research on school effectiveness and discusses implications for evaluation in Brazil and Portugal. Most findings reported over the last decade have been published in Brazilian or Portuguese refereed journals. Thus, a brief literature review of such studies enables that knowledge to reach international scholars and researchers. The magnitude of school effects obtained from longitudinal and cross-sectional data modelling is presented and discussed. In particular, a value-added approach based on multilevel models is used to explore consistency across models with different controlling variables and across different curricular contents, and stability over time. The results show a great deal of regional disparities regarding educational outcomes that are related to pupils’ socioeconomic and prior achievement heterogeneity. In addition, evidence is given for stronger school effects in primary than in lower secondary education, larger amplitude of the variance partition coefficient in primary than in lower secondary education, stronger consistency of value-added estimates across models with different controlling variables and moderate consistency across different curricular contents. Weak-to-moderate stability of value-added estimates is also shown when yearly measured compared to when measured by cycles of education, and moderate-to-strong stability when measured in different curricular contents. Recommendations are outlined in terms of how the results could be used to mentoring evaluation in Portuguese speaking countries and enhance school improvement.     

School engagement and rights‐respecting schools

Disengagement in school is associated with behavioral problems and decreased academic achievement. In contrast, pupils who are engaged in school develop the academic and social efficacies that underlie successful adulthood. Moreover, engagement promotes educational resilience. This study examines pupils’ self‐reported level of engagement in schools that are explicitly respecting of children’s rights compared with pupils in traditional schools. The Young Students’ Engagement in School Scale was developed and used with 1289 9‐ to 11‐year‐olds from 18 schools, six of which had fully implemented the Hampshire Education Authority’s Rights Respect and Responsibility (RRR) Initiative. Factor analysis indicated four dimensions of engagement: rights‐respecting climate; interpersonal harmony; academic orientation; and participation. Pupils in RRR schools had higher scores on all but the academic dimension. The findings suggest the potential of rights‐respecting schools in promoting engagement and the potential utility of the measure in identifying areas in which pupils’ engagement may need intervention.     

The influence of the school in the decision to participate in learning post‐16

The primary aim of this study, funded by the Department for Education and Skills, was to identify the nature and influence of school‐based factors in the choices of young people about their post‐16 education, training and career pathways. The study also contributes to the wider understanding of ‘choice’, and identifies implications for the development of careers education and guidance and decision‐making awareness amongst pupils and students in schools. It also further enhances the modelling of pupil decision making in education and training markets, and in labour markets. The research is based on a series of qualitative interviews in 24 schools across nine local education authorities. Focus groups were undertaken with young people in years 10, 11 and 12. Interviews were also conducted with head teachers, heads of year and heads of careers. A postal survey of parents was also undertaken. Four key school‐based factors were found to have a very strong influence in the choices and decisions of young people about their post‐16 education, training and career pathways. These were: whether the school had a sixth form or not; the characteristics of school leadership, ethos and values; the socio‐economic status (SES) of the schools' catchment; and the organisation and delivery of careers education and guidance at the school level. In the main, high SES schools see themselves as developing pupils for academic university careers, while low SES schools maintain a rather stronger commitment to vocational pathways. The academic ethos of schools offers a very powerful influence on post‐16 choices and decisions of pupils. The usual interventions put in place to influence choices and decisions appear to have greater impact in schools with a less robust academic vision.     

The Sensitivity of Value-Added Teacher Effect Estimates to Different Mathematics Achievement Measures

Using longitudinal data from a cohort of middle school students from a large school district, we estimate separate "value-added" teacher effects for two subscales of a mathematics assessment under a variety of statistical models varying in form and degree of control for student background characteristics. We find that the variation in estimated effects resulting from the different mathematics achievement measures is large relative to variation resulting from choices about model specification, and that the variation within teachers across achievement measures is larger than the variation across teachers. These results suggest that conclusions about individual teachers' performance based on value-added models can be sensitive to the ways in which student achievement is measured.