The impact of test language on PISA scores. New evidence from Wales

In this paper we investigate the link between the language in which pupils take the Programme for International Student Assessment (PISA) test and the scores they achieve in this assessment, focusing upon the case of Wales. Using five rounds of PISA data and an instrumental variable approach, we show how pupils who took the test in Welsh score around 0.3 standard deviations (30 PISA test points) lower in reading, mathematics and science than their peers who took the test in English. This finding is robust to different model specifications and statistical approaches. We argue that this may indicate that the academic performance of teenagers in Wales may be underestimated in PISA – particularly amongst those who take this test in Welsh.     

Assessing the effect of school days and absences on test score performance

While instructional time is viewed as crucial to learning, little is known about the effectiveness of reducing absences relative to increasing the number of school days. Using administrative data from North Carolina public schools, this paper jointly estimates the effect of absences and length of the school calendar on test score performance. We exploit a state policy that provides variation in the number of school days prior to standardized testing and find substantial differences between these two effects. Extending the school calendar by ten days increases math and reading test scores by only 1.7% and 0.8% of a standard deviation, respectively. A similar reduction in absences would lead to gains of 5.5% in math and 2.9% in reading. We perform a number of robustness checks including utilizing flu data to instrument for absences, family-year fixed effects, distinguishing between excused and unexcused absences, and controlling for a contemporaneous measure of student disengagement. Our results are robust to these alternative specifications. In addition, our findings indicate considerable heterogeneity across student ability, suggesting that targeting absenteeism among low performing students could aid in narrowing current gaps in performance.     

The effect of childhood bereavement on secondary school performance

This paper reports on a UK survey of GCSE examination results for children from schools throughout England who had lost a parent (n = 73) or sibling (n = 24) through death to establish whether this had a long‐term impact on school performance, levels of anxiety and self‐esteem, and school attendance. Participants were matched for school, age, gender and ethnicity, and a smaller group for socio‐economic status. The Speilberger Anxiety and Coopersmith self‐esteem inventories were administered alongside a customised family history questionnaire.

Bereaved participants underachieved significantly depending upon age, gender, and parents' employment history. There was also a significant rise in anxiety among the parentally bereaved group. School attendance seemed unaffected. Childhood bereavement appears to be a robust factor in underachievement. The totality of the survey findings is discussed prospectively in this paper.     

Public and private school efficiency and equity in Latin America: New evidence based on PISA for development

Using the recent PISA for Development (2017) learning survey, we offer new evidence on whether there is a private–public schools efficiency gap in Latin America and the role of distinct barriers and inequality on efficiency. We obtain school efficiency scores using Data Envelopment Analysis from 705 schools in four countries – Ecuador, Guatemala, Honduras and Paraguay. We find that the private schools efficiency is 0.88 whereas it is lower for public schools (at 0.82). Thus, there is a positive efficiency gap for private schools, with the lower efficiency in public schools may be explained by the additional obstacles they face (such as higher prevalence of student work). However, there is a greater scope in public schools of boosting efficiency by decreasing inequality and the provision of remedial classes. Whole sample results seem to be driven by two countries: Ecuador and Paraguay.     

The effect of school closures on standardised student test outcomes

The school closures owing to the 2020 COVID-19 crisis resulted in a significant disruption of education provision, leading to fears of learning losses and of an increase in educational inequality. This article evaluates the effects of school closures based on standardised tests in the last year of primary school in the Dutch-speaking Flemish region of Belgium. Using a 6-year panel, we find that students of the 2020 cohort experienced significant learning losses in three out of five tested subjects, with a decrease in school averages of mathematics scores of 0.17 standard deviations and Dutch scores (reading, writing, language) of 0.19 standard deviations as compared to previous cohorts. This finding holds when accounting for school characteristics, standardised tests in Grade 4 and school fixed effects. Given the large observed effect sizes, the effect of school closures appears to be a combination of lost learning progress and learning loss. Moreover, we observe that inequality both within schools and across schools rises by 7% for mathematics and 8% for Dutch. The learning losses are correlated with observed school characteristics, as schools with a more disadvantaged student population experience larger learning losses.     

Peer effects and measurement error: The impact of sampling variation in school survey data (evidence from PISA)

Investigation of peer effects on achievement with sample survey data on schools may mean that only a random sample of the population of peers is observed for each individual. This generates measurement error in peer variables similar in form to the textbook case of errors-in-variables, resulting in the estimated peer group effects in an OLS regression model being biased towards zero. We investigate the problem using survey data for England from the Programme for International Student Assessment (PISA) linked to administrative microdata recording information for each PISA sample member's entire year cohort. We calculate a peer group measure based on these complete data and compare its use with a variable based on peers in just the PISA sample. We also use a Monte Carlo experiment to show how the extent of the attenuation bias rises as peer sample size falls. On average, the estimated peer effect is biased downwards by about one third when drawing a sample of peers of the size implied by the PISA survey design.     

PISA truth effects: the construction of low performance

This article seeks to unpack the taken-for-granted notion of low performance, arguing that performance and competency are not a given categories; rather they are “objects-for-thought” that receive their discursive and material contours through a chain of translations. As suggested previously by Gorur, PISA is analyzed through the lens of Latourian Science and Technology Studies. The arguments in this article are based on an analysis of situations constructed to observe how performance is enacted in socio-material practice, as 15-year-old students collaboratively solve PISA scientific-literacy items. As background a text analysis, concerning how scientific literacy and performance are discursively constructed in various PISA materials, is reported. We suggest the notion of ‘competency’ be linked to the historical event of trying to start to detect it and argue that PISA results are products of the situated adjustments that are enacted by students and items created in the very moments of scientific measurement.     

The effect of test expectations on study strategies and test performance: a metacognitive perspective

The aim of the present study was to investigate whether university students can adjust their study strategies to meet the cognitive demands of testing; a metacognitive self‐regulatory skill. One hundred and fifty undergraduates attended three lectures as part of a course on the psychology of individual differences. These participants were then assigned randomly to each of two groups. In the first group, the participants were instructed to study for a test that required deep‐level cognitive processing. In the second group, the participants were instructed to study for a test that required surface‐level cognitive processing. Results of the study showed that university students do adjust their study strategies so that they are in line with the cognitive processing demands of testing. It was also found that study strategies mediated the relationship between the type of test items expected and the test performance.     

The importance of school quality as a determinant of earnings in a developing country: evidence from Honduras

Educational expansion is a prominent feature of the economic development strategies of many developing countries. While the potential benefits of greater schooling attainment for subsequent labor market earnings are acknowledged, the potential role played by school quality has largely been neglected for developing countries. This paper combines detailed household survey data with unique data on school quality for Honduras to investigate the importance of school quality for earnings. We employ a structural equations model that treats school quality as a latent variable. Our objective measures of school quality capture teacher training, school infrastructure, and two measures of school crowding. We find strong positive effects of school quality on earnings across a variety of model specifications. In addition, we find that teacher training and school infrastructure are the most important indicators of school quality.     

Discrepancies between school grades and test scores at individual and school level: effects of gender and family background

The purpose of the study was to use multivariate multilevel techniques to investigate whether it was possible to separate different dimensions in grades that relate to subject-matter achievement and to other factors. Data were derived from The Gothenburg Educational Longitudinal Database (GOLD), and the subjects were 99,070 ninth-grade students born in 1987. The analyses were based on subject grades and scores on national tests in Swedish, English, and mathematics. The results showed that, at both individual and school levels, the greatest part of the variance in grades was due to achievement in the different subject areas. At both levels, it was possible to identify a dimension that cut across the grades in all 3 subjects, which suggests that grading is influenced by factors other than achievement. One of the most interesting results concerns the relation between parental education and the common grade dimension at the school level.     

Identifying patterns of students' performance on simulated inquiry tasks using PISA 2015 log-file data

Previous research has demonstrated the potential of examining log-file data from computer-based assessments to understand student interactions with complex inquiry tasks. Rather than solely providing information about what has been achieved or the accuracy of student responses (product data), students' log files offer additional insights into how the responses were produced (process data). In this study, we examined students' log files to detect patterns of students' interactions with computer-based assessment and to determine whether unique characteristics of these interactions emerge as distinct profiles of inquiry performance. Knowledge about the characteristics of these profiles can shed light on why some students are more successful at solving simulated inquiry tasks than others and how to support student understanding of scientific inquiry through computer-based environments. We analyzed the Norwegian PISA 2015 log-file data, science performance as well as background questionnaire (N = 1,222 students) by focusing on two inquiry tasks, which required scientific reasoning skills: coordinating the effects of multiple variables and coordinating theory and evidence. Using a mixture modeling approach, we identified three distinct profiles of students' inquiry performance: strategic, emergent, and disengaged. These profiles revealed different characteristics of students' exploration behavior, inquiry strategy, time-on-task, and item accuracy. Further analyses showed that students' assignment to these profiles varied according to their demographic characteristics (gender, socio-economic status, and language at home), attitudes (enjoyment in science, self-efficacy, and test anxiety), and science achievement. Although students' profiles on the two inquiry tasks were significantly related, we also found some variations in the proportion of students' transitions between profiles. Our study contributes to understanding how students interact with complex simulated inquiry tasks and showcases how log-file data from PISA 2015 can aid this understanding.     

The effect of extracurricular activities on academic performance in secondary school: The case of Azerbaijan

Extracurricular activities play an important role in the school lives of students. To understand this effect, the authors of this article consider public schools in Azerbaijan and analyse the effect of extracurricular activities on students’ academic performance. They conducted a study investigating three groups of extracurricular activities: sports, fine arts and student clubs. For data on academic performance, they used results of a centralised state school examination which assesses students in two subjects: language of instruction (either Azerbaijani or Russian) and mathematics. The results of this study show that some extracurricular activities have a positive effect on students’ academic performance, while others show no effect. Moreover, the authors consider the number of different extracurricular activities a student takes part in and analyse the significance of this number in predicting student success.

     

Decomposing inequalities in performance scores: the role of student background, peer effects and school characteristics

This paper analyses the mechanisms of stratification and inequalities in educational achievements. The main objective is to determine how stratification leads to unequal educational outcomes and how inequalities are channelled through student characteristics, school characteristics and peer effects. This analysis is undertaken in five countries differentiated by their schooling systems. The countries are Japan, Finland, Germany, Italy and the UK, and the dataset used is PISA 2003. The analysis consists of a multilevel econometric model used to explain variations in performance scores. The explanatory variables are student, school and peer characteristics. The institutional context of each education system is used to interpret the results and to describe how inequalities arise. In the last section, policy implications, based on the regression results, are derived.     

关于75～85分成绩段情况调查的报告——兼谈影响普通话水平测试成绩的主客观因素

本分析了问卷调查和试卷调查的材料。用教学方法计算出广西普通话水平测试的各题分数的分布区间。再进一步考察出各题的稳定性：第四题“说话”最稳定，第三题“朗读”次之，第一题“读单字”次之，最不稳定的是第二题“读词语”。从稳定性的角度提出影响普通话水平测试成绩的主客观因素。     

The effect of familiarity with the examiner on WISC-r verbal,performance, and full scale scores

The Wechsler Intelligence Scale for Children-Revised (WISC-R) was administered to 72 elementary school children (36 first graders and 36 fifth graders). Three methods of pretest rapport establishment were compared for each of two age groups. Treatment Group 1 received less than 5 minutes of pretest rapport establishment immediately preceding testing. Treatment Group 2 received 15 minutes of rapport establishment immediately preceding testing. Treatment Group 3 received 5 minutes of rapport establishment on two of the five school days prior to testing and 5 minutes immediately preceding testing. It was found that time spent in establishment of familiarity with the examiner as determined by the three treatment conditions had no significant effect on WISC-R Verbal, Performance, or Full Scale scores for this sample. Also, no significant difference was found between 6-year-old and 10-year-old students under the various treatment conditions. Finally, no significant differences on WISC-R Verbal, Performance, or Full Scale scores were found as a result of the interaction of age and familiarity conditions.     

Immigration and its effect on the college-going outcomes of natives

In this paper, I analyze immigration's effect on the SAT-scores and college application patterns of high school students in California and Texas. The student-level dataset used is longitudinal in nature and is matched via a unique algorithm to the Census 2000 summary tabulation files to determine immigration at the local census-place level. The Census measure of immigration developed here is extremely specific and improves upon existing immigration measures. Using empirical strategies to account for issues of selection, this essay's main finding is that the 1990s immigration did not harm, and possibly benefited the student outcomes of U.S. citizens.     

The legitimation of OECD's global educational governance: examining PISA and AHELO test production

Although international student assessments and the role of international organisations (IOs) in governing education via an evidence-based educational policy discourse are of growing interest to educational researchers, few have explored the complex ways in which an IO, such as the OECD, gains considerable influence in governing education during the early stages of test production. Drawing on a comparative analysis of the production of two international tests – the Programme for International Student Assessment (PISA) and the Assessment of Higher Education Learning Outcomes (AHELO) – we show how the OECD legitimises its power, and expertise, and defines ‘what counts’ in education. The OECD deploys three mechanisms of educational governance: (1) building on past OECD successes; (2) assembling knowledge capacity; and (3) deploying bureaucratic resources. We argue that the early stages of test production by IOs are significant sites in which the global governance of education is legitimated and enacted.