The legitimation of OECD's global educational governance: examining PISA and AHELO test production

Although international student assessments and the role of international organisations (IOs) in governing education via an evidence-based educational policy discourse are of growing interest to educational researchers, few have explored the complex ways in which an IO, such as the OECD, gains considerable influence in governing education during the early stages of test production. Drawing on a comparative analysis of the production of two international tests – the Programme for International Student Assessment (PISA) and the Assessment of Higher Education Learning Outcomes (AHELO) – we show how the OECD legitimises its power, and expertise, and defines ‘what counts’ in education. The OECD deploys three mechanisms of educational governance: (1) building on past OECD successes; (2) assembling knowledge capacity; and (3) deploying bureaucratic resources. We argue that the early stages of test production by IOs are significant sites in which the global governance of education is legitimated and enacted.     

Problematizing global educational governance of OECD PISA: Student achievement,categorization, and social inclusion and exclusion

Abstract

This study problematizes the global educational governance of OECD PISA and its statistical data as a governing technology in contemporary discourses of education reforms. The study examines principles that order the discourses and practices of the international comparative assessment. The effort of analysing the impact of an education reform regime led by OECD PISA reveals how statistical reasoning defines problems in educational systems and forms social discourse surrounding educational reform to solve such problems. In doing so, this article focuses on standardization, classification, and normalization for measuring and comparing student achievement and national effectiveness. The study also offers an alternative way of considering the politics of inclusion and exclusion embedded in practices of education reforms propelled by the international comparative assessment.     

The OECD and the expansion of PISA: new global modes of governance in education

This paper examines the expansion of the OECD's Programme for International Student Assessment (PISA) and associated growth in the influence of the OECD's education work. PISA has become one of the OECD's most successful ‘products’ and has both strengthened the role of the Directorate for Education within the organization and enhanced the significance of the organization in education globally. We provide an overview of the OECD, including organizational changes in response to globalization and the changing place of the Directorate for Education within the organization, particularly with the development of PISA in the late 1990s. We show how the OECD is expanding PISA by broadening the scope of what is measured; increasing the scale of the assessment to cover more countries, systems and schools; and enhancing its explanatory power to provide policy‐makers with better information. The OECD has also developed the Programme for International Assessment of Adult Competencies (PIAAC) and PISA‐based Tests for Schools, which draw on the PISA template to extend the influence of its education work to new sites. The paper draws on data from 33 interviews with past and present personnel from the OECD, the International Association for the Evaluation of Educational Achievement (IEA) and the English and Australian education systems, as well as analysis of relevant OECD documents. We argue that PISA, and the OECD's education work more broadly, has facilitated new epistemological and infrastructural modes of global governance for the OECD in education.     

Looking East: Shanghai,PISA 2009 and the reconstitution of reference societies in the global education policy field

This paper examines the outstanding performance of Shanghai, China on PISA 2009 and its effects on other national systems and within the global education policy field. The OECD's PISA is helping to create this field by constituting the globe as a commensurate space of school system performance. The effects of Shanghai's success are considered in three other national contexts: the USA, England and Australia. We combine (a) analysis of data from more than 30 research interviews with senior policy actors at the OECD, the IEA and within Australia and England; and (b) document analysis of policy speeches, commissioned research reports and media coverage from the three national contexts. Shanghai's performance in PISA 2009 produced a global ‘PISA-shock’ that has repositioned this system as a significant new ‘reference society’, shifting the global gaze in education from Finland to the ‘East’ at the beginning of the so-called ‘Asian century’.     

The impact of OECD‐PISA results on Japanese educational policy

The author describes the results of PISA, including those of 2015 and Japan's reaction, as well as their impact. Highly‐ranked in PISA, Japan has always tried to improve its education system. The promotion of reading comprehension remains an important issue and low interest and motivation to learn subjects are crucial problems. The author discusses these questions and reform policies from a Japanese point of view. He explains the latest reform plan which will be implemented in 2020. The Japanese peculiarity in education is referred to in the conclusion.     

Slippage and/or symbolism: gender,policy and educational governance in Scotland and Sweden

This article provides an overview and analysis of the relationship between gender, educational policy, and governance in Scotland and Sweden and the two countries’ response to European Union and global legislative and policy change. In Scotland, gender is mainly invisible in recent policies on inclusion, achievement beyond academic attainment, and the idealisation of the child. Gender is thus marginalised within a range of factors contributing to social in/equality. In Sweden, in contrast, gender has higher visibility in policy and governance as both an indicator of democracy and a means of preserving social democratic consensus and prosperity. However, recently its privileged position has come under attack. We draw on social capital, gender, and policy theory to analyse the range of influences on gender and educational governance in the two countries including that of the social capital of organised feminism.     

Using TIMSS and PISA results to inform educational policy: a study of Russia and its neighbours

In this paper, we develop a multi-level comparative approach to analyse Trends in International Mathematics and Science Survey (TIMSS) and Programme of International Student Achievement (PISA) mathematics results for a country, Russia, where the two tests provide contradictory information about students’ relative performance. Russian students do relatively well on the TIMSS mathematics test but relatively poorly on the PISA. We compare the performance of Russian students with different levels of family academic resources over the past decade on these tests compared to students with similar family resources in Russia’s neighbours and to Russian students studying in Latvian and Estonian Russian-medium schools. These comparisons and interviews with educators in Latvia and Estonia help us understand why students in Russia may perform lower on the PISA and to draw education policy lessons for improving international test performance generally and Russian students’ PISA mathematics performance specifically.     

Single-sex schooling,gender and educational performance: Evidence using PISA data

The advantages and disadvantages of single-sex schooling continue to be a source of policy and public debate. Previous empirical evidence is somewhat ambiguous, with some studies finding a positive impact of single-sex schooling on education achievement and others finding no differences across school types. The relationship between single-sex schooling on academic outcomes is typically problematic to examine, as in most countries single-sex schools are selective and the numbers attending them are relatively small. In Ireland, a high proportion of secondary school children (~1/3) attend a single-sex school. In addition, these schools are largely state-funded and non-selective but differing in composition compared to mixed-sex schools. For this reason, the Irish educational system provides an interesting setting for exploring the outcomes of single-sex schooling. In this context, this study utilises the 2018 PISA data for Ireland to examine the relationship between single-sex education and mathematics, reading and science literacy performance for boys and girls, respectively, as well as differences across gender in these outcomes. We find significant raw gaps in reading, science and mathematics scores between females in single-sex and mixed-sex schools and in mathematics scores for males across the same school types. However, after controlling for a rich set of individual, parental and school-level factors we find that, on average, there is no significant difference in performance for girls or boys who attend single-sex schools compared to their mixed-school peers in science, mathematics or reading. In terms of heterogeneous analysis, this finding is consistent across the performance distribution.     

The PISA calendar: Temporal governance and international large-scale assessments

Abstract

This article analyses international large-scale assessments in education from a temporal perspective. The article discusses and compares the different conceptions of time in the early international assessments conducted in the 1960s and 1970s by the IEA with the PISA studies conducted by the OECD from the year 2000 onwards. The paper argues that there has been a shift in the ways that the assessments structure time. The early IEA surveys were characterized by a relative slowness, lack of synchronization and lack of trend analyses. PISA, by contrast, is characterized by high pace, simultaneous publication of results around the world and regular and recurrent studies making the analysis of trends possible. The emergence of this new time regime, it is argued, has implications for how education is governed. At the transnational level, it strengthens the influence and importance of OECD as a significant policy actor. At the national level, as educational discourse and policy adapts to the temporalities of the PISA calendar, two kinds of effects can be distinguished. First, there is a tendency towards searching for “retrotopian” solutions for contemporary problems. Second, there is a tendency towards acceleration and short-term planning when it comes to educational reforms.     

Golden relics & historical standards: how the OECD is expanding global education governance through PISA for Development

ABSTRACT

Setting this paper against the backdrop of scholarly research on recent changes in the OECD’s approach and workings in education, I analyse how the OECD has reinforced its infrastructural and epistemological global governance through the Programme for International Student Assessment (PISA) for Development (PISA-D). Drawing on qualitative data, this paper makes three arguments. First, there has been a reinforced effort at the OECD to align national and international large-scale assessments; second, the OECD-ensured PISA-D was enhanced only in so far as it remained comparable with PISA, with a view to joining up lower- and middle-income data infrastructures with the global PISA infrastructure; and third, the OECD has bound together the aims of PISA, PISA-D, the Education and Skills Directorate, the Organisation’s Strategy on Development and the global education agenda (Sustainable Development Goals), thus strengthening its global education governance potential. With a note of concern, I suggest these recent changes in the OECD’s work in education may be spreading a very narrow framework of educational values, which does not sufficiently recognise the complexity of learning and teaching.     

Policy internationalization,national variety and governance: global models and network power in higher education states

This article analyzes policy convergence and the adoption of globalizing models by higher education states, a process we describe, following Thatcher (2007), as policy internationalization. This refers to processes found in many policy domains and which increasingly are exemplified in tertiary education systems too. The focus is on governmental policymakers, their transnational networks, and the dilemmas they face in responding to the increasing global diffusion of governance models, such as the new public management. The notion of structuration is introduced to convey the inextricability of autonomy and structural constraint for decisionmakers in globally-situated higher education states. Primary aims are to understand the forces that drive policy internationalization, not least those associated with network power, as well as those factors that generate the basis for continuing forms of localism. It is suggested national policy ‘divergences’ may aid the worldwide diffusion of governance models rather than necessarily act as impediments.     

Comparing OECD educational models through the prism of PISA

The PISA survey influences educational policies through an international competitive process which is not wholly rationally-oriented. Firstly, PISA league tables act normatively upon the definition of formal educational aims while the survey tests cannot evaluate the educational systems’ relative strengths with regards to such aims. We argue that they measure a kind of academic potential of the students. Secondly, errors in interpreting national success or failure stem from the causal inferences developed from observing national cases. In order to give such comparisons a stronger basis, we distinguish five main educational models within the OECD countries and compare their PISA 2006 results to their population’s socio-cultural levels as well as their performance recorded in an academic-programme-based survey such as TIMSS. Our conclusions contrast heavily with the usual lessons inferred from PISA, which follow OECD-promoted guidelines, and explain French students’ weakening as well as Finnish students’ success.     

On the supranational spell of PISA in policy

Abstract

Background: PISA results appear to have a large impact upon government policy. The phenomenon is growing, with more countries taking part in PISA testing and politicians pointing to PISA results as reasons for their reforms.

Purpose: The aims of this research were to depict the policy reactions to PISA across a number of jurisdictions, to see whether they exhibited similar patterns and whether the same reforms were evident.

Sources of evidence: We investigated policy and media reactions to the 2009 and 2012 PISA results in six cases: Canada, China (Shanghai), England, France, Norway and Switzerland. Cases were selected to contrast high-performing jurisdictions (Canada, China) with average performers (England, France, Norway and Switzerland). Countries that had already been well reported on in the literature were excluded (Finland, Germany).

Design and methods: Policy documents, media reports and academic articles in English, French, Mandarin and Norwegian relating to each of the cases were critically evaluated.

Results: A policy reaction of ‘scandalisation’ was evident in four of the six cases; a technique used to motivate change. Five of the six cases showed ‘standards-based reforms’ and two had reforms in line with the ‘ideal-governance’ model. However, these are categorisations: the actual reforms had significant differences across countries. There are chronological problems with the notion that PISA results were causal with regard to policy in some instances. Countries with similar PISA results responded with different policies, reflecting their differing cultural and historical education system trajectories.

Conclusions: The connection between PISA results and policy is not always obvious. The supranational spell of PISA in policy is in the way that PISA results are used as a magic wand in political rhetoric, as though they conjure particular policy choices. This serves as a distraction from the ideological basis for reforms. The same PISA results could motivate a range of different policy solutions.