The uses of the national PISA results by Finnish officials in central government

In this article we attempt to analyse how OECD knowledge production is integrated with the process in which Finnish education policy takes shape. This is done by analysing the uses of the OECD PISA Study by Finnish central government officials. The main question posed is: How do these officials interpret the PISA results so as to justify the decisions made in Finnish education policy in the past or to point out new areas of development concerning basic education? The analysis shows that the interpretations of the PISA results tend to favour those responsible for actions within the central government. In the texts analysed, the scientificity of the PISA programme is presented as beyond question, while the direct usefulness of the research results for the further development of national education is also proclaimed. As to the specific results of PISA, the excellent learning outcomes of Finnish students are claimed to be due to educational reforms conducted and decisions made by the central government, whereas shortcomings and areas in which the officials see a need for improvement are argued to be dependent on the actions of other agents. Thus, the analysis shows that the conclusions drawn from the PISA results in texts representing the views of central government are biased and serve to justify its policy agenda.     

On the supranational spell of PISA in policy

Abstract

Background: PISA results appear to have a large impact upon government policy. The phenomenon is growing, with more countries taking part in PISA testing and politicians pointing to PISA results as reasons for their reforms.

Purpose: The aims of this research were to depict the policy reactions to PISA across a number of jurisdictions, to see whether they exhibited similar patterns and whether the same reforms were evident.

Sources of evidence: We investigated policy and media reactions to the 2009 and 2012 PISA results in six cases: Canada, China (Shanghai), England, France, Norway and Switzerland. Cases were selected to contrast high-performing jurisdictions (Canada, China) with average performers (England, France, Norway and Switzerland). Countries that had already been well reported on in the literature were excluded (Finland, Germany).

Design and methods: Policy documents, media reports and academic articles in English, French, Mandarin and Norwegian relating to each of the cases were critically evaluated.

Results: A policy reaction of ‘scandalisation’ was evident in four of the six cases; a technique used to motivate change. Five of the six cases showed ‘standards-based reforms’ and two had reforms in line with the ‘ideal-governance’ model. However, these are categorisations: the actual reforms had significant differences across countries. There are chronological problems with the notion that PISA results were causal with regard to policy in some instances. Countries with similar PISA results responded with different policies, reflecting their differing cultural and historical education system trajectories.

Conclusions: The connection between PISA results and policy is not always obvious. The supranational spell of PISA in policy is in the way that PISA results are used as a magic wand in political rhetoric, as though they conjure particular policy choices. This serves as a distraction from the ideological basis for reforms. The same PISA results could motivate a range of different policy solutions.     

Competitive comparison and PISA bragging rights: sub-national uses of the OECD's PISA in Canada and the USA

Participation in the Organization for Economic Co-operation and Development's (OECD) Program for International Student Assessment (PISA) has continuously expanded: from 43 systems in 2000 to 65 systems in the 2012 cycle, with 71 signed up for PISA 2015. There also has been a growth in sub-national participation, expanding PISA's reach beyond the nation-state. This paper explores sub-national PISA participation in Canada and the USA, asking how PISA is being used within sub-national education policy spaces. We draw on analysis of documents and data from interviews with officials at sub-national, national, and international levels. Findings illustrate some of the diverse motivations and uses of PISA, providing insights into the effects of PISA at the sub-national scale. As such, we argue that competitive comparison in education has deepened through the enhanced granularity of international large-scale assessment data to new scales beyond the nation-state.     

Steering the National: Exploring the Education Policy Uses of PISA in Spain

This article presents findings from a recent study of the education policy uses and impact of international large-scale assessments, namely the Programme for International Student Assessment (PISA). The paper focuses on two overlapping dimensions of PISA’s education policy use in the context of Spain. These include political dimensions, such as the use of PISA to initiate and justify the 2012?2013 educational reforms; and technical dimensions, namely the use of PISA in the development of national-level indicators used to benchmark progress and guide education and curriculum reform. The study points to the growing dominance of PISA as a powerful policy tool. Findings from the paper add to the body of literature on the different ways in which international assessments are used to guide education policy within national spaces and the role of the OECD as an agent of transnational policy steering.     

The politics of international league tables: PISA in Japan’s achievement crisis debate

Using the political‐economic analysis of globalisation and education as well as a culturalist approach to education policy borrowing, the paper analyses the role of local actors, specifically, national newspapers and the Ministry of Education, in mediating the potentially homogenising curricular policy pressure of globalisation exerted through the PISA league tables. Using the recent Japanese education policy debate as a case study, the author demonstrates how the Japanese media interpreted the PISA 2003 findings in a way that resonated with the specific cultural, political, and economic context of the time and how the Ministry used the findings to legitimise otherwise highly contentious policy measures. Questioning the conventional interpretation that the PISA 2003 shock caused the Ministry to redirect its controversial yutori (low pressure) curricular policy, the paper reconstitutes the Ministry as an active agent that capitalised on an external reference (PISA) to re‐establish its political legitimacy in a time of increasing neo‐liberal state‐restructuring.     

Placing PISA and PISA for schools in two federalisms,Australia and the USA

ABSTRACT

This paper accepts that the OECD’s PISA has become influential in policy terms globally, but analyses the ways that the main PISA and PISA for Schools tests are positioned differently in Australia and the USA because of contrasting educational federalisms in the two nations. Our argument is that while PISA is undoubtedly influential, its effects are nonetheless mediated by the political structures – here, the different models of federalism – present within different nations, which in turn leads to quite distinct ‘PISA effects’. For instance, Australia oversamples on main PISA to make its data available for national and state-level policymaking, whereas the USA, with its focus on local governance in schooling, does not oversample, meaning that main PISA does not have a comparable policy salience as in Australia. Conversely, the newer PISA for Schools test originated in the USA with pressure from educators and philanthropic interests and has been implemented in a good number of schools, but it has not been taken up in the same way in Australia. Our analyses show how these differences reflect the idiosyncratic workings of federalism in the two nations, in which the federal government has a stronger policy and funding role in Australia than has hitherto been the case for the federal government in the USA.     

Globalisation of education policies: does PISA have an effect?

The paper examines the role of PISA in the globalisation of education policies. It approaches the question by assessing the effects of PISA on the ways in which new legislation was debated in national contexts in the period 1994–2013. The study asks: Has there been an increase in the number of references to the international community in debates on education policy due to PISA, and, if so, is this change confined to debates on education policy? Our analysis shows that education policy debates feature an increasingly global discourse in which organisations such as the OECD have an authoritative role. Yet, our findings do not support the claim that PISA is the cause of a change in this respect. Debating national policies in a global context and utilising the same transnational discourses regardless of the policy issue area in question has long been with us, yet there is a global trend in which national policies are increasingly often debated through appeals to models and policy advice promulgated by international organisations.     

The contribution of PISA to the convergence of education policies in Europe

The international comparative studies on students’ outcomes have initiated analyses that have had a growing influence on national and sub‐national education policies in industrialised and developing countries. It is particularly the case of the OECD's Programme for International Student Assessment (PISA) which started in 2000 and has organised surveys every 3 years, so that the 2015 survey was the 6^th. Its influence has been particularly important for several reasons: 1) it assesses the basic competences in reading literacy, maths and science of 15 year‐olds students, i.e. around the end of compulsory education in many countries; 2) the assessment is based on a reliable methodology and the tests are completed by qualitative surveys and studies; 3) and the results lead to recommendations and are amplified by the media in most countries. However, it is not easy to evaluate the real impact of PISA because of the existence of other international studies such as IEA's TIMSS and, particularly in Europe, the influence of the recommendations and benchmarks of the EU that has been growing steadily in the last 25 years. Our analysis of the impact of PISA and EU policy focuses on the evolution of the education policy in France, but also studies its evolution in a few other European countries. Finally, we underline the limits of the influence of PISA and international standards in education towards a convergence of education systems because of the importance of their specific historic and cultural contexts.     

The PISA calendar: Temporal governance and international large-scale assessments

Abstract

This article analyses international large-scale assessments in education from a temporal perspective. The article discusses and compares the different conceptions of time in the early international assessments conducted in the 1960s and 1970s by the IEA with the PISA studies conducted by the OECD from the year 2000 onwards. The paper argues that there has been a shift in the ways that the assessments structure time. The early IEA surveys were characterized by a relative slowness, lack of synchronization and lack of trend analyses. PISA, by contrast, is characterized by high pace, simultaneous publication of results around the world and regular and recurrent studies making the analysis of trends possible. The emergence of this new time regime, it is argued, has implications for how education is governed. At the transnational level, it strengthens the influence and importance of OECD as a significant policy actor. At the national level, as educational discourse and policy adapts to the temporalities of the PISA calendar, two kinds of effects can be distinguished. First, there is a tendency towards searching for “retrotopian” solutions for contemporary problems. Second, there is a tendency towards acceleration and short-term planning when it comes to educational reforms.     

The (mis)use of the Finnish teacher education model: ‘policy-based evidence-making’?

Abstract

Background: International achievement studies such as the Programme for International Student Assessment (PISA) have an increasing influence on education policy worldwide. The use of such data can provide a basis for evidence-based policy-making to initiate educational reform. Finland, a high performer in PISA, is often cited as an example of both efficient and equitable education. Finland’s teachers and teacher education have not only garnered much attention for their role in the country’s PISA successes, but have also influenced education policy change in England.

Main argument: This article argues that the Finnish model of teacher education has been borrowed uncritically by UK policy-makers. Finnish and English philosophies of teacher preparation differ greatly, and the borrowing of the Finnish teacher education model does not fit within the teacher training viewpoint of England. The borrowed policies, thus, were decontextualised from the wider values and underpinnings of Finnish education. This piecemeal, ‘pick “n” mix’ approach to education policy reform ignores the fact that educational policies and ‘practices exist in ecological relationships with one another and in whole ecosystems of interrelated practices’. Thus, these borrowed teacher preparation policies will not necessarily lead to the outcomes outlined by policy-makers in the reforms.

Sources of evidence: Two teacher preparation reforms in England, the University Training Schools (outlined in the UK Government’s 2010 Schools White Paper, The Importance of Teaching) and the Master’s in Teaching and Learning (MTL), are used to illustrate the problematic nature of uncritical policy borrowing. This article juxtaposes these policies with the Finnish model of teacher education, a research-based programme where all candidates are required to complete a Master’s degree. The contradictions exposed from this analysis further highlight the divergent practices of teacher preparation in England and Finland, or the disparate ‘ecosystems’. Evidence of educational policy borrowing in other settings is also considered.

Conclusions: Both the MTL and the White Paper reforms overlook the ‘ecosystem’ surrounding Finnish teacher education. The school-based MTL contrasts with the research-based Finnish teachers’ MA. Similarly, the University Training Schools scheme, based on Finnish university-affiliated, teaching practice schools, contrasts heavily with the rest of the White Paper reforms, which contradict the philosophies and ethos behind Finnish teacher education by proposing the move of English teacher preparation away from the universities. The analysis highlights the uncritical eye through which politicians may view international survey results, looking for ‘quick fix’ options instead of utilising academic evidence for investigation on education and education reform.     

Science career aspiration and science capital in China and UK: a comparative study using PISA data

ABSTRACT

The concept of science capital has a growing influence in science education research for understanding young people’s science trajectories. Popularised in the UK, this paper aims to extend and evaluate the applicability of science capital in the context of China by drawing on PISA2015. More specifically, we make use of existing items in the PISA2015 survey as a proxy for operationalising the construct of science capital to explore the science career aspirations and attainments of 15-year-old Chinese and UK students (n?=?23,998). Our findings indicate that science capital has more explanatory power for understanding UK students’ science career aspirations than for Chinese students, where science attainment seems most important. We raise the potential challenge for Chinese students to convert their science capital into scientific self-efficacy and science career aspirations as we highlight the importance of recognising cultural and national differences in operationalising science capital.     

Fifteen years of research on PISA effects on education governance: A critical review

This article provides a literature review on the effects of the OECD's Programme for International Student Assessment (PISA) on education governance and policy process across participating countries. This review seemed necessary because there has been a growing body of literature on this topic since 2003, especially since 2010, because this literature is not always well‐known and because the discourse on the so‐called ‘PISA shock’ remains important, even if it is more of a metaphor than a concept and may be politically partial. The article exploits a dataset of 87 references which show that PISA introduced major changes in the governance of education worldwide. Driven by soft power strategies and new policy transfers, this governance is based on data and measurement tools which redefine the scales of education policies. It also shows that PISA has a strong influence on a variety of national reforms, as illustrated in many case studies. However, this influence strongly depends on domestic policy contexts that scholars intended to capture through different theoretical frameworks. Nonetheless, few propose overarching theorisations of the political meaning of PISA effects on education governance and policy processes. The article concludes by stressing three main challenges for the subsequent studies on these PISA effects: better conceptualising these effects, preserving an epistemology of uncertainty in order to avoid taken for granted views and normalising the research on PISA effects not to perpetually and artificially rediscover its so‐called novelty.     

Artificial intelligence and education in China

ABSTRACT

This paper examines the political economy of artificial intelligence (AI) and education in China, through an analysis of government policy and private sector enterprise. While media and policy discourse often portray China’s AI development in terms of a unified national strategy, and a burgeoning geopolitical contestation for future global dominance, this analysis will suggest a more nuanced internal complexity, involving differing regional networks and international corporate activity. The first section considers two key policy documents published by the central Chinese government, which are shown to implicate educational institutions as influential actors in national and regional strategies for AI development, with a significant role in plans to train domestic expertise. The second section outlines three prominent private education companies: New Oriental Group, Tomorrow Advancing Life (TAL), and Squirrel AI. These companies are selected to represent important aspects of China’s development of educational AI applications, including the influence of a well-established private education sector, and a growing interest in international corporate activity. The paper concludes with the suggestion that while central government policy reserves a significant role for education in the national AI strategy, the private sector is utilising favourable political conditions to rapidly develop educational applications and markets.     

Golden relics & historical standards: how the OECD is expanding global education governance through PISA for Development

ABSTRACT

Setting this paper against the backdrop of scholarly research on recent changes in the OECD’s approach and workings in education, I analyse how the OECD has reinforced its infrastructural and epistemological global governance through the Programme for International Student Assessment (PISA) for Development (PISA-D). Drawing on qualitative data, this paper makes three arguments. First, there has been a reinforced effort at the OECD to align national and international large-scale assessments; second, the OECD-ensured PISA-D was enhanced only in so far as it remained comparable with PISA, with a view to joining up lower- and middle-income data infrastructures with the global PISA infrastructure; and third, the OECD has bound together the aims of PISA, PISA-D, the Education and Skills Directorate, the Organisation’s Strategy on Development and the global education agenda (Sustainable Development Goals), thus strengthening its global education governance potential. With a note of concern, I suggest these recent changes in the OECD’s work in education may be spreading a very narrow framework of educational values, which does not sufficiently recognise the complexity of learning and teaching.     

PISA and its consequences: Shaping education policies through international comparisons

As the field of education has become a highly internationalised policy field in the last decade, international organisations such as the OECD play an ever more decisive role in the dissemination of knowledge, monitoring of outcomes, and research in education policy. Although the OECD lacks any binding governance instruments to put coercion on States or to provide material incentive, it has successively expanded its competences in this field. OECD advanced its status as an expert organisation in the field of education mainly by designing and conducting the international comparative PISA study. With PISA, the OECD was able to greatly influence national education systems. Basically, States were faced with external advice based on sound empirical data that challenged existing domestic policies, politics, and ideas. One prominent case for the impact of PISA is Germany. PISA was a decisive watershed in German education policy‐making. Almost instantly after the PISA results were publicised in late 2001, a comprehensive education reform agenda was put forward in Germany. The experienced reform dynamic was highly surprising because the traditional German education system and politics were characterised by deep‐rooted historical legacies, many involved stakeholders at different levels, and reform‐hampering institutions. Hence, a backlog of grand education reforms have prevailed in Germany since the 1970s. The external pressure exerted by PISA completely changed that situation.     

Problematizing global educational governance of OECD PISA: Student achievement,categorization, and social inclusion and exclusion

Abstract

This study problematizes the global educational governance of OECD PISA and its statistical data as a governing technology in contemporary discourses of education reforms. The study examines principles that order the discourses and practices of the international comparative assessment. The effort of analysing the impact of an education reform regime led by OECD PISA reveals how statistical reasoning defines problems in educational systems and forms social discourse surrounding educational reform to solve such problems. In doing so, this article focuses on standardization, classification, and normalization for measuring and comparing student achievement and national effectiveness. The study also offers an alternative way of considering the politics of inclusion and exclusion embedded in practices of education reforms propelled by the international comparative assessment.     

Looking East: Shanghai,PISA 2009 and the reconstitution of reference societies in the global education policy field

This paper examines the outstanding performance of Shanghai, China on PISA 2009 and its effects on other national systems and within the global education policy field. The OECD's PISA is helping to create this field by constituting the globe as a commensurate space of school system performance. The effects of Shanghai's success are considered in three other national contexts: the USA, England and Australia. We combine (a) analysis of data from more than 30 research interviews with senior policy actors at the OECD, the IEA and within Australia and England; and (b) document analysis of policy speeches, commissioned research reports and media coverage from the three national contexts. Shanghai's performance in PISA 2009 produced a global ‘PISA-shock’ that has repositioned this system as a significant new ‘reference society’, shifting the global gaze in education from Finland to the ‘East’ at the beginning of the so-called ‘Asian century’.     

Seeing education with one's own eyes and through PISA lenses: considerations of the reception of PISA in European countries

The paper addresses the Organization for Economic Cooperation and Development's Program for International Student Assessment (PISA) as a public policy instrument, whose worldwide circulation is mediated by processes of reinterpretation, negotiation, and re-contextualization, where national, local, and international agencies intertwine. It is focused on the active reception of PISA in six European spaces (Francophone Belgium, France, Hungary, Portugal, Romania, and Scotland) along its first three cycles. The paper identifies two contrasting developments: the Program's divergent uses and its attractiveness in different social worlds. The paper gives particular attention to what is called the ‘update of reference societies’ in the context of national receptions of PISA. These ‘updates’ are analyzed as part of a composite process that involves domestic reasons, either related to current agendas for education or to deep historical factors, and injunctions related to PISA's rationale and PISA objects.     

The OECD and the expansion of PISA: new global modes of governance in education

This paper examines the expansion of the OECD's Programme for International Student Assessment (PISA) and associated growth in the influence of the OECD's education work. PISA has become one of the OECD's most successful ‘products’ and has both strengthened the role of the Directorate for Education within the organization and enhanced the significance of the organization in education globally. We provide an overview of the OECD, including organizational changes in response to globalization and the changing place of the Directorate for Education within the organization, particularly with the development of PISA in the late 1990s. We show how the OECD is expanding PISA by broadening the scope of what is measured; increasing the scale of the assessment to cover more countries, systems and schools; and enhancing its explanatory power to provide policy‐makers with better information. The OECD has also developed the Programme for International Assessment of Adult Competencies (PIAAC) and PISA‐based Tests for Schools, which draw on the PISA template to extend the influence of its education work to new sites. The paper draws on data from 33 interviews with past and present personnel from the OECD, the International Association for the Evaluation of Educational Achievement (IEA) and the English and Australian education systems, as well as analysis of relevant OECD documents. We argue that PISA, and the OECD's education work more broadly, has facilitated new epistemological and infrastructural modes of global governance for the OECD in education.     

PISA and policy-borrowing: A philosophical perspective on their interplay in mathematics education

Mathematics achievement in different education systems around the world is assessed periodically in the Programme for International Student Assessment (PISA). PISA is deemed to yield robust international comparisons of mathematical attainment that enable individual countries and regions to monitor the performance of their education systems relative to standards being achieved internationally, with a view to informing their mathematics education policy decisions. Initially, the role of PISA in instigating mathematics education policy borrowing is outlined using England as a case study, and some existing technical critiques of PISA are then reviewed. Following this, aspects of Ludwig Wittgenstein’s later philosophy of mind are used to reason that an over-reliance on the use of PISA to inform policy decisions in mathematics education may be problematic. It is suggested that, when PISA is viewed through a later Wittgensteinian lens, a potential deficiency in the underpinning psychometric model, pertaining to the inherent indeterminism in unmeasured mathematical abilities, may weaken PISA’s utility in guiding mathematics education policy decisions. It is concluded that, whilst PISA mathematics scores may give some indication of the mathematical proficiency of a nation’s students, caution is required before mathematics education policies are borrowed from other jurisdictions on the basis of PISA performance. Implications for the other PISA domains are also outlined.