Golden relics & historical standards: how the OECD is expanding global education governance through PISA for Development

ABSTRACT

Setting this paper against the backdrop of scholarly research on recent changes in the OECD’s approach and workings in education, I analyse how the OECD has reinforced its infrastructural and epistemological global governance through the Programme for International Student Assessment (PISA) for Development (PISA-D). Drawing on qualitative data, this paper makes three arguments. First, there has been a reinforced effort at the OECD to align national and international large-scale assessments; second, the OECD-ensured PISA-D was enhanced only in so far as it remained comparable with PISA, with a view to joining up lower- and middle-income data infrastructures with the global PISA infrastructure; and third, the OECD has bound together the aims of PISA, PISA-D, the Education and Skills Directorate, the Organisation’s Strategy on Development and the global education agenda (Sustainable Development Goals), thus strengthening its global education governance potential. With a note of concern, I suggest these recent changes in the OECD’s work in education may be spreading a very narrow framework of educational values, which does not sufficiently recognise the complexity of learning and teaching.     

Problematizing global educational governance of OECD PISA: Student achievement,categorization, and social inclusion and exclusion

Abstract

This study problematizes the global educational governance of OECD PISA and its statistical data as a governing technology in contemporary discourses of education reforms. The study examines principles that order the discourses and practices of the international comparative assessment. The effort of analysing the impact of an education reform regime led by OECD PISA reveals how statistical reasoning defines problems in educational systems and forms social discourse surrounding educational reform to solve such problems. In doing so, this article focuses on standardization, classification, and normalization for measuring and comparing student achievement and national effectiveness. The study also offers an alternative way of considering the politics of inclusion and exclusion embedded in practices of education reforms propelled by the international comparative assessment.     

The OECD and the local: PISA-based Test for Schools in the USA

The Organization for Economic Cooperation and Development (OECD) has recently piloted the Program for International Student Assessment (PISA)-based Test for Schools in the USA. In this paper, I contend that by connecting directly with local school boards this new initiative has the potential to further promote the OECD's educational agenda in local policy debates. I begin to develop this argument by providing an overview of the OECD and its work. I then lay out a theoretical framework around global governance and knowledge production within the context of the OECD. Next, I provide a brief overview of the traditional PISA study and compare it to the new PISA-based Test for Schools initiative. This context provides the foundation for a discussion of the ways in which a school-based international assessment can operate as a governance tool, allowing international organizations to have greater influence in the formation and implementation of local educational policies.     

Fifteen years of research on PISA effects on education governance: A critical review

This article provides a literature review on the effects of the OECD's Programme for International Student Assessment (PISA) on education governance and policy process across participating countries. This review seemed necessary because there has been a growing body of literature on this topic since 2003, especially since 2010, because this literature is not always well‐known and because the discourse on the so‐called ‘PISA shock’ remains important, even if it is more of a metaphor than a concept and may be politically partial. The article exploits a dataset of 87 references which show that PISA introduced major changes in the governance of education worldwide. Driven by soft power strategies and new policy transfers, this governance is based on data and measurement tools which redefine the scales of education policies. It also shows that PISA has a strong influence on a variety of national reforms, as illustrated in many case studies. However, this influence strongly depends on domestic policy contexts that scholars intended to capture through different theoretical frameworks. Nonetheless, few propose overarching theorisations of the political meaning of PISA effects on education governance and policy processes. The article concludes by stressing three main challenges for the subsequent studies on these PISA effects: better conceptualising these effects, preserving an epistemology of uncertainty in order to avoid taken for granted views and normalising the research on PISA effects not to perpetually and artificially rediscover its so‐called novelty.     

The PISA calendar: Temporal governance and international large-scale assessments

Abstract

This article analyses international large-scale assessments in education from a temporal perspective. The article discusses and compares the different conceptions of time in the early international assessments conducted in the 1960s and 1970s by the IEA with the PISA studies conducted by the OECD from the year 2000 onwards. The paper argues that there has been a shift in the ways that the assessments structure time. The early IEA surveys were characterized by a relative slowness, lack of synchronization and lack of trend analyses. PISA, by contrast, is characterized by high pace, simultaneous publication of results around the world and regular and recurrent studies making the analysis of trends possible. The emergence of this new time regime, it is argued, has implications for how education is governed. At the transnational level, it strengthens the influence and importance of OECD as a significant policy actor. At the national level, as educational discourse and policy adapts to the temporalities of the PISA calendar, two kinds of effects can be distinguished. First, there is a tendency towards searching for “retrotopian” solutions for contemporary problems. Second, there is a tendency towards acceleration and short-term planning when it comes to educational reforms.     

面向未来的参与能力——PISA“素养”概念的发展

素养是PISA测试的核心概念。本文解读了PISA 2000以来素养概念的四个方面的发展:⑴逐步纳入非认知因素和元认知因素;⑵跨学科的"问题解决能力";⑶突破传统纸笔测试素养的"计算机化测评素养";⑷顺应经济社会的趋势,纳入新的测评领域"财经素养"。指出素养是一个不断发展的概念,其发展的两条路径分别是基础领域的深入挖掘和新兴领域的不断加入,发展动力是社会对个体适应能力不断提出更高、更多样的要求。     

评估关键能力 培育生活智慧——OECD／PISA问题解决能力之构想、设计与评估

培养学生的问题解决能力是学校教育的重点内容。“国际学生评价项目”在2003年增加了对学生问题解决能力的测评。该评估项目旨在考察学生综合运用学科领域的知识,识别问题关键特征及其内在关系,能够明确界定问题,合理表征问题和有效解决问题,并能够对问题解决方案进行真实性评估、判断与交流。     

The OECD and the expansion of PISA: new global modes of governance in education

This paper examines the expansion of the OECD's Programme for International Student Assessment (PISA) and associated growth in the influence of the OECD's education work. PISA has become one of the OECD's most successful ‘products’ and has both strengthened the role of the Directorate for Education within the organization and enhanced the significance of the organization in education globally. We provide an overview of the OECD, including organizational changes in response to globalization and the changing place of the Directorate for Education within the organization, particularly with the development of PISA in the late 1990s. We show how the OECD is expanding PISA by broadening the scope of what is measured; increasing the scale of the assessment to cover more countries, systems and schools; and enhancing its explanatory power to provide policy‐makers with better information. The OECD has also developed the Programme for International Assessment of Adult Competencies (PIAAC) and PISA‐based Tests for Schools, which draw on the PISA template to extend the influence of its education work to new sites. The paper draws on data from 33 interviews with past and present personnel from the OECD, the International Association for the Evaluation of Educational Achievement (IEA) and the English and Australian education systems, as well as analysis of relevant OECD documents. We argue that PISA, and the OECD's education work more broadly, has facilitated new epistemological and infrastructural modes of global governance for the OECD in education.     

基于计算机的PISA数学素养测评的特点与发展

基于计算机的测评逐渐成为PISA数学素养测评的主要方式,并在测评框架、测评题目设计、作答环境和模式、评分过程及测评结果等方面都呈现出较为明显的计算机化特点。PISA2021数学测评将与计算机技术进行更深层次的融合,通过更具交互性、智能化和适应性的方式达到更好的数学素养测评效果。     

Competitive comparison and PISA bragging rights: sub-national uses of the OECD's PISA in Canada and the USA

Participation in the Organization for Economic Co-operation and Development's (OECD) Program for International Student Assessment (PISA) has continuously expanded: from 43 systems in 2000 to 65 systems in the 2012 cycle, with 71 signed up for PISA 2015. There also has been a growth in sub-national participation, expanding PISA's reach beyond the nation-state. This paper explores sub-national PISA participation in Canada and the USA, asking how PISA is being used within sub-national education policy spaces. We draw on analysis of documents and data from interviews with officials at sub-national, national, and international levels. Findings illustrate some of the diverse motivations and uses of PISA, providing insights into the effects of PISA at the sub-national scale. As such, we argue that competitive comparison in education has deepened through the enhanced granularity of international large-scale assessment data to new scales beyond the nation-state.     

PISA2006科学试题的设计和开发及其启示

学业成就评价是当前新课程改革研究的热点之一。如何科学地设计和开发试题,对深化新课改、进行基础教育质量监控有着重要意义。PISA是一项权威的国际学生评价项目,具有较高的可比性、可信性和有效性。PISA2006科学评估框架包含情境、知识、态度和能力等相互联系的四个方面,其试题设计和开发技术采用了"双位编码"评分设计,增加了态度评估试题,保证了试题与标准的匹配。     

评估科学素养 培育关键能力——OECD／PISA科学素养之构想、设计与评估

科学素养被认为是21世纪公民生活和工作所必须具备的关键能力的核心,而培养学生的科学素养就成了教育的重要使命之一,如何测评学生的科学素养也因此成了研究的重点。本文试图通过介绍欧洲经济合作与发展组织的“国际学生评价项目”对科学素养的界定和评估框架,为评价学生的综合能力提供借鉴。     

The existence and impact of floor effects for low-performing PISA participants

ABSTRACT

Modern international studies of educational achievement have grown in terms of participating educational systems. Accompanying this development is an increase in heterogeneity, as more and different kinds of educational systems take part. This growth has been particularly pronounced among low-performing, less economically developed systems. Although studies such as PISA have made modifications to account for increased diversity, the degree to which international assessments serve educational systems at the lower ends of the achievement continuum is understudied. We used modified Wright maps and PISA’s definition of proficiency to evaluate the fitness of PISA, especially among low performers. Our findings suggest that there is mismatch between some populations and PISA. Results from a simulation show that such disparities produced biased achievement estimates and correlations with policy relevant variables. Projected PISA growth and new instantiations of PISA, particularly geared toward developing educational systems, make these findings timely and especially relevant.     

PISA在中国:教育评价新探索

本文通过对当前国际上颇具影响力的大规模教育评价项目——"学生能力国际评价("PISA)的评价理念和技术的解析,结合教育部考试中心开展PISA2006中国试测研究实践收获的启示,阐述了作者对我国教育评价研究与发展的思考和建议。     

Placing PISA and PISA for schools in two federalisms,Australia and the USA

ABSTRACT

This paper accepts that the OECD’s PISA has become influential in policy terms globally, but analyses the ways that the main PISA and PISA for Schools tests are positioned differently in Australia and the USA because of contrasting educational federalisms in the two nations. Our argument is that while PISA is undoubtedly influential, its effects are nonetheless mediated by the political structures – here, the different models of federalism – present within different nations, which in turn leads to quite distinct ‘PISA effects’. For instance, Australia oversamples on main PISA to make its data available for national and state-level policymaking, whereas the USA, with its focus on local governance in schooling, does not oversample, meaning that main PISA does not have a comparable policy salience as in Australia. Conversely, the newer PISA for Schools test originated in the USA with pressure from educators and philanthropic interests and has been implemented in a good number of schools, but it has not been taken up in the same way in Australia. Our analyses show how these differences reflect the idiosyncratic workings of federalism in the two nations, in which the federal government has a stronger policy and funding role in Australia than has hitherto been the case for the federal government in the USA.     

Seeing education with one's own eyes and through PISA lenses: considerations of the reception of PISA in European countries

The paper addresses the Organization for Economic Cooperation and Development's Program for International Student Assessment (PISA) as a public policy instrument, whose worldwide circulation is mediated by processes of reinterpretation, negotiation, and re-contextualization, where national, local, and international agencies intertwine. It is focused on the active reception of PISA in six European spaces (Francophone Belgium, France, Hungary, Portugal, Romania, and Scotland) along its first three cycles. The paper identifies two contrasting developments: the Program's divergent uses and its attractiveness in different social worlds. The paper gives particular attention to what is called the ‘update of reference societies’ in the context of national receptions of PISA. These ‘updates’ are analyzed as part of a composite process that involves domestic reasons, either related to current agendas for education or to deep historical factors, and injunctions related to PISA's rationale and PISA objects.     

全球治理理论与中国乡村治理结构的转换

通过考察加拿大新布尔维克省充满实验性质的＂新乡村地区治理模式＂,归纳出乡村治理所要强调的落实地方分权、增加地方居民参与基层政策决策机制以及凝聚力、动员乡村社区资本等重要方案。而在目前中国的乡村社会变迁过程中,正经历着一场乡村治理的结构性转换问题。在转换导向和转换方式等问题上,其核心是基层政府如何理解和消解在推进乡村治理过程中可能遭遇到的问题和挑战。     

The uses of the national PISA results by Finnish officials in central government

In this article we attempt to analyse how OECD knowledge production is integrated with the process in which Finnish education policy takes shape. This is done by analysing the uses of the OECD PISA Study by Finnish central government officials. The main question posed is: How do these officials interpret the PISA results so as to justify the decisions made in Finnish education policy in the past or to point out new areas of development concerning basic education? The analysis shows that the interpretations of the PISA results tend to favour those responsible for actions within the central government. In the texts analysed, the scientificity of the PISA programme is presented as beyond question, while the direct usefulness of the research results for the further development of national education is also proclaimed. As to the specific results of PISA, the excellent learning outcomes of Finnish students are claimed to be due to educational reforms conducted and decisions made by the central government, whereas shortcomings and areas in which the officials see a need for improvement are argued to be dependent on the actions of other agents. Thus, the analysis shows that the conclusions drawn from the PISA results in texts representing the views of central government are biased and serve to justify its policy agenda.     

The national politics of educational advocacy in the context of global governance: international funding and support for civil society engagement in Cambodia

This paper examines the Civil Society Education Fund’s (CSEF) impact on the non-governmental organisation education partnership (NEP) in Cambodia. With financial backing from the World Bank and the Fast Track Initiative, the CSEF is an initiative that is managed internationally by the Global Campaign for Education. Its goal is to help national networks of non-governmental organizations participate in education decision-making and to serve as a watchdog for progress related to internationally agreed upon goals. Through the CSEF, the deployment of various strategies, and other external factors, the NEP was able to able to achieve recognition, legitimacy and influence at the national level. However, the NEP has had to balance working with the state and working for the state. This case study highlights strategies used by civil society actors to engage state actors, the efficacy of international support, and the conflicts inherent in both.