What can evidence-use in practice learn from evidence-use in policy?

Abstract

Background: This paper approaches evidence-informed practice from the perspective of evidence-informed policy-making. Using the findings of a recent study of evidence-use by educational policy-makers to raise questions about evidence-use by educational practitioners, it seeks to explore what such a study might tell us about how to understand and improve evidence-use by educational practitioners.

Purpose: The paper aims, therefore, to identify potential connections, shared insights and common issues between evidence-use in policy and evidence-use in practice. It does this by focusing on two specific areas: the nature of the evidence (i.e. what evidence is used) and the nature of the use (i.e. how evidence is used). The paper outlines what was found about each of these aspects of evidence-use in policy, and then considers what questions and issues these findings might raise for evidence-use in educational practice.

Sample: The empirical study on which this paper is based was an in-depth study of the use of evidence within educational policy development in Australia. It focused on the development of three specific education policies within one Australian state education department and involved interviews with 25 policy-makers who were actively involved in the development of these policies.

Design and methods: The policy-based study involved the following data collection processes: (i) in-depth semi-structured interviews with 25 policy-makers who were involved in the development of the selected policies; (ii) documentary analysis of policy documents, background research reports and other relevant papers relating to the selected policies; (iii) unstructured observation (where possible) of meetings and events connected with the development of the selected policies; and (iv) feedback from 40 wider policy staff who took part in a verification workshop to discuss the project’s emerging findings.

Findings: Drawing on the findings from the original policy study, two areas of potential connection to evidence-use in practice are explored. First, in relation to ‘varieties of evidence and uses’, the negotiation of diverse evidence types and the potential for using evidence in multiple and varied ways appear to be features of evidence-use that are common to educational policy-makers as well as educational practitioners. Secondly, in relation to ‘narrowness of evidence sources’, there is potential for both policy-makers and practitioners to use a narrow (rather than broad) selection of evidence, due to a tendency to work with certain evidence types as a starting point (e.g. performance data) and a tendency to draw on certain evidence sources more frequently (e.g. well-known, familiar research sources).

Conclusions: This paper emphasises: (i) the need for more integrated (or joined-up) understandings of evidence-use across contexts of practice and contexts of policy; (ii) the importance of continued efforts to understand and represent evidence-use more effectively within educational practices; and (iii) the value of paying careful attention to the quality and qualities of evidence-use within and across the different settings of educational practice and policy.     

Racializing educational change: Melting pot and mosaic influences on educational policy and practice

This article racializes educational change by examining literature on the history of educational approaches to diversity in the United States and Ontario, Canada to demonstrate how their respective national myths for engaging with diversity—the melting pot and mosaic—have impacted their educational policies and practices over three definable eras of educational change. The educational policies and practices of the two countries are evaluated in relation to four significant and—within the existing literature—widely used political and educational strategies for responding to racial and ethnocultural diversity in schools. The paper cautions that the current era of curriculum standardization and high stakes assessments that reflects a melting pot approach to education reinstitutes and reinforces an inequitable vertical mosaic structure of schooling experiences and outcomes for diverse student populations. It urges policy makers to consider how the current movement toward post-standardization, which reflects a mosaic approach, is presently influencing educational policy and practice in international contexts and achieving more just and effective learning outcomes for diverse student groups.

Analyzing Educational Policies: A Learning Design Perspective

In this article we describe and illustrate an analytical perspective in which educational policies are viewed as designs for supporting learning. From the learning design perspective, a policy comprises 3 components that we term the what, how, and why of policy: the goals for the learning of members of the group targeted by the policy, the supports for their learning, and an often implicit rationale for why these supports might be effective. We unpack the how of policy by describing 4 types of support for learning: new positions, learning events, new organizational routines, and new tools. Based on our discussion of the rationale for each type of support we conjecture that policies that are effective in supporting consequential professional learning will involve some combination of new positions that provide expert guidance, ongoing intentional learning events in which tools are used to bridge to practice, carefully designed organizational routines carried out with a more knowledgeable other, and the use of new tools whose incorporation into practice is supported. We present an analysis of a policy that was central to an urban district's efforts to support middle school mathematics teachers' development of ambitious instructional practices. The data that we analyzed included audio-recorded interviews conducted with teachers, mathematics coaches, school leaders, and district leaders. The sample analysis illustrates that the learning design perspective is useful both when designing policies and when revising policies after implementation to make them more effective.     

Educational Equity in the Access to Post-Secondary Education: A Comparison of Ethnic Minorities in China with Aboriginals in Canada

This study provides insight into equity issues in post-secondary education by exploring and assessing the history, the reality and the potential developments in higher education for minority students in China, in comparison to post-secondary education for aboriginal students in Canada. It highlights access to post-secondary education by these minorities in both countries in terms of educational policy enactment, orientation, and its enforcement. The study examines both commonalties and differences in the educational policies of both countries to shed light on how each country is able to grapple with the issue of equity in their respective post-secondary educational systems in response to the principles of liberty, equity and dignity as outlined in the Universal Declaration of Human Rights. This study employs a historical approach to identify the common trends by examining issues concerning access to post-secondary education for ethnic minorities in China and the aboriginals in Canada. Specifically, the study employs a comparative method analyzing how access and equity are defined, how policies have evolved, and the impact that the policy has on the minority students in China and aboriginal students in Canada. The assessment of the access issues in these two countries not only reveals some common trends in the evolution of access norms in the post-secondary education of both countries, but also identifies certain differences in response to their historical traditions, national cultures, and diverse educational structures. The findings are presented in three sections: growth and gap, equality versus equity, and equity and the difference among the difference.     

'All the Evidence Shows …': Reasonable expectations of educational research

After years of being criticised in Britain as irrelevant to the task of raising educational standards, educational researchers have been invited to join in the improvement of policy and practice. Welcome though the invitation is, some views of how they should deserve influence reflect unrealistic or narrow definitions of really useful research. This paper argues against exaggerating the prospects for a science of teaching; against tying the national research agenda tightly to what are currently identified as 'central issues'; and for a broader approach to relevance than direct applicability to improving educational practice.     

From Access to Widening Participation: Responses to the changing population in Higher Education in the UK

An emphasis on widening participation, on making higher education accessible to students who have never traditionally considered it as an option, is being increasingly stressed. The article examines tutors' responses to non-traditional students in an institution with a well-established access policy. The responses reveal that the emphasis was on helping students to adjust to the existing undergraduate provision rather than engaging in a radical rethink on possible approaches appropriate to a more diverse population. Recent policy initiatives (ILT and QAA) are considered in order to identify to what extent they might achieve a cultural change at the micro level. In addition, their limitations are considered, in the light of competing demands. The article concludes that the current top-down approach to policy implementation, supported by related research, should be modified; account needs to be taken of how tutors, in interaction with students, interpret policy initiatives. In drawing attention to practice at the micro level, the article should be relevant to both policy makers and tutors in higher education.     

Examining the influence of international large-scale assessments on national education policies

This paper examines whether, to what extent, and how international large-scale assessments (ILSAs) have influenced education policy-making at the national level. Based on an exploratory review of the research and policy literature on ILSAs and two surveys administered to educational policy experts, researchers, policymakers, and educators, our research found that ILSAs, with their multiple and ambiguous uses, increasingly function as solutions in search for the right problem – that is, they appear to be used as tools to legitimize educational reforms. The survey results pointed to a growing perception among stakeholders that ILSAs are having an effect on national educational policies, with 38% of respondents stating that ILSAs were generally misused in national policy contexts. However, while the ILSA literature indicates that these assessments are having some influence, there is little evidence that any positive or negative causal relationship exists between ILSA participation and the implementation of education reforms. Perhaps the most significant change associated with the use of ILSAs in the literature reviewed is the way in which new conditions for educational comparison have been made possible at the national, regional, and global levels.     

Centring and marginalizing: the “soft middle” and Japanese minority education

Examining the gap between national level minority education policies and local implementation in Japan, this paper considers this gap as a “soft middle” whereby local communities and schools have considerable leeway in how they implement policies. In particular, this paper focuses on Japan's largest minority group, the burakumin and their educational experiences. It traces the origins of educational policies targeting burakumin and how these policies were designed with this soft middle in mind. Following this, the paper considers two different localized approaches, both under policies directed at the burakumin and after the termination of such policies. It argues that schools can selectively implement policies while still fitting within the parameters of the law. Finally, it demonstrates that considering the soft middle provides for a richer, more nuanced understanding of educational systems, beyond monolithic assumptions of national systems.     

WHAT IS THE PURPOSE OF LEARNING SCIENCE? AN ANALYSIS OF POLICY AND PRACTICE IN THE PRIMARY SCHOOL

ABSTRACT:  The paper explores the current rationale for primary science in England with a focus on how competing perspectives arising from perceptions of educational ideology and policy discourse have helped to shape current practice. The aim will be to provide a conceptual understanding of this by focusing specifically on how policy has influenced practice. In particular it will consider the way in which discourse and policy text have contributed to the emergent rationale for primary science which in many ways reflects conflicting influences, views and policies. Data were collected over a year from a regional survey and from four case-study primary schools. The findings suggest that teachers in primary schools face tensions between promoting both an educational and a political rationale for learning primary science. The paper will conclude by suggesting that the justification for primary science should be based on what we already know about how children learn science as well as helping them to develop an understanding of science and how it influences and is intrinsically linked to the needs of society.     

Beyond synthesis: augmenting systematic review procedures with practical principles to optimise impact and uptake in educational policy and practice

ABSTRACT

Whilst systematic reviews, meta-analyses and other forms of synthesis are considered amongst the most valuable forms of research evidence, their limited impact on educational policy and practice has been criticised. In this article, we analyse why systematic reviews do not benefit users of evidence more consistently and suggest how review teams can optimise the impact of their work. We introduce the Beyond Synthesis Impact Chain (BSIC), an integrated framework of practical strategies for enhancing the impact of systematic reviews. Using examples from health professions education, we propose that review teams can optimise the impact of their work by employing strategies that (1) focus on practical problems and mindful planning in collaboration with users; (2) ensure reviews are relevant and syntheses reflexively account for users’ needs; and (3) couch reports in terms that resonate with users’ needs and increase access through targeted and strategic dissemination. We argue that combining practical principles with robust and transparent procedures can purposefully account for impact, and foster the uptake of review evidence in educational policy and practice. For systematic review teams, this paper offers strategies for enhancing the practical utility and potential impact of systematic reviews and other forms of synthesis.     

Thin versus thick democracy in education: Porto alegre and the creation of alternatives to neo-liberalism

In this article, the authors describe the ways in which a set of policies has had what seem to be extensive and long lasting effects precisely because these policies are coherently linked to larger dynamics of social transformation and to a coherent strategy that aims to change the mechanisms of the state and the rules of participation in the formation of state policies. They describe and analyze the policies of the 'Popular Administration' in Porto Alegre, Brasil but, more specifically, they focus on the 'Citizen School' and on proposals that are explicitly designed to radically change both the municipal schools and the relationship between communities, the state, and education. This set of polices and the accompanying processes of implementation are constitutive parts of a clear and explicit project aimed at constructing not only a better school for the excluded, but also a larger project of radical and thick democracy. The reforms being built in Porto Alegre are still in formation, but the authors argue that they have crucial implications for how we might think about the politics of education policy and its role in social transformation.     

Further education and skills in Northern Ireland: policy and practice in a post-conflict society

ABSTRACT

This paper contextualises the Further Education (FE) sector in Northern Ireland (NI). It outlines the specific political, social and economic influences that have shaped its position as a major but understated educational provider in what remains a highly divided educational system that is slowly transitioning in a post-conflict environment. Key policy frameworks underpinning sectoral development are described, showing how many policy initiatives have been both ‘borrowed’ from the English context and adapted to local need. The article proceeds to highlight a number of curricular and institutional innovations that have contributed to the development of a small-scale, but distinctive educational, social and economic model. The piece concludes by suggesting that the NI FE experience has the potential to contribute not only to its own specific conditions but, through its ‘policy and practice’ adaptations’, to positively influence FE policy and practice in other parts of the United Kingdom (UK) that require interventions around skills development economic growth and social cohesion.     

Implementation and Incentive of Education Policies: Experience from the High School Admissions Quota Allocation Policy

Abstract

Improving administrative efficiency is the core problem in administrative governance. This case study of quota allocation policy implementation in City A reveals that a set of education policy implementation and incentive mechanisms revolving around responsibility contracts and target evaluations has already taken shape, to guarantee effective policy implementation. However, these highly efficient policy implementation and incentive methods were established on the foundation of quantified policy goals, while nonmeasurable, nonperformance-based targets such as the goal of educational equality suffer the problem of ineffective incentive. Therefore, in pursuing the goal of educational equality, it is necessary to explore more possibilities for implementation and incentive of education policies.     

Is it Possible to Live a Philosophical,Educational Life in Education,Nowadays?

I consider if and how far it is possible to live an educational philosophical life, in the fast‐changing, globalised world of Higher Education. I begin with Socrates’ account of a philosophical life in the Apology. I examine some tensions within different conceptions of what it is to do philosophy. I then go on to focus more closely on what it might be to live a philosophical, educational life in which educational processes and outcomes are influenced by philosophy, using examples taken from published sources and from conversational interviews with philosophers carried out by myself with Kenneth Wain, Bas Levering and Richard Pring. I then outline the directions of current European policy for Higher Education. Finally I discuss how far current policies and trends leave room for doing philosophy of education, concluding that it is possible, but only for individuals who are very much in sympathy with current policy trends or who are creative in constructing smoke screens.     

Against the Dark: Antiblackness in Education Policy and Discourse

I argue that analyses of racial(ised) discourse and policy processes in education must grapple with cultural disregard for and disgust with blackness. This article explains how a theorization of antiblackness allows one to more precisely identify and respond to racism in education discourse and in the formation and implementation of education policy. I contend that deeply embedded within racialized policy discourses is not merely a concern about disproportionality or inequality, but also a concern with the bodies of Black people, the signification of (their) Blackness, and the threat posed by the Black to the educational well-being of other students. Using school (de)segregation as an example, I demonstrate how policy discourse is informed by antiblackness, and consider what an awareness of antiblackness means for educational policy and practice.     

Contesting educational assessment policies in Australia

ABSTRACT

Assessment is a major component of education, significant in directing what is identified as valued student learning. This paper is framed within an understanding of imperative and exhortative policy. Two paradigmatically different, and potentially contesting, assessment policy directions in Australian education – educational accountability to monitor school and teacher performance, and teacher assessment practices to improve learning (assessment for learning [AfL] or formative assessment) – are examined for their impact on teacher professionalism. Both approaches have official endorsement in Australian policy. Mandated participation in national tests is indicative of educational accountability assessments under national direction. While also endorsed nationally, AfL implementation is reliant on state and territory direction. Our examination reveals tensions in the alignment of both policies. This is evident in the impact of accountability assessment on AfL implementation, in particular, teachers’ understandings of valued assessment evidence. We conclude that a paradigmatic shift to support student learning in Australian schools is a policy imperative that includes the need for professional development and learning support for teachers.     

Designing effective educational policies: what can we learn from an Italian case study

We discuss the design and implementation of educational programmes aimed at increasing university enrolment. We focus on a means-test policy introduced in the Italian province of Trento, which consists of a monetary incentive in the form of a grant. To reduce inequalities in access to higher education, aid is awarded according to merit and financial need. Previous research has shown that this programme has no effect on the university enrolment. Therefore, our aim is to understand, through examination of a unique data set, why the policy failed; we also argue that the most effective design of educational policies considers the targeted population's demand for higher education, as well as socioeconomic context. In this way, we provide insight into the formalisation and efficacy of the eligibility conditions. Though our research is concerned with a province, our findings have important implications for how educational programmes should be designed and implemented nationwide and abroad.     

Bringing a student-centered participatory pedagogy to scale in Colombia

This paper presents the experience of going to scale of Escuela Nueva, an educational innovation that started in Colombia and has influenced educational policy in several countries around the world during the last 30 years. The paper is based on the experience of the leaders of the innovation and its expansion over time. As an insider´s perpective, it relies on the knowledge of senior staff of Fundación Escuela Nueva and on the published evidence and experience accumulated during the decades following the creation of the innovation. The paper describes the context where Escuela Nueva was created and the practices of participating students, teachers, schools administrators and parents when undergoing learning and teaching. It presents the conceptual underpinnings of the innovation, and discusses existing evidence of impact of Escuela Nueva on students, teachers and parents. It then examines the experience of bringing Escuela Nueva to scale, using a three-stage framework: learning to be effective, learning to be efficient and learning to expand. The paper then discusses key factors for success, and the challenges of scale, impact and sustainability, and advances the argument that good design, adequate implementation and sustained support are essential to take an innovation to scale. To conclude, the following four key lessons of Escuela Nueva to the educational change field are discussed : (1) Teachers and students need to be the key actors of the change; (2) the proposed improvements have to be easily replicable within existing conditions; (3) the attitudinal change of teachers is positively affected as a result of demonstrating that the model was feasible, a good fit for its beneficiaries and that they could do it; and (4) a systemic approach is necessary to impact simultaneously on the critical factors of success.     

Beyond the poverty of national security: toward a critical human security perspective in educational policy

This article examines the intersecting logics of human capital and national security underpinning the corporate school reform movement in the United States. Taking a 2012 policy report by the Council on Foreign Relations as an entry point, it suggests that these logics are incoherent not only on their own narrow instrumental terms, but also more importantly in terms of progressive ideals of human and educational flourishing. The article proceeds to draw on discussions within the fields of international relations and critical security studies in order to think through what it might mean to reframe educational policy within the terms of human security rather than human capital and national security. It further explores both the possibilities and distinct limitations of extant human security discourses and policies in relation to global neoliberal governance and biopolitics. The article concludes by advocating for a critical human security framework in educational policy along three conceptual lines: (1) A Human beyond Human Capital; (2) Symbiotic Parallelism; and (3) Altersecurity. Ultimately, it suggests that human security for educational flourishing might offer insight into transcending the idea of security altogether.