Evaluating the potential effect of the increased importance of the impact component in the Research Excellence Framework of the UK

The UK's Research Excellence Framework (REF) is a system that is intended to evaluate the quality of the research produced by higher education institutes (HEIs) in the UK in three areas: quality of research outputs; impact of this research beyond academia; and research environment. For the next REF, the funding bodies have reviewed the importance of the three assessment elements and decided to increase the weight of ‘impact’ to 25% (from 20% in REF2014) and decrease the weight of ‘outputs’ to 60% (from 65% in REF2014). This article first examines the relevance of some factors for the quality of impact submissions in REF2014 and finds that larger submissions and institutes with higher external research income received better impact scores in the REF. The article then examines the units of assessment (UoAs) and HEIs that benefitted from the inclusion of the impact agenda as part of REF2014 by examining the distribution of the quality-related research (QR) funding in the 2017–2018 period and finds that the QR funding gap among different UoAs tends to decrease but the gap among HEIs in most of the UoAs increased. With the increased importance of the impact agenda as a criterion for funding bodies, it is expected that research income will be concentrated in fewer universities in the future, with the increased importance of non-academic impact. This article also discusses some of the gaming strategies and long-term investment priorities that HEIs may engage in based on the new submission rules of the next REF.     

Rating and rewarding higher education for sustainable development research within the marketised higher education context: experiences from English universities

This paper explores one key aspect of marketisation in English universities, the quality-related research funding (QR) system, which central government in the United Kingdom uses to allocate funds to Higher Education Institutions (HEIs), based on a system of reviewing and rating the ‘quality’ of research at different HEIs. Specifically, it focuses on the experiences of 30 members of academic staff from eight universities across England who were engaged in research and/or scholarly activity in the broad field of Higher Education for Sustainable Development (HESD), during the time leading up to the 2014 Research Excellence Framework (REF) submission period. Interviewees were asked to talk about their experiences of being a HESD researcher/scholar within the context of the QR system and their perceptions of the relationship between the QR system and interdisciplinary, sustainability, pedagogic and HESD research. Findings from the semi-structured interviews and resulting qualitative analysis, reveal a number of obstacles facing HESD researchers which are outlined and explored in the paper. Strategies and rationale for improving the quality, reputation and ultimately, the ‘REF-ability’ of HESD research are discussed, and highlight the complex interface between marketisation and sustainability in higher education.     

Performance-based university funding and the drive towards ‘institutional meritocracy’ in Italy

Many countries, including Italy, are increasingly managing their public higher education systems in accordance with the New Public Management principle that private-sector management practices improve efficiency and quality. A key mechanism has been the introduction of performance-based funding systems designed to reward ‘high-performing’ institutions and incentivise ‘lesser-performing’ institutions to improve. Instead of improving efficiency and quality across the board, however, we argue that performance-based funding systems naturalise longstanding structurally determined inequalities between institutions by recasting national higher education systems as competitive institutional meritocracies in which institutional inequalities are redefined as objective indicators of intrinsic ‘merit’ or worth. We illustrate how performance-based university funding systems naturalise pre-existing inequalities between universities drawing on the case of Italy, a country characterised by longstanding inequalities between its northern and southern regions which demonstrably impact on the apparent ‘performance’ of universities. The concept of institutional meritocracy captures the illusory nature of this performance game.     

Selling ‘impact’: peer reviewer projections of what is needed and what counts in REF impact case studies. A retrospective analysis

Abstract

The intensification of an audit culture in higher education is made no more apparent than with the growing prevalence of performance-based research funding systems like the UK’s Research Excellence Framework (REF) and the introduction of new measures of assessment like ‘impact’ or more specifically, the economic and societal impacts of research. Detractors of this regulatory intervention, however, question the legitimacy and credibility of such a system for, and focus within, the evaluation of research performance. Within this study, we specifically sought to understand the process of evaluating the impact of research by gaining unique access as observers of a simulated impact evaluation exercise populated by aproximately n = 90 senior academic peer reviewers and user assessors, undertaken within one UK research-intensive university prior to and in preparation of its submission to REF2014. Over an intensive two-day period, we observed how peer reviewers and user assessors grouped into four overarching disciplinary panels went about deliberating and scoring impact, presented in the form of narrative-based case studies. Among other findings, our observations revealed that in their efforts to evaluate impact, peer reviewers were indirectly promoting a kind of impact mercantilism, where case studies that best sold impact were those rewarded with the highest evaluative scores.     

My tongue on your theory: the bittersweet reminder of every-thing unnameable

ABSTRACT

Across research in UK Higher Education, the most immanent demands for quality have taken the shape of the Research Assessment Exercise and the Research Excellence Framework (REF). The theorist, Martin, is cautious of the relationship academics have engendered with the process of the REF, asking are we actually creating a Frankenstein monster, becoming complicit in generating quality thresholds and standards that will become our own tormentors? I am taken by the idea of the monster when pursuing alternative discourses of childhood in educational research – fear of its potential to torment seduces me with the promise of dis-order, de-formity, chaos and mutation. The aim of this paper is to resist a fixed, knowable form of ‘quality’ (in) research, moving between the idea of ‘monster’ and the formlessness of ‘monstrosity’ to oppose the epistemological, ontological and ethical paradigms of reason.     

The Ethics of Research Excellence

We here analyse the ethical dimensions of the UK's ‘Research Excellence Framework’ (REF), the latest (2014) version of an exercise which assesses the quality of university research in the UK every seven or so years. We find many of the common objections to this exercise unfounded, such as that it is excessively expensive by comparison with alternatives such as various metrics, or that it turns on the subjective judgement of the assessors. However there are grounds for concern about the crude language in which for example all relevant scholarship becomes called ‘research’ and publications become ‘outputs’. The focus on the impact of research, which was a new feature of the most recent exercise, is particularly problematic, creating as it does a tendency to what Aristotle called alazony, self‐aggrandisement, on the part of academics. We conclude that the REF is a mixed good from an ethical point of view, and that more could be done to mitigate its more unfortunate features.     

Editor's preface

While ‘paradigm wars’ have raged internationally during the past decade in particular, the research community, qua community in the Irish context has been largely silent on these important issues. This paper provides a synthesis of key aspects of these international discourses against a brief historical backdrop of the field of educational research. Thereafter, this theoretical lens is used to interrogate the more than 200 papers published in Irish Educational Studies (IES) over a period of 10 years: 1996–2006. This analysis seeks to establish the relative health and quality of educational research in the Irish context, and in the third section of the paper, this analysis is the basis for discussion of lessons that may be learned from insights and understandings gained. The paper concludes that a more systematic and comprehensive review of educational research funding in the Irish context would be particularly apposite and timely, while also advocating the necessity for a comprehensive educational research policy and the creation of a national educational research council. In the absence of such endeavours, research is likely to remain fragmented, small scale and easily dismissed by policy-makers, thus enabling advocacy rather than evidence and research generated elsewhere to overly-influence educational reform, while failing to enhance and extend a comprehensive ‘native’ research literature, a vibrant research culture, while funding and systematically supporting and developing the next generation of educational researchers. In the absence of policy and funding, quality and capacity will continue to falter in a more complex, sophisticated interdependent world.     

美国联邦政府资助大学科研的政策与机制

美国研究开发活动的分权资助体系,以及美国大学将教育与前沿研究活动相结合的特征,是美国充满巨大创新活力的主要原因。而联邦政府竞争性科研拨款机制,包括价值评议与同行评议,则保证了联邦政府资助科研活动的质量,促进了研究型大学的发展。分权和多样化的科研资助来源则使大学的自主权得以保障。     

一般本科院校科研经费管理研究——以泉州师范学院为例

以一般本科院校（泉州师范学院）科研经费管理为研究主题,通过对一般本科院校科研经费管理的现状、存在问题进行阐述分析,结合一般本科院校的独特性,提出完善一般本科院校科研经费管理的对策,以促进科研经费使用效益和科研管理水平的提高。     

Funding,reputation and targets: the discursive logics of high-stakes testing

This paper provides insights into teacher and school-based administrators’ responses to policy demands for improved outcomes on high-stakes, standardised literacy and numeracy tests in Australia. Specifically, the research reveals the effects of the National Assessment Program – Literacy and Numeracy (NAPLAN), and associated policies, in the state of Queensland. Drawing suggestively across Michel Foucault’s notions of disciplinary power and subjectivity, and Pierre Bourdieu’s concept of social fields, the research utilises interviews with teachers and school-based administrators to reveal how high-stakes, standardised testing practices served to discursively constitute performative teacher subjectivities around issues of funding, teacher and school reputation and target-setting within what is described as the ‘field of schooling practices’. The paper argues that the contestation evident within this field is also reflective and constitutive of more educative schooling discourses and practices, even as performative logics dominate.     

Searching for the public: school funding and shifting meanings of ‘the public’ in Australian education

School funding is a principal site of policy reform and contestation in the context of broad global shifts towards private- and market-based funding models. These shifts are transforming not only how schools are funded but also the meanings and practices of public education: that is, shifts in what is ‘public’ about schooling. In this paper, we examine the ways in which different articulations of ‘the public’ are brought to bear in contemporary debates surrounding school funding. Taking the Australian Review of Funding for Schooling (the Gonski Report) as our case, we analyse the policy report and its subsequent media coverage to consider what meanings are made concerning the ‘publicness’ of schooling. Our analysis reveals three broad themes of debate in the report and related media coverage: (1) the primacy of ‘procedural politics’ (i.e. the political imperatives and processes associated with public policy negotiations in the Australian federation); (2) changing relations between what is considered public and private; and (3) a connection of government schooling to concerns surrounding equity and a ‘public in need’. We suggest these three themes contour the debates and understandings that surround the ‘publicness’ of education generally, and school funding more specifically.     

The rise and decline of the International Baccalaureate Diploma Programme in the United Kingdom

The three main programmes of the Geneva-registered International Baccalaureate (IB) have grown substantially worldwide over the past decade, although the programmes have found a natural ‘home’ in the United States. This paper charts the growth of the IB in the United Kingdom (UK) revealing that involvement there, mainly in England and mainly with the original pre-university Diploma Programme (IBDP), peaked at about 230 schools in 2010, but since then the IBDP has begun suddenly to decline. Yet, in no other country has there been a fall in IBDP provision. This paper offers some key explanations for this phenomenon, where a lack of funding and continued lack of university recognition in the face of Advanced Level (A-Level) reform and numerous ‘baccalaureate’ developments has led to many state-funded schools in particular dropping the IBDP. Thirdly, this paper discusses a number of implications, both for the IB itself and education in the UK in general.     

The invisible impact of educational research

Although there are policy calls for educational research to discover ‘what works’ and thereby inform decision making directly, the research literature argues instead for research to have a ‘conceptual’ impact on practice. Empirical studies also suggest that, when teachers use research, their use is conceptual; research influences the content and the process of their thinking, changing attitudes and perceptions and making educational decision making more intelligent. This study investigates the ways in which educational research has achieved impact on practice from the perspective of the researchers. A sample of highly-rated impact case studies in the UK’s research assessment exercise (REF2014) were subject to content analysis, using qualitative coding techniques. Analysis shows that most research is ‘invisible’ to education practitioners because it is embedded in educational policies, technologies, and services. This ‘invisible use’ is unlikely to realise the conceptual benefits claimed for research utilisation. If educational research is to make educational decision making more intelligent at its point of use, it will be necessary to re-think current notions of quality in research impact.     

Managing Quality by Action Research—Improving Quality Service Delivery in Higher Education as a Marketing Strategy

ABSTRACT

This paper demonstrates how the use of self directed and self initiated research by staff in an International Office in one of Australia's largest universities can be a useful tool in improving the quality performance of service delivery to full fee paying international students as a deliberate means of improving the marketable image of the university, accepting that the university marketing strategy recognises that ‘word of mouth’ or informal marketing forms one part of the overall scheme. This paper describes two action research projects undertaken by staff in the International Office to engender improved quality of service delivery and subsequently an improved image of the university in the process of informal marketing. The results of the action research projects have produced specific outcomes, further planning and changes to management practices in the International Office, all related to improving the quality of services being delivered and the quality of management, a strategy that can be adopted in universities worldwide. Each research project was small, but facilitated changes or improvement to a target, strategically identified and discussed with management. These projects did not occur as strategic parts of an overall marketing plan, but were recognised as making contributions to effective outcomes of the existing marketing process. The paper then discusses the role of management in employee empowerment and the action research process arguing that management's role is to ensure that stimuli such as this are not Stifled.     

When soda is a social justice issue: design and documentation of a participatory action research project with youth

Schools are increasingly seen as having a promising role to play in reducing adverse health and wellness outcomes among young people. This paper uses a collaborative action research approach to examine the effects of one school’s efforts to change its students’ eating habits by implementing a ‘junk-food free campus.’ By engaging school administrators and students in a six-month process of joint research design and analysis, the author found that students understood but did not necessarily support the policy. Despite students’ uneven support of the policy, however, there was some evidence that some students were developing healthier eating habits. Moreover, student researchers reported developing greater perspective and respect for the policy as a result of studying it.     

Research assessment as a pedagogical device: Bernstein,professional identity and Education 1 in New Zealand

Recent restructuring of research funding for New Zealand’s higher education institutions is ‘outputs‐driven’. Under the Performance Based Research Fund, units of assessment of research quality are individuals, every degree teacher receiving a confidential score of A, B or C (if deemed ‘research active’) or ‘R’ (‘Research Inactive’). Despite its relatively high number of A and B rated individuals, Education’s collective ranking was low. I interviewed staff and draw on Bernstein to explore how this process affects professional identity formation, a process involving engagement with changing ‘official’ external identities. I overview Bernsteinian concepts, historicise Education’s changing official identities and illustrate how these enabled and constrained participants’ self‐definitions before, during, and immediately after, the quality evaluation. The imposition of audit culture reproduces old theory/practice binaries.     

研究型大学经费配置如何促进学术产出——基于2007—2018年间部属高校面板数据的直接效应和中介效应分析

已有相关研究多集中于探讨经费收入与学术产出的关系，但经费无法直接转化为学术研究成果，对于这一过程的“黑箱”还未有明确认识。本研究尝试分析经费支出对大学发展的影响机制并优化经费配置路径。基于2007—2018年间58所教育部直属高校的校际面板数据，研究采用逐步法和双向固定效应模型分析了经费支出对学术产出的影响程度与作用机制。研究发现：从直接效应来看，经费支出规模对学术产出具有显著正向影响；就中介效应而言，经费支出通过专任教师、硕博士生、固定资产和课题等路径实现学术产出水平的提升，其中正高级教师、硕博士生和课题项目发挥着更大的中介效应。基于此，政府可继续增加对研究型大学的经费投入，并鼓励其多渠道筹措经费；政府还要减少经费使用限制，鼓励大学优化经费配置路径和方式，将更多经费用于教师、硕博士生等人员支出，并设置更多课题项目，从而在遵循高等教育发展规律的基础上，提高经费配置效率，为提升高水平研究型大学建设水平提供实证数据、理论依据和政策参照。     

Difficulties associated with the coding and categorization of students with emotional and behavioural disabilities in Alberta

In Canada, there is a recent trend toward non-categorization of services of students with emotional and behavioural disabilities (EBD). Yet in Alberta, the coding of students with EBD provides opportunities to diagnose students' learning difficulties but is hindered in this process, in large part, by being tied into special needs funding. Current coding and subsequent funding practices favour a medical model approach that provides opportunity to over- and misidentify students for higher funding levels. Though this is likely done with good intentions to get services otherwise denied, it may have led to a perverse situation of the practice of ‘coding for dollars’ for these students. In this article we discuss the Alberta EBD coding and funding model using administrative data as well as conclusion remarks from a seminar participated in by different stakeholders. Recommendations for policy changes will be addressed.     

Analysis of academic administrators’ attitudes: annual evaluations and factors that improve teaching

This article examines academic administrators’ attitudes towards the academic evaluation process in the US and those factors that are utilised to improve teaching. We use path regressions to examine satisfaction with evaluation procedures, as well as the direct and indirect effects of these factors on perceptions of whether the evaluation process facilitates quality instruction. With increased pressure for accountability being placed on higher education, it is important to ensure that we are meeting both public and academic expectations. The evaluation process is an important tool to ensure the university’s goals and values are articulated and that academics can be successful in their individual career paths. The problem is most research finds flaws with the current method of evaluation, and academics and academic administrators are sceptical about the process and results. We find there are environmental factors that influence academic administrators’ perceptions of academic evaluations and the ability to improve classroom instruction.     

我国乡镇农校经费保障制度的缺失及其重构

20世纪80、90年代,在我国乡镇农校经费保障制度的初始设计中,乡镇农校的办学经费虽然有乡镇从农村教育事业费附加中提取一定的比例、集体自筹、收取学费、勤工俭学收入补贴等多种形式,但从农村教育事业费附加中提取一定的比例无疑是乡镇农校办学经费的主要来源。进入21世纪,随着我国农村税费改革的推进、农村教育费附加的取消,我国乡镇农校经费保障制度的原有基础已经丧失,需要对乡镇农校经费保障制度进行优化或进行重新设计。