Major influences on the teaching and assessment of graduate attributes

Efforts to systematically integrate graduate attributes across university curricula have relied on academic staff acceptance and ability to translate top-down policy into teaching practice. The literature and anecdotal reports over the past two decades show limited success in the area using this approach. Why is this so? Teaching staff across 16 Australian universities were surveyed, as part of an Australian Learning and Teaching Council-funded grant, regarding their beliefs about graduate attributes and their willingness and confidence to teach and assess them. The study found that although 73% of academic staff surveyed believed graduate attributes were important, there was a substantial difference between beliefs and actual emphasis reported in practice. We consider the major influences on the teaching and assessment of graduate attributes, the implications for policy development and implementation, and propose a rethink in relation to how to work towards improved graduate attribute outcomes in universities.     

Policy makers’ rhetoric of educational change: A critical analysis

Reforms in education often fail due to insufficient time to implement them, too few charismatic leaders, the influence of the preceding culture, scarce resources or an incapacity to comprehend the complexity of the education system. Most studies dealing with the cognitive aspects of educational change investigate the meaning of change through the lens of the educators and principals and there is not as much research available on the perception and interpretation of change by policy makers and the language they use to communicate messages about policy that affect sensemaking–sensegiving during the process of learning about the changes. Guided by the premise that the language of a reform movement is shaped by its ideology, this article illustrates how press releases of the Ministry of Education in Canton Sarajevo in Bosnia–Herzegovina reflect an exploitation-oriented dominant reform logic. The results of frequency analysis, hierarchical clustering and multidimensional scaling imply that MoE is driven by a top-down perspective to educational change, with a focus on the procedural, organizational and collaborational aspects of reforms that primarily alter the structure of the education system but not its internal substance. We end this paper by suggesting that educational system as complex adaptive system cannot be controlled, but effectively managed through distribution of control and creation of few attractors, benefiting from self-organization in the technical core and making change in the official top-down rhetoric.     

Strategies for Developing Positive Behaviour Management. Teacher Behaviour Outcomes and Attitudes to the Change Process

This paper describes an extended action research project run in a large secondary school over an 18‐month period. The work was part of a wider strategy for change within the school. The data presented here describes some of the features of the change process and reflections on its impact. A key aim was to challenge and enable teachers to modify their behaviour in order to increase the number of specific positive feedback statements they made in their classrooms to their students. The paper reports on three main strands of data that were collected: observation of teacher and pupil behaviours in classrooms, feedback from questionnaires completed by teachers who took part in staff groups formed to discuss behaviour management, and qualitative data from interviews and a focus group with staff at the end of the project. All provide evidence of a change in teachers’ behaviour and information about teachers’ attitudes to the change process.     

‘Measuring Up’? Assessment and students with disabilities in the modern university

In this article, I ask how university students with disabilities negotiate with staff arrangements for alternative assessment practices. I draw on three case studies using a personal pronoun perspective to challenge the conventional view that educational policy and teaching practice are forms of rational action. I demonstrate how the lives of students and staff are typically characterised by unexpected events, disorder, emotion and prejudice. The analytic perspective offered here establishes how meanings, intentions and different viewpoints and alliances emerge as social actors work to create specific faculty and institution cultures. The case studies also reveal what does and what does not work – some of the obstacles – and what needs to be done if we are serious about equity and inclusive education. They include practical assistance in recognising the specific requirements of students with disabilities and how to design alternative assessment for students with specific ‘conditions’. I argue that professional development and specific techniques in curriculum design are needed. Some staff also require help in recognising their policy and legal obligations. A cultural change which identifies and challenges prejudice is a larger task if universities are to become places in which equal opportunity principles and inclusive education are present and actively practised.     

Academic Understandings of and Responses to Bologna: a three-country perspective

This article aims to describe how academics in England, Denmark and Portugal understand the Bologna Process, their attitudes towards the initiative, and how understandings and attitudes determine action. It shows how contextually and culturally determined academic understandings, expectations, preferences, priorities and constraints are key influences in the process of educational change envisaged by the Bologna Process. Second, inspired by the implementation staircase metaphor which illustrates different actors' perceptions conditioned by their location on the path of the policy, the article presents three different examples of interplay between national authorities responsible for policy-making in the wake of the Bologna Process and the institutional field responsible for policy implementation. It reveals variations from country to country with regard to bottom-up and top-down initiatives, both at the level of actors' expectations and of actual measures to implement Bologna policy. This article comes in response to the frailty of an empirical base for the understanding of Bologna from an institutional and academic perspective. A video abstract of this article can be viewed at http://youtu.be/9YVouJ35XuE     

Social justice and technocracy: tracing the narratives of inclusive education in the USA

ABSTRACT

Over the past two decades, the percentage of American students with disabilities educated in general classrooms with their nondisabled peers has risen by approximately 50%. This gradual but steady policy shift has been driven by two distinct narratives of organisational change. The social justice narrative espouses principles of equality and caring across human differences. The narrative of technocracy creates top-down, administrative pressure through hierarchical systems based on quantitative performance data. This article examines these two primary policy narratives of inclusive education in the USA, exploring the conceptual features of each and initiating an analysis of their application in the public schools.     

Developing Procedures for Implementing Peer Assessment in Large Classes Using an Action Research Process

Peer assessment has been used successfully in higher education, with important benefits reported in terms of student learning. However, most of the literature has focused on its use with small groups of students taught by staff who are committed to the peer assessment process. This paper reports the development of peer assessment procedures for use in large classes, using a cyclical process of action, reflection and refined action. The project was carried out in three phases and after each phase changes were made to the procedures in response to student and staff feedback. The development of procedures is discussed in relation to assessment tasks, assessment criteria, anonymity, procedural guidelines, distribution systems, marking procedures and tutor remarking. Although there are specific difficulties associated with the use of peer assessment in large classes, this study suggests that these are outweighed by the learning benefits for students. Based on the findings of this study, recommendations are made for ways in which peer assessment might be successfully applied in large classes.     

Translating norms from Europe to Turkey: Turkey in the Bologna Process

This article focuses on the structural changes Turkish higher education is going through as a result of Turkey’s integration into the European educational space. The focus of the article is the process of policy transfer. For this purpose, the article outlines the changes in Turkish higher education comprehensively to explain the dynamics of the policy transfer. Creation of a national qualifications system and establishment of a quality assurance system are given as examples of policy transfer. The article concludes that while the system is becoming more student-centred, change in higher education is happening in a very top-down fashion, where participation of relevant stakeholders is not ensured.     

Pathways to co-impact: action research and community organising

This article introduces the concept of ‘co-impact’ to characterise the complex and dynamic process of social and economic change generated by participatory action research (PAR). It argues that dominant models of research impact tend to see it as a linear process, based on a donor-recipient model, occurring at the end of a project following the take-up and use of findings. PAR challenges this approach, as impact is embedded in cycles of the action research process; the distinction between researchers, research informants and research users is blurred; and micro process-based impacts, including changes in the thinking and practices of co-researchers, are as significant as findings-based changes in policy and practice. A conceptual framework is developed, based on a three-fold distinction between ‘participatory’, ‘collaborative’ and ‘collective’ impact. This is applied to a case study action research project, Debt on Teesside, working with low-income households in North-east England. The project is analysed in terms of participatory impact (e.g. developing skills of participating households, mentor-researchers, and university staff); collaborative impact (e.g. findings-based changes in thinking, policies and practices of advice, community finance and housing agencies, and local authorities resulting from collaborative research); and ‘collective impact’, adapted from the field of social interventions, which involves organisations collectively targeting specific actions based on research (e.g. changing policy and practices of lenders and government relating to high-cost loans).     

Faculty and Students Conceptions of Assessment in Higher Education

Assessment in higher education serves multiple purposes such as providing information about student learning, student progress, teaching quality, and program and institutional accountability. Yet, little is known about faculty and students’ attitudes regarding different aspects of assessment that have wide-ranging implications for policy and practice in tertiary institutions. To investigate these views, parallel surveys of conceptions of assessment were administered to faculty and undergraduate students across four tertiary institutions including universities, an indigenous tertiary institution, and an institute of technology. A mean and covariance structures approach was used to test for measurement invariance and latent means differences between faculty and students regarding their conceptions of assessment. Results revealed differences in the latent means across the two groups. Faculty were likely to view assessment as a trustworthy process aiding teaching and learning, whereas students viewed assessment as focussed primarily on accountability and perceived assessment as irrelevant or even ignored in the teaching and learning process. These findings highlight the importance of ensuring that assessment policy and practices are fit for purposes, and are being carried out with integrity in ways that are transparent to and understood by both staff and students. While these results show how staff and students view assessment practices, one should keep in mind that while the sample was large and did incorporate different types of tertiary institutions, the inclusion of a broader range of disciplines would make the conclusions more generalizable.     

Pressures and influences on school leaders navigating policy development during the COVID-19 pandemic

The pressures and influences experienced by school leaders as school policymakers, during the first responses to COVID-19 pandemic, made their task of interpreting, translating and implementing school guidance both difficult and stressful. During COVID-19 pandemic, school leaders faced challenges to prioritise and balance ever-changing government policy advice with the limitations of school buildings, the welfare of students and staff, and the needs of communities, a task made complicated by the methods of communication used by government. By surveying and interviewing headteachers, senior leaders and governors, this article identifies the reactions and responses of school leaders who construct and enact policy in the context of COVID-19 pandemic. The article addresses the nature of, and factors affecting, pressures school leaders feel in authoring and implementing policy. These pressures are non-trivial and increase tension in already highly stressful work. The report draws on data collected from a randomised, stratified sample of primary and secondary school leaders from across England in early June 2020, during the time of national lockdown and remote learning. Findings suggest quality, quantity and frequency of top-down communication contributed to school leader stress, while horizontal communication and collaboration between school leaders and across school communities supported leaders during rapid change. We recommend that government and the Department of Education strengthen and streamline stressful communication systems while building cooperative communities, mitigating against the challenges identified by school leaders during the COVID-19 pandemic.     

Assessment thresholds for academic staff: constructive alignment and differentiation of standards

Abstract

This qualitative study utilized episodic narrative interviews to investigate assessment thresholds involved in the development of assessment literacy. The goal of the study was to inform efforts toward quality improvements in higher education. Thirty-five academic staff from universities in Australia, Canada and Sweden shared stories of significant changes they made to their assessment practice. Thematic analysis found troublesome aspects to include instructor expectations, lack of consistency, differentiation of performance, student expectations, time constraints/workload, logistical/technological constraints and assessment policy. A belief in meaningful learning, embracing constraints and the desire to meet the needs of students, added to other enablers for assessment change, such as resources, support and strategic use of technology. Findings suggested assessment thresholds of constructive alignment and differentiation of standards. Reflection, collaboration and professional development were found to support the integration of assessment knowledge and build conceptual understanding of assessment thresholds. Authors recommend that higher education institutions provide academic staff with a foundation of conceptual understanding of these key areas to promote moves toward quality assessment practices.     

Evaluating computer‐based assessment in a risk‐based model

There are three purposes for evaluation: evaluation for action to aid the decision making process, evaluation for understanding to further enhance enlightenment and evaluation for control to ensure compliance to standards. This article argues that the primary function of evaluation in the ‘Catherine Wheel’ computer‐based assessment (CBA) cyclic model is evaluation for action and secondary functions are evaluation for understanding and control. By studying segment and cyclic dependencies, conducting a risk analysis and identifying key stakeholders it is possible to identify where action evaluation has the most effect, what is required to be evaluated, who will perform the action and how it will be controlled. Each segment and cycle in the CBA model must therefore be monitored and the observations made must have meaning for the elimination or reduction of pedagogic, operational, technical non web‐based, web‐based and financial risks. It is important to consider stakeholders because they have differing views on what is most important in the evaluation process and inevitably will be involved in the evaluation process itself. CBA system stakeholders include management, academic, support and administrative staff together with external verifiers and students. The information gained from these stakeholders provides quality enhancement and assurance and forms the basis for change.     

Formative Peer Assessment of Academic Writing Between Postgraduate Students

Reciprocal paired qualitative formative peer assessment of academic writing was undertaken by 12 postgraduate students of educational psychology. Overall, staff and peer assessments showed a very similar balance between positive and negative statements, but this varied according to assessment criterion. However, only half of all detailed formative assessment statements made showed some degree of correspondence between staff and peers. Nevertheless, there was very little evidence of conflict between the views of staff and peers-rather, they focused on different details. Subjective feedback from students indicated that most found the process time consuming, intellectually challenging and socially uncomfortable, but effective in improving the quality of their own subsequent written work and developing other transferable skills. The reliability and validity of this type of peer assessment thus appeared adequate, and the partiality of overlap in detail between staff and peer assessments suggested that the triangulation peer assessment offers is likely to add value. However, caution is indicated regarding the generalisation of this finding. Implications for action are outlined.     

Needs Assessment as a Change Strategy: A Case Study

Needs assessment provides a tool for gathering data which, when applied back upon the situation it addresses, becomes a first step in a change process. A major criticism of needs assessment has been that frequently the process is an exercise, and that the results are seldom actually applied in the planning process. The question arises whether the process, separate from the intended results, can be beneficial to the organization. This article documents a case study in which needs assessment and a change model were applied in conjunction with each other in such a way that the needs assessment process, independent of data collected, served as a change strategy. In this case, while the results of the needs assessment were important to the planning process, the needs assessment process itself, independent of its conclusions, also contributed to the organization's change goals.     

‘New Partnerships for Learning’: teachers and teaching assistants working together in schools – the way forward

In comparison with the school sector, formative and diagnostic assessment have received relatively little attention in higher education. This article reports on an in‐depth study of assessment across one university, exploring views and practices in each of the five faculties, using semi‐structured interviews with eight staff and small group interviews with 60 students. Our data indicate that: there is very little common understanding of the terms often used to describe forms of assessment in policy documents and other literature; students, contrary to popular belief, do value assessment that carries no marks, although a form of ‘deferred instrumentalism’ may be at work here; staff are sometimes engaging in formative and diagnostic assessment without explicitly recognising it; and that students in this case study do value assessment which relates to and will be valuable for life after university. The paper concludes by suggesting frameworks and terminology for future discussion and issues for staff development.     

Travelling towards change in assessment: Policy,practice and research in education

Whilst there is evidence of significant investment in policy-led initiatives to raise attainment in schools, there is rather less evidence of the positive impact of such initiatives. In this paper we explore stakeholder views of recent initiatives in assessment in Scotland in an attempt to discern the relationships between assessment policy, research and practice in schools. Against a background of major assessment initiatives and by drawing on data from two national consultations, the paper illustrates the complexities inherent in following advice for policy developments to begin from where people are now. Finally, the paper explores the possibility of a new assessment journey for researchers, teachers and policy-makers, one which acknowledges the complex process of community-based transformational change.     

Assessment and teacher professionalism

ABSTRACT

This paper draws from a multi‐site qualitative case‐study research project entitled From Policy to Practice funded by the University of Exeter. The research sought to identify the impact that the Key Stage 3 tests would have on the working practices of eight heads of English departments and their staff in mixed maintained comprehensive schools in different regions of England.

The paper begins by presenting the way that the national assessment system has undergone change since its inception, highlighting that the function of assessment is to act as the most significant indicator of a school's performance through government policy that promotes and prioritizes the measurement of pupils’ achievements through each key stage.

The English teachers’ responses to the revisions of the English National Curriculum Programmes of Study are described and the paper documents the thoughts and feelings of these teachers as they had to try to make sense of the changes in their classroom working practices. They are articulate in categorizing those aspects of their pedagogical practice that the assessment process has stifled and those that the assessment process has encouraged. The paper then offers an interpretative analysis of the impact that the externally imposed curriculum and assessment changes have had on the teachers’ sense of being professionals.     

Action groups as a participative strategy for leading whole-school health promotion: Results on implementation from the INCLUSIVE trial in English secondary schools

Education policy increasingly promotes action groups as a key strategy for student and/or staff participation in school improvement and whole-school health promotion. Such groups can coordinate multi-component interventions, increase participation and engagement, and enable local adaptations, but few process evaluations have assessed this. We evaluated fidelity, feasibility and acceptability of action groups as part of a trial of a whole-school intervention to reduce bullying and aggression and promote health in English secondary schools, which reported multiple health and educational impacts. Action groups involved students and staff, supported by external facilitators, and drew on data on student needs. They aimed to: coordinate implementation of restorative practices and a social and emotional competencies curriculum; review policies and rules; and enact local decisions to modify school environments. Our process evaluation used interviews, focus groups, observations and questionnaires to assess action groups’ fidelity, role in coordination, role in local adaptation, support from external facilitators and data on student needs, and acceptability in engaging members. Fidelity was high in the first two years but lower in the third year when external facilitators withdrew. Student needs data were perceived as useful, but views on external facilitators were mixed. Groups successfully reviewed policies and rules, planned activities and coordinated restorative practices, but were less successful in implementing the curriculum. Success was facilitated by the involvement of school leaders. Members reported high satisfaction and empowerment. Action groups are a promising strategy for leading whole-school health promotion. Implementation is supported by external facilitation, local data and involvement of senior managers.     

论扶贫政策制定与执行中公民参与的运用

伴随着公共决策方式的变化和人们对于公共管理性质与作用的重新认识,公民在公共管理中的作用也急剧扩大。始于20世纪60年代一直持续发展至今的新公民参与运动, 不断激发着人们的研究兴趣,并使公民参与成为当今公共决策中的一个不可分割的组成部分。在这样的一个大背景下,扶贫政策的制定和执行也不可避免地引入了公民参与,并且经过理论和实践的证明,公民参与对于扶贫政策的制定和执行起着积极的作用,推动了扶贫方式的创新。我们由传统的扶贫方式开始转向“参与式”扶贫,变自上而下的决策方式为公民自下而上的参与方式,提高了扶贫政策的效果,取得了巨大成就。