Qualifications,examinations and assessment: views and perspectives of students in the 14–19 phase on policy and practice

This paper brings to the forefront students’ views on one of the most significant aspects of educational reform in the 14–19 phase in England, specifically qualifications, examinations and assessment reform. In this respect, the paper foregrounds students as ‘policy actors’, they are significant players in the mediation of national qualification systems rather than just subjects in their implementation. Data from a national dataset of focus groups with 243 students are presented. Key themes are highlighted relating to young people’s experiences of qualifications, examinations and assessment at this stage of education in a context of continuous initiatives and change as well as the impact on students of qualifications reform in situ which can be confusing, unsettling and ultimately detrimental to future success.     

Assessing mathematics within advanced school science qualifications

Following sustained discussion regarding the relationship between advanced mathematics and science learning in England, the government has pursued a reform agenda in which mathematics is embedded in national, high stakes A-level science qualifications and their assessments for 18-year-olds. For example, A-level Chemistry must incorporate the assessment of relevant mathematics for at least 20% of the qualification. Other sciences have different mandated percentages. This embedding policy is running in parallel with an adding policy that is encouraging all young people to include the study of mathematics to 18. In this paper, we present a detailed scrutiny of the published sample assessment materials in the new A-level Physics, Chemistry and Biology qualifications in order to consider what the impact of this policy move might be for the teaching and learning of mathematics, its applications in upper secondary school advanced science studies and the implications in the transition to mathematically-demanding undergraduate studies.     

The judgement processes involved in the moderation of teacher-assessed projects

Classroom-based assessments have the potential to enhance validity by facilitating the assessment of important skills that are difficult to assess in written examinations. Such assessments tend to be marked by teachers. To ensure consistent marking standards, quality assurance procedures are needed. In the context of continued debate over the robustness of assessment by teachers, this research aimed to provide insights into the cognitive and social processes involved in the moderation of project work in General Certificate of Secondary Education qualifications (taken by many 16 year olds in England). The research involved nine examiners across three subject areas ‘thinking aloud’ whilst moderating the marks given to six students (treated as a mock school). The participants were also interviewed. These methods were chosen to provide an in-depth look at the judgement processes involved. The research provides an improved understanding of moderation judgements and how aspects of this process relate to existing theories of judgement. Improving our understanding of the judgement processes involved when an examiner moderates teacher marking can help to evaluate the appropriateness of this assessment method, may contribute to debate on whether teacher marking can be sufficiently well verified by quality assurance procedures, and may inform moderation practice.     

Educational assessment in England

This profile explains the assessment system in England, concentrating on those aspects that are related to government policy. It begins by putting the system in context; it then describes the national educational structure, curriculum and assessment arrangements. The government agencies responsible for carrying out education policies are introduced. In order to illustrate the intersection of curriculum and assessment development and political policy, the profile describes and discusses assessment issues that highlight the role of government and its agencies in the development, implementation and monitoring of England's national curriculum tests and 14‐to‐19 qualifications. It argues that successive governments have increasingly intervened directly in curriculum and qualifications development, and observes the growing governmental policy influence over assessment issues and the repercussions of that influence.     

PISA式汉语阅读测验的编制与维度评价

PISA测验着眼于学生的终生发展,其测验编制思想给各国教育评价带来了深刻的变革。本研究在PISA阅读测验理论与框架基础上,编制了PISA式汉语阅读测验。该测验包含三篇阅读材料,共18个测验项目。通过对测验难度、区分度、信度、效度的检测,并使用全息Bifactor模型进行维度评价。结果表明,编制的PISA式汉语阅读测验难度适中,具有较好区分度,信效度基本合格。同时,基本达到PISA对阅读测验能力结构的要求,较好地考查了学生的一般阅读理解能力,以及信息提取、文本解释、反思和评价等三个子维度的能力。     

The Return to Final Paper Examining in English National Curriculum Assessment and School Examinations: Issues of Validity,Accountability and Politics

There are sound educational and examining reasons for the use of coursework assessment and practical assessment of student work by teachers in schools for purposes of reporting examination grades. Coursework and practical work test a range of different curriculum goals to final papers and increase the validity and reliability of the result. However, the use of coursework and practical work in tests and examinations has been a matter of constant political as well as educational debate in England over the last 30 years. The paper reviews these debates and developments and argues that as accountability pressures increase, the evidence base for published results is becoming narrower and less valid as the system moves back to wholly end-of-course testing.     

The Coproduction illusion: considering the relative success rates and efficiency rates of securing an Education,Health and Care plan when requested by families or education professionals

This comparative study examines success and efficiency rates for Education, Health and Care (EHC) needs assessments in England when requested by families or education professionals. England is the only nation in the UK with EHC needs assessments. These are requested from the Local Authority (LA) if the school’s ordinarily-available provision is insufficient for individuals to make expected progress. Policy states that needs assessments should include parents, pupils and professionals in all decision making (coproduction). After a needs assessment, the LA may issue an EHC plan. This study considers whether the outcome of a needs assessment (issuing an EHC plan) is independent of the source of the request. Statistical analysis compared outcomes of needs assessments requested by families and education professionals. Analysis of publicly-available datasets and policy provided further context. Findings included: (i) a need for further research to explore inconsistencies in outcomes of needs assessments across England; (ii) LAs efficient at issuing EHC plans were consistently efficient for requests from education professionals and families; (iii) a potential discrepancy between families’ perceived level of involvement and the reality: a ‘coproduction illusion’. It is proposed that policy and guidance in England requires revision to ensure consistent use of language, roles, and responsibilities.     

School-based assessment of practical competence in science: Some issues from English experience

Teacher-conducted assessments of practical skills are a compulsory component of GCSE examinations in science in England and Wales. The paper presents some of the findings of a research project that investigated this aspect of a science teacher's work. The project was concerned principally with the ways in which science teachers interpreted and operationalised the policy decision of central government and examination boards that teachers assume responsibility for the internal assessment of the practical competence of pupils aged 14–16 preparing for GCSE examinations in science. Specializations: the social history and politics of school science education     

National qualifications in Scotland: A lightning rod for public concern about equity during the pandemic

Scotland, traditionally, has high levels of confidence in teachers. Fairness and justice are key concepts in policy and practice in Scottish education. For more than 100 years, the high-stakes assessment system in Scotland, with the Scottish Higher qualification at its heart, has been crucial to that sense of opportunity and justice. However, in 2019–2020, public confidence in high-stakes assessment in Scotland, as in other United Kingdom countries, was dented. In Scotland, the Covid-19 pandemic meant that schools were closed, teachers provided online learning opportunities for pupils working at home and, for the first time in 130 years, it was not possible to run national examinations. To ensure that learners were not further disadvantaged, alternative approaches to gathering evidence for qualifications were instigated. However, these results were challenged as socially unjust and the results that had been nationally moderated were replaced by results based on locally moderated teachers' professional judgement. As Scotland looks to qualifications beyond Covid-19, trust must be re-built. This article reports on a participative research project that sought to understand public perceptions of standards and fairness across a range of key communities following this experience. Drawing on both qualitative and quantitative data, we analyse factors which affected trust in National Qualifications under the pandemic. The evidence suggests that when considering what matters for qualifications to be trusted, technocratic solutions are likely to be rejected by stakeholders. Understanding and responding to what led to the mistrust of qualifications in Scotland will be crucial to inform its future qualifications system.     

A Conceptual Evaluation of Primary Assessment Policy and the Education Policy Process in the Republic of Ireland

This paper offers a conceptual evaluation of primary assessment policy and the policy process in the Republic of Ireland. It considers, in particular, the National Council for Curriculum and Assessment's policy document, A Programme for Reform: curriculum and assessment policy towards the new century and within that, the policy statement, The Curriculum at Primary Level: programme development and pupil assessment and it also refers to the primary teachers' union stance on assessment. It critically evaluates the Irish policy position on assessment in the light of recent literature in this field. The main argument of the paper is that the proposed policy is preoccupied with measurement issues and it does not sufficiently recognise the complexity of implementing formative assessment. The paper also seeks to explain the nature of current policy in Ireland with reference to the decision-making process and comparisons with other systems outside Ireland, particularly England/Wales, are made, where appropriate.     

英国基础教育质量标准《国家课程》及监控系统

《国家课程》是英国全国性的基础教育质量标准,其涵盖5-16岁儿童四个关键阶段14门课程的标准及九个学业评价表现水平划分。为确保"标准"切实落实,推行配套的"国家测评",成立相应管理机构,制订出台相关问责章程,是支持这一系统工程顺利运行的必要措施。当前,英国基础教育质量标准体系在制订与实施,设计与编制,管理与问责既显示出特色又反映出问题,为学者持续探讨提供空间。     

Controlled assessments in 14–19 Diplomas: implementation and effects on learning experiences

The Principal Learning components of 14–19 Diplomas (introduced in England in 2008) are assessed predominantly via “controlled assessments”. These assessments are conducted within the learning context under specified conditions (or “controls”) and require learners to apply their skills to work-related tasks. In this research, teachers and learners at 6 consortia (groups of schools/colleges working together to deliver Diplomas) were interviewed about how controlled assessments in the Diploma qualifications were affecting teaching and learning experiences.

The nature of the assessments was seen as encouraging learning and facilitating a less didactic learning environment. The assessments were generally considered less pressurised than traditional assessments, and most learners found the assessment tasks enjoyable and motivating. The assessments were reportedly encouraging the development of valuable skills (e.g., teamwork, communication, independent working) and improving students' confidence. However, there was some evidence that not all teachers had yet fully understood the requirements around the assessment “controls”.     

The learner experience of Curriculum 2000: implications for the reform of 14–19 education in England

In September 2000, new qualifications for 16–19 year olds, known as Curriculum 2000, were introduced in England, Wales, and Northern Ireland with the aim of broadening the advanced level curriculum. It was left to schools, colleges, and their learners, however, to decide how these new qualifications would be used to build learner programmes of study. In this voluntarist context, the views and actions of learners have been an important factor in determining the course of the reforms. This article, which is based primarily on a study of 42 learners in a school/college consortium in the south west of England, provides a chronological account of learner experiences of these reforms during their first 2 years of implementation. The consortium discussed here offered a high‐volume study programme similar in size and content to those being proposed by the Tomlinson Working Group on 14–19 curriculum and qualifications reform in England. Learners found their expanded programmes onerous, due in part to the nature of the new qualifications and in part to the context of their implementation. The article concludes by identifying a number of policy lessons for 14–19 reform arising from this research.     

Policy impact of PISA on mathematics education: the case of Norway

This article addresses the policy implications of participation in international large-scale assessments (ILSAs), particularly the Programme for International Student Assessment (PISA), and the ways in which such implications might influence mathematics education. Taking Norway as a special case, this discussion focuses on insights into teaching, learning and assessment practices that can be inferred from the PISA study, and how participation in ILSAs has contributed to educational policy and even changed policymakers’ perspectives on schools, teachers and students. Following publication of the PISA 2000 results, Norway experienced a ‘PISA shock’, leading to the implementation of a national quality assessment system and national tests. In addition, changes were made to the mathematics curriculum for compulsory school and to mathematics teacher education. More recently, public debate has focused less on rank and league tables, shifting instead to the high number of low-achieving students and the low number of high achievers. Moreover, there has been little uptake of policy advice provided by the Organisation for Economic Co-operation and Development (OECD), which focuses on strengthening accountability measures. Furthermore, although the Norwegian educational system in the past decade has undergone a decentralisation process, the educational system still follows the Nordic model, which focuses on equity and ‘education for all’. Analyses of the Norwegian case indicate that policymaking takes place in highly cultural contexts, and that international studies might be used merely to validate existing policy directions.     

Exploring the Impact of Rater Effects on Person Fit in Rater-Mediated Assessments

Researchers have documented the impact of rater effects, or raters’ tendencies to give different ratings than would be expected given examinee achievement levels, in performance assessments. However, the degree to which rater effects influence person fit, or the reasonableness of test-takers’ achievement estimates given their response patterns, has not been investigated. In rater-mediated assessments, person fit reflects the reasonableness of rater judgments of individual test-takers’ achievement over components of the assessment. This study illustrates an approach to visualizing and evaluating person fit in assessments that involve rater judgment using rater-mediated person response functions (rm-PRFs). The rm-PRF approach allows analysts to consider the impact of rater effects on person fit in order to identify individual test-takers for whom the assessment results may not have a straightforward interpretation. A simulation study is used to evaluate the impact of rater effects on person fit. Results indicate that rater effects can compromise the interpretation and use of performance assessment results for individual test-takers. Recommendations are presented that call researchers and practitioners to supplement routine psychometric analyses for performance assessments (e.g., rater reliability checks) with rm-PRFs to identify students whose ratings may have compromised interpretations as a result of rater effects, person misfit, or both.     

A Tale of Two Tests: tensions in assessment policy

A comparison of assessment policy in England and Vermont suggests the utility of viewing the assessment development process as resulting from the interplay of three loosely related games: policy making, assessment, and teaching. This process is characterized by two universal tensions. The technical one stems from the need to reconcile the goals of instructional improvement and accountability. The political tension stems from the differences in perspectives of policy makers, educationists, and educators. How tensions are resolved depends on the size of the unit, the introduction of broader ideological issues into the debate, and the quality of communication within and across games.     

英国高校招生考试制度改革动态及其启示

历经改革完善,英国建立了特色鲜明的现行高校招生考试制度,并对世界很多国家或地区的高校招生考试制度产生影响。进入新世纪,英国高校招生考试制度改革出现新的动向,逐步建立起国家资格证书体系架构,高校入学证书考试由单一走向多样化,不断扩大考生考试科目选择空间,更多采取综合评定方式和"高校招生分数转换系统"招收录取新生,重视招收弱势群体家庭学生。这些改革动向对我国正在进行的高考制度改革具有一定的启示。     

PISA as an ideational roadmap for policy change: exploring Germany and England in a comparative perspective

Education policy has undergone transformation in many countries over the last decade. In this article, we focus on the effects of the most significant international initiative in secondary education, which is the Organisation for Economic Co-operation and Development's (OECD) Programme for International Student Assessment (PISA). We analyse two countries that provide variation regarding the degree of change in their respective education policy-making due to this study; while Germany substantially reformed its education system in reaction to its mediocre PISA results, almost no change has been observed in England. As we show, alterations and shifts in ideas of education policy best account for such a change.     

Peer assessment in thesis oral presentation

Peer assessment has been studied in various situations and actively pursued as a means by which students are given more control over their learning and assessment achievement. This study investigated the reliability of staff and student assessments in two oral presentations with limited feedback for a school-based thesis course in engineering where the different disciplines ran their own assessment. Staff scores were generally found to be more reliable than student scores, but it was not consistent. The engineering disciplines displayed widely varying differences between the staff and student scores. A large variation in reliability was found for the different disciplines making it difficult to reconcile to a standard school-based overall grading scheme for the course when the reliability of marking was low. The results also showed that the average scores for oral presentations are generally much higher than for written examinations, based on the grade point average. Future research will need to investigate how best to develop a consistent assessment framework for oral presentations and moderation of scores for consistency, particularly for large classes where assessment are done by different groups of staff and students and collated into a single set of marks for the class.     

An Attempt to Reinterpret Student Learning Outcomes: A Cross-National Comparative Study

Attracting significant attention from the media and policymakers, international student assessments are serving as pressures on and challenges for individual countries. Even so, paying attention only to publicly released rankings in international student assessments may lead to superficial interpretations of the assessment results. Simply equating the rankings with educational outcomes can be a questionable perspective because the assessment results can be interpreted differently. In this context, this article has explored student achievement in the Programme for International Student Assessment (PISA) tests in comparison with civic knowledge achievement in the IEA Civic Education Study across 17 countries. This article has also evaluated efficiency in educational processes by assessing outputs (student performance in PISA) against inputs (educational spending and learning time) directly used in the education system. These two approaches clarify that publicly released rankings in PISA change to some extent when the rankings are reevaluated by taking other factors into consideration. This implication should be allowed for in interpreting the results of international assessment and the relative rankings of participating countries.