The judgement processes involved in the moderation of teacher-assessed projects

Classroom-based assessments have the potential to enhance validity by facilitating the assessment of important skills that are difficult to assess in written examinations. Such assessments tend to be marked by teachers. To ensure consistent marking standards, quality assurance procedures are needed. In the context of continued debate over the robustness of assessment by teachers, this research aimed to provide insights into the cognitive and social processes involved in the moderation of project work in General Certificate of Secondary Education qualifications (taken by many 16 year olds in England). The research involved nine examiners across three subject areas ‘thinking aloud’ whilst moderating the marks given to six students (treated as a mock school). The participants were also interviewed. These methods were chosen to provide an in-depth look at the judgement processes involved. The research provides an improved understanding of moderation judgements and how aspects of this process relate to existing theories of judgement. Improving our understanding of the judgement processes involved when an examiner moderates teacher marking can help to evaluate the appropriateness of this assessment method, may contribute to debate on whether teacher marking can be sufficiently well verified by quality assurance procedures, and may inform moderation practice.     

Purpose,Value and Competence: Contextualising competence based assessment in the civil aviation sector

Abstract

This paper outlines some of the issues emerging from research in to competence based training and assessment, and National Vocational Qualifications. A model for analysing the key purposes of formal assessment systems is suggested and a conceptual framework is presented which uses the notions of use and exchange value for gauging the worth of qualifications. The author presents material collected during a study of the training and assessment of student air traffic controllers to illustrate how contextual features, particularly at the occupational level, can affect the purpose and design of a qualification as well as the value attached to it by recipients and employers. It is suggested that elements such as: ‘purpose’, ‘perceived value’ ‘qualification design’ and ‘context’ provide a conceptual framework within which further understanding of the role of qualifications can be achieved. The longer term aspiration of the analysis begun in this paper is the development of a context‐sensitive theory of occupational qualifications.     

Assessing mathematics within advanced school science qualifications

Following sustained discussion regarding the relationship between advanced mathematics and science learning in England, the government has pursued a reform agenda in which mathematics is embedded in national, high stakes A-level science qualifications and their assessments for 18-year-olds. For example, A-level Chemistry must incorporate the assessment of relevant mathematics for at least 20% of the qualification. Other sciences have different mandated percentages. This embedding policy is running in parallel with an adding policy that is encouraging all young people to include the study of mathematics to 18. In this paper, we present a detailed scrutiny of the published sample assessment materials in the new A-level Physics, Chemistry and Biology qualifications in order to consider what the impact of this policy move might be for the teaching and learning of mathematics, its applications in upper secondary school advanced science studies and the implications in the transition to mathematically-demanding undergraduate studies.     

Designing and developing a programme-focused assessment strategy: a case study

Abstract

This case study describes the process that the Humanities Programme Team, in Dublin City University’s Open Education Unit, has undertaken with regard to developing a systematic, programme-focused assessment strategy. It charts the development of an Assessment Matrix that facilitated the enhancement of programme coherence in the context of a distributed modular provision model, and facilitated the use of an appropriate range and scaffolding of assessment types in relation to learning outcomes. To conclude, four recommendations on developing a programme-focused assessment strategy are presented: explicitly link assessments to both programme and module learning outcomes; provide adequate supports and resources for those designing and writing assessments; provide those Assessment Writers with appropriate professional development, as well as an opportunity to be part of an assessment writing community; and to embed a programme-focused assessment strategy into a programme’s quality assurance processes.     

关于建设我国高等教育质量保证体系的若干思考

加强科学管理,建设科学完善的高等教育质量保证体系,是高等学校提高教学质量的重要措施之一。高等教育质量保证离不开教学评估。创新高等教育质量保证体系,应在对以往本科教学评估进行科学、系统评估(元评估)的基础上进行。高等教育质量保证体系由内部质量保证体系和外部质量保证体系构成,其中内部质量保证体系是基础,外部质量保证体系是动力。面对新的形势和环境的变化,我们应努力探索新的高等教育质量评估范式。     

澳大利亚学前教育质量评估研究的发展与启示

2012年1月,澳大利亚推出了新的"早期教养国家质量框架",该框架包括立法、质量标准、评估程序及评估机构四部分内容。以这一框架为核心,澳大利亚政府鼓励各学术机构参与学前教育质量评估研究,推动了"学前教育有效性研究"等大型评估项目的开展。这告诉我们,必须高度重视评价的作用,拓展评价主体,并不断提高评价质量。政府主导下的多元、科学的评价体系也将是我国学前教育质量评价研究发展的趋势。     

Validating written feedback in clinical formative assessment

Abstract

Formative assessment is widely accepted as being crucial to promoting student learning and, since 2010, the UK General Medical Council has mandated its use in workplace-based clinical training for all new doctors. As a result, the Royal College of Radiologists (RCR) instituted a range of formative workplace-based assessments including the Radiology Direct Observation of Procedural Skills (Rad-DOPS), in which supervisors appraise trainees’ performance in carrying out clinical procedures. This paper reports on the quality of the written feedback in 2,500?Rad-DOPS online feedback forms in addressing the aims of the new assessment approach. Random samples of 500 were selected from the first three years of the new assessment implementation, 2010–13, and from 2016 to 17. Using an appropriate coding frame, the feedback was analysed across the samples against key trainee attributes including stage of training and level of adjudged competence. Criteria for identifying high quality feedback were derived from the literature and a simplified form of qualitative comparative analysis was used to identify the conditions associated with high quality feedback. An average of 97% of the assessments contained written feedback but the number of instances of high quality feedback was found to be exceedingly small at around 5%. The paper offers suggestions for making the feedback process more purposeful in achieving the aims of formative assessment.     

Globalisation and higher education: from within-border to cross-border

Globalisation, the shift to a knowledge economy, and changing demographics are increasingly challenging higher education systems. The move from elite through mass to universal education, coupled with the internationalisation of higher education, has profoundly influenced the system, especially in terms of academic mobility. It has created new fields and challenges for policy-makers. New forms of international education have emerged, including cross-border education. Commercialised for-profit cross-border degree mills and rogue providers have gained new grounds and jeopardised the quality of higher education services and qualifications because of the worthless qualifications and shoddy service levels that they may provide. The focus of national quality assurance agencies, which was previously not related to the assessment of the quality of ‘imported’ and ‘exported’ programmes, has been challenged. This paper provides a theoretical overview of the main developments in the cross-border education level. It focuses on cross-border education opportunities and threats, and on the various international frameworks that regulate these new forms of higher education.     

Quality Assurance Issues in European Distance Education

Four major issues concerning quality in European distance education are addressed. (1) The lack of flexibility in educational institutions is a threat to the quality of the present education system. (2) New technologies offer new potential, but also confront us with a need for quality assessment and assurance. (3) New markets in industry and in new regions of Europe highlight the need for quality standards and guidelines. (4) Finally, public assessment and control of quality are under continuous discussion. In general, quality assurance should be approached in various ways and with different measures at institutional, national and international levels.     

普通高中学生学业评价体系的重构

新课程的评价理念、结构特点和内容都和现行高中的学生学业评价不吻合,有必要重构普通高中学生学业评价体系。新的高中学生学业评价体系分模块水平和毕业水平两个层次、采用包括过程性评价与终结性评价两种方式,简称为2×2的评价体系。过程性评价的主要作用在于促进学生及时地进行反思和展示学生学习的过程。终结性评价可根据模块的内容和类型的不同,组织书面笔试或开放式、表现式的测试。     

跨国教育的质量保障、认证和资格认可

20世纪80年代以来,跨国教育快速发展.各种形式的跨国教育对现行的质量保障、认证和资格认可机制提出了一系列的挑战.跨国教育今后的发展,在很大程度上取决于国家和国际层面上质量保障、认证和资格认可方面的制度安排.本文对跨国教育的现行质量保障、认证和资格认可机制进行了梳理和分析.     

Classroom assessment as the co-regulation of learning

ABSTRACT

Until recently, the classroom assessment literature has emphasized the role of teachers and tests, for example investigating teachers’ assessment practices or the quality of classroom tests and other assessments. In contrast, current understandings of teaching and learning emphasize the role of students, as well as the complex interactions between teachers, students, and contexts. We use the literature review method to give substance to a theory of classroom assessment as the co-regulation of learning by teachers, students, instructional materials, and contexts. We organize the literature using a version of Pintrich and Zusho’s theory of the phases and areas of the self-regulation of learning, expanded to include the co-regulation of learning, in order to demonstrate how classroom assessment is related to all aspects of the regulation of learning. We conclude that this is a useful expansion for the field.     

Qualifications,examinations and assessment: views and perspectives of students in the 14–19 phase on policy and practice

This paper brings to the forefront students’ views on one of the most significant aspects of educational reform in the 14–19 phase in England, specifically qualifications, examinations and assessment reform. In this respect, the paper foregrounds students as ‘policy actors’, they are significant players in the mediation of national qualification systems rather than just subjects in their implementation. Data from a national dataset of focus groups with 243 students are presented. Key themes are highlighted relating to young people’s experiences of qualifications, examinations and assessment at this stage of education in a context of continuous initiatives and change as well as the impact on students of qualifications reform in situ which can be confusing, unsettling and ultimately detrimental to future success.     

“苏格兰学分与资格框架”的质量保障机制研究

质量保障机制应成为资格框架建立及运行过程中的核心问题。作为涵盖苏格兰311个组织11500个“资格”在内的“苏格兰学分与资格框架”,其质量保障机制的特色是“‘共担责任’的质量保障模式”“坚守质量保障‘原则’”以及“将‘学分评级机构’作为资格框架质量保障的核心和关键环节”。“苏格兰学分与资格框架”的质量保障机制,对推动“苏格兰学分与资格框架”成为国际上最成熟的资格框架之一功不可没,其经验可以为我国尚处于探索阶段的国家资历框架建设提供重要的借鉴。     

A Fair Comparison? The Evolution of Methods of Comparability in National Assessment

Comparability is a term used in educational assessment to describe the relative outcomes of one assessment against another. It is investigated in order to inform the Awarding Bodies, centres, candidates and the general public about the extent to which standards are consistent between assessments. The purpose of this paper is to demystify some technical aspects of how national assessment standards are guarded, by making details of these methods of comparability accessible to practitioners, managers, policy makers and researchers. In doing so, the authors review the use and techniques of comparability, describe the methodology currently in use, and discuss its application towards ensuring justice within national assessment.     

Quality assurance and assessment in education in Papua New Guinea

Education in Papua New Guinea (PNG) has experienced major shifts and changes over the last decade or so under the National Education Reform. In that context there is recognition for schools to appreciate their local contexts and conditions regarding assessment. The National Educational Department is also aware of the fact that there is need for some level of control from the national perspective to ensure a level playing field for assessment. Along with school–based assessment and national assessment processes, teacher inspections and appraisal, school and curriculum observations and appraisal are also recognized as elemental in ensuring quality control across the country. This article represents the modes and methods of assessment and quality assurance in PNG within the context of the current policy debates and policy development on education reform.     

Designing and Validating Assessments of Complex Thinking in Science

Typical assessment systems often measure isolated ideas rather than the coherent understanding valued in current science classrooms. Such assessments may motivate students to memorize, rather than to use new ideas to solve complex problems. To meet the requirements of the Next Generation Science Standards, instruction needs to emphasize sustained investigations, and assessments need to create a detailed picture of students’ conceptual understanding and reasoning processes.

This article describes the design process and potential for automated scoring of 2 forms of inquiry assessment: Energy Stories and MySystem. To design these assessments, we formed a partnership of teachers, discipline experts, researchers, technologists, and psychometricians to align curriculum, assessments, and rubrics. We illustrate how these items document middle school students’ reasoning about energy flow in life science. We used evidence from review by science teachers and experts in the discipline; classroom experiments; and psychometric analysis to validate the assessments, rubrics, and automated scoring.     

Analysis of assessment practice and subsequent performance of third year level students in natural sciences

Abstract

Summative assessment qualifies the achievement of a student in a particular field of specialization at a given time. Questions should include a range of cognitive levels from Bloom's taxonomy and be consistent with the learning outcomes of the module in question. Furthermore, a holistic approach to assessment, such as the application of the principles of the Herrmann Whole Brain Model, needs to be used to accommodate learning style diversity. The purpose of this study was to analyse, assess and compare the summative assessment of two third year level modules in the Bachelor of Science degree programme, namely Biochemistry and Zoology as part of action research with a view to enhancing the professional development of the lecturers involved. The questions posed in summative assessments were classified in terms of Bloom's differentiation of cognitive levels and the four different learning styles determined by Herrmann. Spearman's non-parametric analysis indicated that no correlation existed in this study between cognitive level and student performance based on achievement. In addition, there was not much difference between the cognitive levels and student performance between the two disciplines. Although the students seemed to do better at application level questions, the authors need to reflect on whether the assessments were valid with respect to the learning outcomes, methods of facilitating learning, and the assessments based on cognitive levels and learning style preferences. We conclude that continuous action research must be taken to improve the formulation of learning outcomes and students' achievement of these outcomes and quality of student learning – the main aim being the successful completion of the modules.     

Contesting educational assessment policies in Australia

ABSTRACT

Assessment is a major component of education, significant in directing what is identified as valued student learning. This paper is framed within an understanding of imperative and exhortative policy. Two paradigmatically different, and potentially contesting, assessment policy directions in Australian education – educational accountability to monitor school and teacher performance, and teacher assessment practices to improve learning (assessment for learning [AfL] or formative assessment) – are examined for their impact on teacher professionalism. Both approaches have official endorsement in Australian policy. Mandated participation in national tests is indicative of educational accountability assessments under national direction. While also endorsed nationally, AfL implementation is reliant on state and territory direction. Our examination reveals tensions in the alignment of both policies. This is evident in the impact of accountability assessment on AfL implementation, in particular, teachers’ understandings of valued assessment evidence. We conclude that a paradigmatic shift to support student learning in Australian schools is a policy imperative that includes the need for professional development and learning support for teachers.     

A systematic review of trends and findings in research employing drawing assessment in science education

ABSTRACT

In this study, we reviewed 76 journal articles on employing drawing assessment as a research tool in science education. Findings from the systematic review suggest four justifications for using drawing as a type of research tool, including assessment via drawing as (a) an alternative method considering young participants’ verbal or writing abilities, and affective or economic reasons, (b) a unique method that can reveal aspects not easily measured by other methods, (c) a major method that reflects characteristics of science subjects, and (d) a formative assessment to diagnose students’ ideas to benefit their learning. Furthermore, five research trends of studies using drawing as assessment tools are identified, including: (a) students’ conceptions of scientists from the Draw-a-Scientist-Test (DAST) and evolving studies, (b) students’ understanding or mental models of science concepts, (c) participants’ conceptions of science learning or teaching, (d) students’ inquiry abilities and modelling skills via drawing, and (e) technology to support drawing. For each trend, we synthesised and commented on the current findings. A framework conceptualising phases and issues when designing research and instruments employing drawing assessments is proposed. The review provides insights into the design and future direction of research employing drawing assessments in science education.