公开考试评卷工作的重要程序——香港的经验

公开考试的设计是一项专门的学问。公开考试一般包括三个重要元素：设计考试大纲、编制试题及评分、报告成绩。在考生人数众多的科目,评卷环节面临的重要问题之一就是须聘用大量评卷员参与评卷。但是,评卷是一项以专业判断为主的工作,为保证对所有考生公平,考评机构便须采取适当步骤,划一评卷尺度。本文旨在探讨香港考试及评核局为保持评卷信度,确保考生的答卷得到公平及客观的处理,在评卷环节所采取的程序,包括评卷参考的设计、评卷的配套程序、评卷教师的培训等,并分析这些程序的重要意义。     

Application of ICT and rubrics to the assessment process where professional judgement is involved: the features of an e‐marking tool

This paper describes a new ICT assessment tool that reduces multi‐handling of marks, comments and scores specifically where professional judgement is involved. Whereas previous ICT applications in most fields of education have focused on student learning, this tool focuses on the lecturer task of the assessment process. Unlike many ICT based assessment tools, it enables the assessor to make and record professional judgements. This tool moves the marking/recording sheet off the desk and onto the desktop (computer screen). The aim in using the tool is to reduce the unproductive busy work of marking such as adding up of marks, recording, and spell checking of comments by hand, also it allow for many different views of the information, and to increase the time spent on feedback, reflection and moderation. The tool combines features of the word processor, spreadsheet and database applications, and paper‐based marking. These features are described and discussed as working examples are presented. The paper also describes other features or possible uses of the tool, including the development and refinement of rubric‐based scales; the recording of feedback by markers about the quality of the marking key during and after marking; the ability to compare one’s marks with those of the coordinator electronically; the evaluating and analysing of results; and the printing out of many different views of the data.     

The judgement processes involved in the moderation of teacher-assessed projects

Classroom-based assessments have the potential to enhance validity by facilitating the assessment of important skills that are difficult to assess in written examinations. Such assessments tend to be marked by teachers. To ensure consistent marking standards, quality assurance procedures are needed. In the context of continued debate over the robustness of assessment by teachers, this research aimed to provide insights into the cognitive and social processes involved in the moderation of project work in General Certificate of Secondary Education qualifications (taken by many 16 year olds in England). The research involved nine examiners across three subject areas ‘thinking aloud’ whilst moderating the marks given to six students (treated as a mock school). The participants were also interviewed. These methods were chosen to provide an in-depth look at the judgement processes involved. The research provides an improved understanding of moderation judgements and how aspects of this process relate to existing theories of judgement. Improving our understanding of the judgement processes involved when an examiner moderates teacher marking can help to evaluate the appropriateness of this assessment method, may contribute to debate on whether teacher marking can be sufficiently well verified by quality assurance procedures, and may inform moderation practice.     

The impact of anonymous marking on students’ perceptions of fairness,feedback and relationships with lecturers

Anonymity in marking is a contentious issue within higher education. Conflicting research findings have identified issues surrounding gender bias, ethnicity bias and fairness in marking. However, the effects of anonymity upon feedback mechanisms have not been systematically explored. This study sought to understand the effects of anonymous marking and feedback upon students’ perceptions of its potential for future learning and relationship building with their lecturers. First year United Kingdom undergraduate business, politics, pharmacy and french students experienced anonymous and non-anonymous marking of coursework across different modules. Student performance data were collected, and a survey was administered following the completion of their modules. Results revealed that anonymous marking did not seem to advantage or disadvantage particular groups of students in terms of grade outcome. There was no significant difference in perceptions of fairness according to whether or not marking was anonymous. Furthermore, the results suggest that anonymous marking might undermine the learning potential of feedback, and minimise the strength of the relationship between lecturers and students, which may minimise the role of dialogue in the feedback process.     

Curtailing marking variation and enhancing feedback in large scale undergraduate chemistry courses through reducing academic judgement: a case study

Variation in marks awarded, alongside quality of feedback, is an issue whenever large-scale assessment is undertaken. In particular, variation between sessional teaching staff has been studied for decades resulting in many recorded efforts to overcome this issue. Attempts to curtail variation range from moderation meetings, extended training programmes, electronic tools, automated feedback or even audio/video feedback. Decreased marking variation was observed whenever automated marking was used, potentially due to less academic judgment being used by the markers. This article will focus on a case study of three interventions undertaken at Monash University that were designed to address concerns around the variability of marking and the feedback between sessional teaching staff employed in the chemistry teaching laboratories. The interventions included the use of detailed marking criteria, Excel marking spreadsheets and automated marked Moodle reports. Results indicated that more detailed marking criteria had no effect whilst automated processes caused a consistent decrease. This was attributed to a decrease in the academic judgment markers were expected to use. Only the Excel spreadsheet ensured the provision of consistent feedback to students. Sessional teaching staff commented that their marking loads were reduced and the new methods were easy to use.     

Investigating the relationship between quality,format and delivery of feedback for written assignments in higher education

Feedback can have a great impact on student learning. However, in order for it to be effective, feedback needs to be of high quality. Electronic marking has been one of the latest adaptations of technology in teaching and offers a new format of delivering feedback. There is little research investigating the impact the format of feedback has on quality of feedback and subsequently on student learning. This study sets out to investigate the impact paper-based and electronic methods of assignment submission and return have on students’ and markers’ perceived quality of feedback. Students and markers on an undergraduate course were asked to complete an anonymous online survey investigating their perceptions of quality, format and timeliness of feedback delivered electronically and on paper. The results showed that marking and providing feedback electronically was an acceptable method for markers, reporting improved speed and consistency of marking. There was no increase or decrease in satisfaction with the feedback received. Overall, electronic marking was found to be an acceptable method of delivery of feedback on written assignments by both students and markers. The findings of this study suggest that electronic marking can result in more timely feedback for students without impacting on quality.     

From piloting e‐submission to electronic management of assessment (EMA): Mapping grading journeys

The increasing interest in electronic management of assessment is a sign of a gradual institutionalisation of e‐submission and e‐marking technologies in UK Higher Education. The effective adoption of these technologies requires a managed approach, especially a detailed understanding of current assessment practices within the institution and the development of new or adapted business processes. The findings from close participant observation of assessment processes over a 2‐year period across a large Faculty reveal that three independent variables around (1) initial marking, (2) internal quality assurance and (3) the timing of the return of e‐feedback to students, determine variance in grading journeys. Despite the apparent wide variety of processes, five major grading journeys prevail: three varieties of moderation (moderation of multiple markers and moderation of single markers either before or after grades and feedback are released to students); and two forms of second marking (either blind or open to peers). Within an institution, the identification of major workflows is fundamental to both an effective implementation of assessment technologies and in conducting change. The identification of major workflows across UK Higher Education Institutions remains critical to attain the necessary software development from global vendors.     

A comparative judgement approach to teacher assessment

We report one teacher’s response to a top-down shift from external examinations to internal teacher assessment for summative purposes in the Republic of Ireland. The teacher adopted a comparative judgement approach to the assessment of secondary students’ understanding of a chemistry experiment. The aims of the research were to investigate whether comparative judgement can produce assessment outcomes that are valid and reliable without producing undue workload for the teachers involved. Comparative judgement outcomes correlated as expected both with test marks and with existing student achievement data, supporting the validity of the approach. Further analysis suggested that teacher judgement privileged scientific understanding, whereas marking privileged factual recall. The estimated reliability of the outcome was acceptably high, but comparative judgement was notably more time-consuming than marking. We consider how validity and efficiency might be improved and the contributions that comparative judgement might offer to summative assessment, moderation of teacher assessment and peer assessment.     

The influence of reputation information on the assessment of undergraduate student work

The present study employed an experimental design to examine the influence of knowledge of a student’s previous performance and the general quality of their writing style on the assessment of undergraduate student work. Fifteen sport and exercise physiology academics were asked to mark and give feedback on two final year undergraduate student essays. The first student essay that participants marked was a control essay. The second essay was the target essay. Participants read one of three student reputation profiles (positive, negative or neutral) prior to marking this essay. Kruskal–Wallis tests for difference indicated that the marks awarded to each essay did not significantly differ between the three student reputation conditions. Thematic analysis of the target essay also revealed no apparent differences in the way in which feedback was presented across the three student reputation profiles. It was therefore concluded that non-anonymous marking did not induce marker bias in this instance.