Relationships between student characteristics and self‐, peer and tutor evaluations of oral presentations

There are many influences on how assessors grade themselves and others. Oral presentations are useful for exploring such factors in peer, self‐ and tutor marked assessments, being rapidly completed and assessed, commonly used in HE and very difficult to anonymize. This opportunistic study examined the effects of gender and level of attainment on the triangulation of marks awarded to student presenters. Grades generated by peer assessment were associated more strongly with tutor‐awarded marks than those from self‐assessment. For self‐assessment there was a strong effect of gender (female students undervalued their performance compared with tutor grades). Peer assessment produced higher marks than from tutors, perhaps because of the close‐knit community developed during residential courses. For tutor marks, the greatest variability was at the lower end of the scale, whereas peer assessors were most variable when marking students who self‐evaluated or peer assessed highly. Students awarded a narrower range of marks to peers compared with tutors, but when self‐assessing used a larger range. Presentations by students who admitted to little sleep the night before received lower grades from both peers and tutors, but this was not reflected by self‐assessments, suggesting they were unaware of their poorer performances. Sessions with fewer talks (four rather than seven) reduced the ‘dip’ in marks previously observed in the middle of sessions. Findings are discussed in the context of bias in this mode of assessment.     

Optional self-assessment: some tensions and dilemmas

Self-assessment is promoted as an ideal form of assessment, particularly for adult learners as they need to be able to make judgements about how well they are doing something – whether it is related to learning, work or social interactions. However, there is also opposition to summative self-assessment on the grounds that students cannot validly and reliably assess their own work. At Massey University students enrolled in adult education qualifications are offered the opportunity to self or peer assess. This small-scale project investigated 472 student responses to self-assessment in this context. Findings show that 38% self-assessed by making comments on their work, and 25% awarded themselves a grade. A statistical analysis showed there was no significant difference (p > .01) between the self-assessed and teacher-assessed grades. Data also showed that higher achieving students tended to underrate and lower achieving to overrate. However, it is suggested that this may be a function of the grading scale rather than necessarily reflecting students’ capability to self-assess. Some tensions and dilemmas around optional self-assessment are identified and discussed.     

Feedback,feedforward: evaluating the effectiveness of an oral peer review exercise amongst postgraduate students

Assessment for learning approaches, such as peer review exercises may improve student performance in summative assessments and increase their satisfaction with assessment practices. We conducted a mixed methods study to evaluate the effectiveness of an oral peer review exercise among post-graduate students. We examined: (1) final assessment grades among students who did and did not take part in the peer review exercise; (2) student perceptions of the impact of the peer review exercise; and (3) student understanding of, and satisfaction with, this new assessment practice. We found that students who took part in the exercise had a significantly higher mean grade in a subsequent summative oral presentation assessment than students who did not take part in the exercise. Students gained a better understanding of assessment and marking criteria and expressed increased confidence and decreased anxiety about completing the subsequent summative assessment. Assessment for learning improves academic attainment and the learning experience in postgraduate students.     

Peer and self-assessment applied to oral presentations from a multidisciplinary perspective

This article analyses the use of peer and self-assessment in oral presentations as complementary tools to assessment by the professor. The analysis is based on a study conducted at the University of Girona (Spain) in seven different degree subjects and fields of knowledge. We designed and implemented two instruments to measure students’ peer and self-assessment, and a rubric to guide the assessment process. Results were compared with the marks awarded by the professor. In contrast with studies by other authors, which show a high correlation between these different assessment systems, our study revealed significant deviations. Applying peer and self-assessment to oral presentation activities also demonstrates their formative value above and beyond their summative usefulness.     

The 'Mini-Viva' as a Tool to Enhance Assessment for Learning

This paper reports on an action research project which aimed to promote assessment for learning within a summative assignment. In an attempt to provide enhanced feedback to students, the process of their group assignment involved lecturer feedback at various stages. A particular focus was on a 'mini-viva', whereby students explained and justified to the lecturer selected aspects of their assignment after it was submitted but before a mark was awarded. Oral and written evaluation data were collected from students, two peer observers and from colleagues taking part in an action research team. Some implications for 'good assessment practices' are discussed.     

Learning through science writing via online peer assessment in a college biology course

This study used an online peer assessment activity to help 47 college students to learn biology through writing. Each student submitted a biology writing report to an online system and then experienced three rounds of peer assessment. During the online peer assessment process, self, peer and expert evaluation scores for the writing were gathered across three rounds. It was found that self-assessment scores were not quite consistent with the expert's scores, but the peer assessment scores demonstrated adequate validity with the expert's evaluation. In particular, when the students had more rounds of peer assessment for reviewing the writing, the validity of the peer scores was enhanced. An examination of the students' writing scores, allocated by peers and expert, indicated that the students significantly improved the writing as the peer assessment activity proceeded. Content analyses of the students' writing also revealed that their writing gradually developed with significantly better coverage, richness and organization resulting from the online peer assessment activity.     

Peer assessment effects on the self-assessment process of first-year undergraduates

Peers carry potential in enhancing students’ self-assessment development, but few studies have explored how peer scaffolding is enacted in the process. This qualitative study explores peer assessment effects on the self-assessment process of 11 first-year undergraduates and the factors limiting peer influence. Drawing on the data from students’ journals, follow-up interviews, observations of in-class formative peer assessment activities and teacher interviews, we ascertained that peers could aid the self-assessment process by enriching student understanding of quality, refining subjective judgement and deepening self-reflection. Yet, peer influence could be reduced by distrust, tensions in feedback communication, competition and lack of readiness for peer learning. Implications for effective use of peers in supporting self-assessment development are discussed.     

'In Most Classes You Sit Around Very Quietly at a Table and get Lectured at…': Debates,assessment and student learning

This paper details a collaborative action research enquiry undertaken while both authors worked at the University of Glasgow. It explores the use of class debates as a teaching method in an International Management Honours course as the framing context for developing students' capacity to assess their own and each other's learning through the debates. In addition, issues of assessment for grading purposes are signalled and explored with the students. The collaborating partners in the study were a Management Studies lecturer and a Higher Education Studies lecturer, who worked together on the framework for the debates, reflected on the unfolding process together, and collected and analysed evidence. This case suggests that the debates enabled students to develop a critical view of the topics under discussion and to acquire a number of 'transferable skills', for example, team work. On peer grading, students were ambivalent. While self-and peer assessment appears to work well for formative purposes, summative peer assessment may not be welcomed by students.     

大学英语自主学习形成性评估实证研究

自主学习要求学习者监控学习进程和评价学习效果,传统的终结性评估不适用于自主学习。构建了适用于自主学习的形成性评估模式,该模式由"学习档案袋评价""教师评价"及"学生自评和他评"构成。经过在两个教学班级一学期的教学实验,结果表明:通过对学生自主学习过程的监控和管理,形成性评估激发了学生语言学习的动机和兴趣,促进了学生学习策略形成,有效培养了学生自主学习能力。     

The conundrum of low achievement and feedback for learning

Abstract

The literature on improving student engagement with assessment and feedback has a tendency to treat all students as if they are the same. Students with lower levels of attainment are generally under-represented within empirical studies and their feedback behaviours are less well understood. The recent drive to improve student assessment and feedback literacy and the move from ‘feedback’ being information about a task to being a process of understanding and using performance information is a larger conceptual leap for some students than others. In this paper, we consider issues surrounding the transition to new modes of feedback, focussing on what is needed for those who find study difficult and persistently are disappointed by their levels of attainment, to benefit from and take advantage of our feedback pedagogies. We examine literature advocating strategies such as increasing agency, using praise, developing feedback literacy and cultivating a growth mind-set. We argue that students who underachieve may benefit from strong relationships with educators and peers, exposure to feedback rich, low stakes environments, which permit repeated integrations of practice and feedback, and building feedback literacy through peer assessment activities.     

Accuracy of students' self‐assessment and their beliefs about its utility

The purpose of the two studies presented here was to evaluate the accuracy of students' self‐assessment ability, to examine whether this ability improves over time and to investigate whether self‐assessment is more accurate if students believe that it contributes to improving learning. To that end, the accuracy of the self‐assessments of 3588 first‐year students enrolled in a post‐secondary institution was studied throughout a semester during which each student made approximately 80 self‐assessments about his or her own learning process. These self‐assessments were then compared with multiple judgements by peers and tutors. The overall correlations between the scores of self‐, peer and tutor assessments suggest weak to moderate accuracy of student self‐assessment ability. The findings also reveal an ability effect; students judged as more academically competent were able to self‐assess with higher accuracy than their less competent peers. Comparing the accuracy of student self‐assessment averaged over four consecutive periods indicates that the accuracy does not improve over time. In a second study, a questionnaire aimed at eliciting student's beliefs about the effects of self‐assessment on their learning was administered to 936 first‐year students. Based on their responses, sub‐groups of students were identified: those who either believed in the usefulness of self‐assessment or did not. Results suggest that there is no significant association between student beliefs about the utility of self‐assessment and the accuracy of their self‐assessments.     

The impact of stress in self‐ and peer assessment

While a large amount of interest has been shown in the use of peer and self‐assessment, few studies have considered the effect of stress on the students involved. None have considered whether the resultant stress itself might account for any noticeable improvements in student performance. The research presented in this paper addresses this question. An experimental design measured the effects of type of assessment and gender on student stress levels and performance. Results suggest that females are more stressed by self‐assessment than males and that being subjected to self‐ and peer assessment, while more stressful, leads to improved student performance in summative tasks.     

Education students and their teachers: comparing views on participative assessment practices

Beliefs about the validity and merit of self‐, peer‐ and group‐assessment practices are presented from 213 pre‐service primary teachers and 30 staff who teach them. Both groups were surveyed using comparable items. A subset of seven staff participated in semi‐structured interviews. Staff were far more supportive of peer‐ and self‐assessment practices than their students with both groups indicating modest support for group assignments. While pedagogical factors best represent the staff support for engaging students in assessment of their own and their peers’ work, several staff in interviews revealed that their high level of support for peer‐assessment was closely linked to a time‐saving factor. Peer‐assessment was more often used than self‐assessment while both practices were reported to have increased over the past two to three years. These findings are consistent with the reported increase in participative assessment practices in higher education generally. An important implication of this research is that in order to optimise the use of participative assessment, staff need to better prepare their students by modelling and communicating their reasons for adopting such practices.     

Introducing Students to Self‐assessment

This study describes the introduction of self‐assessment to third‐year education students as a strategy for improving learning. Students were required to self‐assess a literature review using the marking guide developed by the course lecturer. The results showed a good level of agreement between students and the teacher but with a tendency for more students to overestimate than underestimate their grades. Interviews with students indicated that those who over estimated their grades were less clear about the standard by which to judge their work and looked for more specific guidance on what the lecturer ‘wanted’ than students whose estimates were closer to the lecturer's. The former group also tended to include ‘effort’ as a factor in their judgement. Students who responded more positively to the self‐assessment exercise were also more positive about the value of having a marking guide given out with the assignment. The paper concludes with suggestions for improving the study and some advice to other teachers who are contemplating the introduction of self‐assessment.     

学习性评价:涵义、方法及原理

学习性评价是以"促进学生发展、教师提高和改进教学实践"为目的的内部非正式评价,也是有效教学的一个有机组成部分.学习性评价的主要方法可以概括为:(1)改进课堂提问方式,增加回答"等待时间";(2)提高作业反馈质量,促进学生深入反思;(3)加强同伴与自我评价,形成自主与责任意识;(4)两种评价结合并用,优化评价教育功能.在科学课堂上实施学习性评价,首先改变了科学教师的教学方式,其次促进了学生学习方式的转变,提高课堂互动的效果和教学质量,从而有效地促进师生的共同发展.     

Evaluating the quality of peer and self evaluations as measures of student contributions to group projects

ABSTRACT

Educators in higher education commonly use peer and self evaluations to help assess student performance on group projects. Although these evaluations provide multiple benefits, many educators are wary of using them due to concerns about their quality. This study addresses three questions debated in the literature regarding the quality of these assessments. How much do students differentiate among peer contributions through their ratings? How reliable are peer ratings? How much agreement exists between peer and self ratings? Although these questions have been addressed to varying degrees in past work, their answers have been far from settled. While many studies focus on just one of the questions, this study’s data make it possible to address all three questions for the same group of students as well as examine each question by student performance level. The evaluations assessed in this study were completed by a large number of students under conditions associated with obtaining more valid and reliable ratings. Overall, the results provide support for using peer and self evaluations to help assess student contributions to group projects. Peer ratings were largely reliable as group members generally agreed on the scores given to their peers. In addition, most students differentiated among group member contributions through their ratings. Students also tended to rate themselves higher than their peers rated them. This study has implications for how peer and self evaluations can be most effectively used by educators to measure student performance in group work.     

The effects of peer judgements on teamwork and self-assessment ability in collaborative group work

A crucial determinant of the success or failure of collaborative group work is the effect of peer feedback interventions on learning. Research exploring such effects on developing soft skills is sparse. This study seeks to address whether peer feedback leads to enhanced teamwork behaviour and self-assessment ability, two skills highly sought after by employers. Specifically, this study examines the direct effect of formative performance rating and the mediating effect of praise and criticism in peer feedback messages on achievement in teamwork and self-assessment skills. The sample consists of quantitative and qualitative data from 98 students enrolled in business programmes using a particular form of collaborative group work. The paper finds a direct positive relationship between formative performance rating and summative self-assessment ability. It also finds that praise negatively mediates the relationship between formative performance rating and summative teamwork. Further analyses suggest that a significant proportion of comments provided is past rather than future-oriented. Potential strategies to overcome the limitations of current practices are discussed.     

Group assessment: comparing group and individual undergraduate module marks

This report describes a small study that analysed module marks of one cohort of science undergraduates from one academic year. It explored how group summative assessment marking affected the overall marks in comparison with individual assessment. A tutor allocated students to mixed ability project groups. Individual marks for the group work component were derived by tutor‐, peer‐, and self‐assessment weighting. The results showed that students with high individual marks obtained lower marks in the group component. Similarly, students with low individual marks obtained higher marks in the group component. Study limitations, results and conclusions are reported.