An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features

This paper reports a systematic literature review examining empirical studies on the effects of peer assessment for learning. Peer assessment is fundamentally a social process whose core activity is feedback given to and received from others, aimed at enhancing the performance of each individual group member and/or the group as a whole. This makes peer assessment an interpersonal and interactional process. Using this social perspective in order to study learning effects, we focus on the impact of the structural arrangement of peer assessment on learning, and the influence of interpersonal variables.The literature search, focusing on empirical studies measuring learning outcomes in a peer assessment setting, resulted in 15 studies conducted since 1990 dealing with effects (performance or perceived learning gains) of peer assessment. Our analysis reveals that, although peer assessment is a social process, interpersonal variables have hardly been studied; more specifically, they were measured in only 4 out of 15 studies. Moreover, they are not used to explain learning gains resulting from peer assessment. Finally, comparing the studies with respect to structural features reveals that, although the differences between the studies are significant, there seems to be no relation with the occurrence of learning benefits. The results of this review seem to indicate that research on peer assessment from a social perspective is still in its infancy and deserves more attention.     

Inquiry-Based Learning and Undergraduates’ Professional Identity Development: Assessment of a Field Research-Based Course

Innovative strategies in inquiry-based learning are recognized as improving the quality of higher education learning, but there is a need to explore whether and how these strategies promote the development of professional identity among undergraduates. In this article we describe an inquiry-based course, situated in a European context, which is designed to support students’ professional identity development through reflective practice; and we present the assessment of students’ learning outcomes. Results highlight the key roles of field research, peer groups, and a narrative approach in promoting positive student attitudes toward reflecting on their production of knowledge and in developing their professional identity.     

Implementing Peer Assessment in a Post-Secondary (16–18) Physics Classroom

This study explores the implementation of peer assessment with a group of students studying physics at Advanced level in a post-secondary school in Malta. The study that draws on action research methodology looks at how the views of students regarding peer assessment evolve as they engage with peer assessment. The research involved the actual implementation of an apprenticeship model of peer assessment and looked at the issues faced by the students. The results of the study show that students were very much immersed in a traditional assessment culture and needed to be trained to develop peer assessment skills. Student concerns related to fairness, regarding their abilities as assessors and dependence on the teacher as ‘expert’ remained unchanged as the students engaged with peer assessment. Successful implementation of peer assessment therefore requires a re-examination of the role of student and teachers as assessors within a safe learning environment.     

The assessment cycle: a model for learning through peer assessment

This paper advances a model describing how peer assessment supports self-assessment. Although prior research demonstrates that peer assessment promotes self-assessment, the connection between these two activities is underspecified. This model, the assessment cycle, draws from theories of self-assessment to elaborate how learning takes place through peer assessment. The model is applied to three activity structures described in the literature to analyse their potential to support learning by promoting self-assessment. Broadly speaking, the model can be used to understand learning that takes place in a variety of peer assessment activities: marking/grading, analysis, feedback, conferencing and revision. This approach contrasts most studies on peer assessment, which have focused on calibration of instructor and peer grades, rather than learning opportunities.     

Everyday life and everyday learning: the ways in which pre-service teacher education curriculum can encourage personal dimensions of teacher identity

This article presents and discusses the findings of a research project, the main objective of which is to identify curriculum components that promote personal development as a nuclear part of teacher professional identity formation through pre-service teacher education. Curriculum is viewed as an ecological scenario with different subsystems and both as formal and informal. Identity formation is conceived as an ever-provisional result of a double transaction: the biographical one and the relational one. The curricula of four different historical periods of pre-service teacher education in Portugal and the professional identity of teachers trained within them were characterised through collection and analysis of documents and biographical narratives. Crossing results from the four periods, the quality of school climate emerges as an important variable to the quality of the teachers’ identity. The lifelong learning ethos seems to emerge when the training curriculum connects everyday learning with everyday life, namely by urging the students’ involvement in school life, peer learning activities and peer and teacher educators’ informal learning activities.     

The role of technology-facilitated peer assessment and supporting strategies: a meta-analysis

Abstract

Technology-facilitated peer assessment is gaining increasing attention. However, evidence for the contribution of technology-facilitated peer assessment to learning achievements has not been investigated. The present meta-analysis integrated findings on the effects of technology-facilitated peer assessment based on two main elements: (1) technology-facilitated peer assessment, (2) the use of extra supporting strategies in technology-facilitated peer assessment. A total of 37 empirical studies published from 1999 to 2018 were selected and analysed. Results indicated that technology-facilitated peer assessment had a significant and medium effect on learning achievements with an overall mean effect size of 0.576. The use of extra supporting strategies in technology-facilitated peer assessment also produced a positive and medium effect on students’ learning achievements with an overall mean effect size of 0.543. Different moderator variables, such as task types, assessment modes, training for assessors, durations, grouping types and assessment methods were related to different effect sizes. The results together with the implications for both practice and research are discussed.     

The development of science activities via on-line peer assessment: the role of scientific epistemological views

This study implemented an online peer assessment learning module to help 36 college students with the major of pre-school education to develop science activities for future instruction. Each student was asked to submit a science activity project for pre-school children, and then experienced three rounds of peer assessment. The effects of the online peer assessment module on student learning were examined, and the role of Scientific Epistemological Views (SEVs) in the learning process was carefully investigated. This study found that student peers displayed valid scoring that was consistent with an expert’s marks. Through the online peer assessment, the students could enhance the design of science activities for future instruction; for instance, the science activities became more creative, science-embedded, feasible and more suitable for the developmental stage of pre-school children. More importantly, students with more sophisticated (constructivist-oriented) SEVs tended to progress significantly more for designing science activities with more fun, higher creativity and greater relevancy to scientific knowledge, implying that learners with constructivist-oriented SEVs might benefit more from the online peer assessment learning process. These students also tended to offer more feedback to their peers, and much of the peer feedback provided by these students was categorized as guiding or helping peers to carefully appraise and plan their science activity projects. This study finally suggested that an appropriate understanding regarding the constructivist epistemology may be a prerequisite for utilizing peer assessment learning activities in science education.     

网络同伴互评活动设计与应用研究

同伴互评作为一种形成性评价策略,已经成为现阶段网络学习评价领域研究的热点。本研究将同伴互评策略应用于高校混合式教学之中,设计同伴互评学习活动。研究整体采用行动研究法设计、修改同伴互评活动,并分析学习者对活动的改进意见。研究一方面分析互评活动对在线学习效果的影响以及活动对于两类学习风格学习者影响的差异;另一方面深入分析学习者对于活动的态度及改进意见。主要研究结论认为：（1）研究设计的同伴互评活动能够促进学习者深入理解学习内容,并提升最终成果的质量。（2）相对综合型学习风格的学习者而言,同伴互评更加有助于提升序列型学习者对知识运用的水平。最后,研究总结了同伴互评活动设计的改进方案及后续设计要点。     

Promoting learning by leveraging the collaborative nature of formative peer assessment with instructional scaffolds

Over the past two decades, formative peer assessment has become a popular instructional approach. Initially, it was more readily applied in higher education but has since expanded to other educational levels, including primary and secondary education. The popularity is understandable given the increased amount of feedback by multiple peers and enhanced awareness of performance criteria. Although it is increasingly acknowledged by the research community that formative peer assessment is inherently a social endeavour, the collaborative nature is simultaneously the least-explored mechanism. The contributions in this special issue address this gap conceptualising peer assessment and peer feedback as both an individual and a collaborative learning practice. Furthermore, we highlight core learning conditions: learner characteristics, domain and task characteristics, and, finally, instructional scaffolds.     

Identity and belonging in social learning groups: The importance of distinguishing social,operational and knowledge‐related identity congruence

Collaborative learning has much to offer but not all learners participate fully and peer groups can be exclusive. The article examines how belonging or ‘congruence’ in learning groups is related to identities of gender, age, ethnicity and socio‐economic status. A study of student experiences of collaborative learning on three different blended learning courses illustrated how learners negotiate identity congruence with peer groups to belong and engage. An analytical framework that distinguishes social, operational and knowledge‐related identity congruence has emerged. Contrary to received wisdom, the social aspect appears least important for learner engagement while knowledge‐related identity congruence is fundamental. Some of the consequences of identity incongruence, particularly concerning gender and maturity, are discussed and the article points towards the pedagogies which might enable identities of group members to shift so that collaborative learning can flourish.     

促进学习的评价:在线开放课程中同伴互评投入度研究

后疫情时代,提质增效是在线开放课程内涵式发展的重要抓手。重申促进学习的评价,强调学习者的深度参与,探究学习者参与同伴互评的特点及现存问题具有重要的现实意义。文章基于国内一门在线开放课程,采用学习分析技术从结构化与非结构化数据入手,分析在线开放课程同伴互评过程中学习者的投入度,并进一步探究同伴互评投入度与学习绩效的关系。研究结果表明,在线开放课程中学习者同伴互评的行为投入及认知投入不高,但是参与同伴互评能够识别更有动力通过课程的学习者,且其认知投入度能够预测其学习绩效。基于FBM(学习行为模型)模型提出促进同伴互评投入度的两阶段干预策略,从而提高学习者参与同伴互评的投入度,并为教师及平台提供更具针对性的策略建议。     

Peer Learning and Assessment

Various forms of peer, collaborative or cooperative learning, particularly small group activities, are increasingly used within university courses to assist students meet a variety of learning outcomes. These include working collaboratively with others, taking responsibility for their own learning and deepening their understanding of specific course content. The potential benefits of peer learning have long been recognised and are especially relevant today. However, many existing assessment practices act to undermine the goals of peer learning and lead students to reject learning cooperatively. If assessment gives students the message that only individual achievement is valued, and that collaborative effort is akin to cheating, then the potential of peer learning will not be realised. Inappropriate assessment practices may also lead to unhelpful forms of competition within and between groups that prevent groups functioning effectively. This paper examines some of the main assessment issues in connection with peer learning and suggests ways in which the benefits of this approach can be maintained while still meeting the formal assessment requirements of the course. It discusses the use of group assessment, peer feedback and self‐assessment, assessment of participation and negotiated assessment and concludes with the identification of a number of issues which remain to be addressed.     

Opportunities to learn about Europe at school. A comparative analysis among European adolescents in 21 European member states

In this article, we investigate the relationship between different learning methods and the formation of European identity among adolescents. The analysis is based on the European module of the International Civic and Citizenship Education Study (2009), with 70,502 respondents in 21 European member states. The results show that offering opportunities for cognitive learning is more strongly related to European identity than social learning opportunities, i.e. opportunities for interactions with citizens from other European Union member states. The occurrence of an interaction effect between cognitive and social learning strategies, however, suggests that jointly offering both learning strategies can be considered the most effective tool for the formation of a European identity. The multilevel analysis reveals the impact of a more Eurosceptic climate on the country level, suggesting that living in a Eurosceptic member state is related to a weaker European identity among adolescents. We close with some observations on how curricula can contribute to a strengthening of European identity.     

Unravelling peer assessment: Methodological,functional, and conceptual developments

Peer assessment is an educational arrangement where students judge a peer's performance quantitatively and/or qualitatively and which stimulates students to reflect, discuss and collaborate. However, empirical evidence for peer assessment effects on learning is scarce, mostly based on student self-reports or involving comparison of peers' and teachers' ratings or anecdotal evidence from case studies. Systematic investigation of learning effects necessitates methodological, functional, and conceptual development in peer assessment research. This implies sound (quasi-)experimental studies, the definition of specific peer assessment mechanisms, and affiliations with other research domains. The articles in this special issue address these three needs and offer new directions for research.     

The differential effects of task complexity on domain-specific and peer assessment skills

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial design with the between-subjects factor task complexity (simple, n?=?51; complex, n?=?59) and within-subjects factor task type (domain-specific, peer assessment), secondary school students studied four integrated study tasks, requiring them to learn a domain-specific skill (i.e. identifying the six steps of scientific research) and to learn how to assess a fictitious peer performing the same skill. Additionally, the students performed two domain-specific test tasks and two peer assessment test tasks. The interaction effect found on test performance supports our hypothesis. Implications for the teaching and learning of peer assessment skills are discussed.     

Does peer assessment promote student learning? A meta-analysis

Abstract

In recent years, there has been an increasing use of peer assessment in classrooms and other learning settings. Despite the prevailing view that peer assessment has a positive effect on learning across empirical studies, the results reported are mixed. In this meta-analysis, we synthesised findings based on 134 effect sizes from 58 studies. Compared to students who do not participate in peer assessment, those who participate in peer assessment show a .291 standard deviation unit increase in their performance. Further, we performed a meta-regression analysis to examine the factors that are likely to influence the peer assessment effect. The most critical factor is rater training. When students receive rater training, the effect size of peer assessment is substantially larger than when students do not receive such training. Computer-mediated peer assessment is also associated with greater learning gains than the paper-based peer assessment. A few other variables (such as rating format, rating criteria and frequency of peer assessment) also show noticeable, although not statistically significant, effects. The results of the meta-analysis can be considered by researchers and teachers as a basis for determining how to make effective use of peer assessment as a learning tool.     

Formative peer assessment in a CSCL environment: a case study

In this case study our aim was to gain more insight in the possibilities of qualitative formative peer assessment in a computer supported collaborative learning (CSCL) environment. An approach was chosen in which peer assessment was operationalized in assessment assignments and assessment tools that were embedded in the course material. The course concerned a higher education case‐based virtual seminar, in which students were asked to conduct research and write a report in small multidisciplinary teams. The assessment assignments contained the discussion of assessment criteria, the assessment of a group report of a fellow group and writing an assessment report. A list of feedback rules was one of the assessment tools. A qualitative oriented study was conducted, focusing on the attitude of students towards peer assessment and practical use of peer assessment assignments and tools. Results showed that students’ attitude towards peer assessment was positive and that assessment assignments had added value. However, not all students fulfilled all assessment assignments. Recommendations for implementation of peer assessment in CSCL environments as well as suggestions for future research are discussed.     

基于同侪互评的在线学习评价研究综述

在线同侪互评是一种形成性评价策略。它通过学习者之间相互给予评分或反馈来改善学习成果,对学习者的学习成绩、态度及高阶思维能力等方面能够产生重要的影响。首先回顾了近10年来国际在线同侪互评的主要文献．从概念与内涵、程序与方法、系统与工具,以及效果与影响等四个方面综述了现有研究成果。而后,展望未来研究可能聚焦于：同侪互评与其他网络教学策略整合、同侪互评与教师评价结合运用、同侪评语分析等主题。     

Peer Assessment of Student Constructed Posters: Assessment Alternatives in Preservice Mathematics Education

In this article I discuss a four-year action research project that involved the development of effective assessment tools for preservice mathematics teachers. The focus of the article is on peer assessment in which students reviewed posters created by their peers. The article discusses the strategies that were used and the implications that arose from the project. I argue that peer assessment is an effective tool for assessment in preservice mathematics teacher education but must not be seen as an alternative to teacher-based assessment due to the variability in marks between and within student cohorts. The value of peer assessment is its potential as a learning tool. This revised version was published online in August 2006 with corrections to the Cover Date.