The differential effects of task complexity on domain-specific and peer assessment skills

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial design with the between-subjects factor task complexity (simple, n?=?51; complex, n?=?59) and within-subjects factor task type (domain-specific, peer assessment), secondary school students studied four integrated study tasks, requiring them to learn a domain-specific skill (i.e. identifying the six steps of scientific research) and to learn how to assess a fictitious peer performing the same skill. Additionally, the students performed two domain-specific test tasks and two peer assessment test tasks. The interaction effect found on test performance supports our hypothesis. Implications for the teaching and learning of peer assessment skills are discussed.     

Small group discussion as a key component in online assessment training for enhanced student learning in web-based peer assessment

The purpose of this study is to examine the effect of online assessment training, with synchronous group discussion as a key component, on subsequent web-based peer assessment results. Participants included 81 college students, mostly women, taking a business writing class. After initial submission of a draft counter-offer letter, they completed online assessment training by engaging in a consensus-seeking, synchronous group discussion of the rubric-based ratings they gave to sample counter-offer letters. They then engaged in web-based assessment of randomly assigned peer counter-offer letters and upon receiving peer feedback, submitted revision of their own counter-offer letter. The discussion groups were randomly assigned to either anonymous (using a pseudo name) or identified conditions (using real name). Findings indicate that the quality of student counter-offer letters improved after online assessment training, and improved even more after web-based peer assessment. There were no significant differences between anonymous discussion groups and identified groups.     

Investigating secondary school students’ unmediated peer assessment skills

The purpose of this study was to investigate secondary school students’ unmediated peer assessment skills. Specifically, 36 seventh graders, without receiving any kind of support, were anonymously assigned to reciprocally assess their peers’ science web-portfolios. Additionally, students’ attitudes towards and intentions about the use of unsupported reciprocal peer assessment were examined. Three data sources were used, namely, interviews, screen-video captured data, and the peer assessors’ feedback produced. Findings showed that the students have positive attitudes towards unsupported reciprocal peer assessment and that they intend to implement it again. It was also found that students have the skills, at least the beginnings, needed for the implementation of peer assessment. Specifically, they were found to be able to define and use their own assessment criteria, whose overall validity and reliability, however, were found to be low. Finally, the feedback they produced included grades, positive and negative judgments, as well as suggestions for changes.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

The role of training in improving peer assessment skills amongst year six pupils in primary school writing: an action research enquiry

Peer assessment is where students assess the quality of a peer's work. Studies have demonstrated its positive impact on learning yet most of these are in higher education. This study used training to improve the quality of written feedback in a year six primary school classroom. Action Research was selected as a research strategy given the need to find a solution to a classroom problem. Convenience sampling was used as the research focused on the class of children where the issue lay. Methods used to evaluate the quality of training included documents, observations, informal interviews and mind maps. Thematic analysis was used to analyse the data produced. The findings indicate that training may help to improve the quality of children's written feedback in primary school, as it has done in other contexts. Further research is needed to verify these results given the small sample size and absence of controls.     

The assessment cycle: a model for learning through peer assessment

This paper advances a model describing how peer assessment supports self-assessment. Although prior research demonstrates that peer assessment promotes self-assessment, the connection between these two activities is underspecified. This model, the assessment cycle, draws from theories of self-assessment to elaborate how learning takes place through peer assessment. The model is applied to three activity structures described in the literature to analyse their potential to support learning by promoting self-assessment. Broadly speaking, the model can be used to understand learning that takes place in a variety of peer assessment activities: marking/grading, analysis, feedback, conferencing and revision. This approach contrasts most studies on peer assessment, which have focused on calibration of instructor and peer grades, rather than learning opportunities.     

The role of technology-facilitated peer assessment and supporting strategies: a meta-analysis

Abstract

Technology-facilitated peer assessment is gaining increasing attention. However, evidence for the contribution of technology-facilitated peer assessment to learning achievements has not been investigated. The present meta-analysis integrated findings on the effects of technology-facilitated peer assessment based on two main elements: (1) technology-facilitated peer assessment, (2) the use of extra supporting strategies in technology-facilitated peer assessment. A total of 37 empirical studies published from 1999 to 2018 were selected and analysed. Results indicated that technology-facilitated peer assessment had a significant and medium effect on learning achievements with an overall mean effect size of 0.576. The use of extra supporting strategies in technology-facilitated peer assessment also produced a positive and medium effect on students’ learning achievements with an overall mean effect size of 0.543. Different moderator variables, such as task types, assessment modes, training for assessors, durations, grouping types and assessment methods were related to different effect sizes. The results together with the implications for both practice and research are discussed.     

Teaching domain-specific skills before peer assessment skills is superior to teaching them simultaneously

Instruction in peer assessment of complex task performance may cause high cognitive load, impairing learning. A stepwise instructional strategy aimed at reducing cognitive load was investigated by comparing it with a combined instructional strategy in an experiment with 128 secondary school students (mean age 14.0?years; 45.2% male) with the between-subjects factor instruction (stepwise, combined). In the stepwise condition, study tasks in Phase 1 were domain-specific and study tasks in Phase 2 had both domain-specific and peer assessment components. In the combined condition, these two components were present in all tasks in both phases. Final performance (i.e. speed and accuracy in domain-specific skills and peer assessment skills) showed no significant differences, but performance improved more from Phase 1 to Phase 2 in the stepwise condition than in the combined condition. The results suggest that, with complex study tasks, it might be beneficial to teach domain-specific skills before peer assessment skills.     

Peer assessment in the digital age: a meta-analysis comparing peer and teacher ratings

Given the wide use of peer assessment, especially in higher education, the relative accuracy of peer ratings compared to teacher ratings is a major concern for both educators and researchers. This concern has grown with the increase of peer assessment in digital platforms. In this meta-analysis, using a variance-known hierarchical linear modelling approach, we synthesise findings from studies on peer assessment since 1999 when computer-assisted peer assessment started to proliferate. The estimated average Pearson correlation between peer and teacher ratings is found to be .63, which is moderately strong. This correlation is significantly higher when: (a) the peer assessment is paper-based rather than computer-assisted; (b) the subject area is not medical/clinical; (c) the course is graduate level rather than undergraduate or K-12; (d) individual work instead of group work is assessed; (e) the assessors and assessees are matched at random; (f) the peer assessment is voluntary instead of compulsory; (g) the peer assessment is non-anonymous; (h) peer raters provide both scores and qualitative comments instead of only scores; and (i) peer raters are involved in developing the rating criteria. The findings are expected to inform practitioners regarding peer assessment practices that are more likely to exhibit better agreement with teacher assessment.     

Dancing with the devil: formative peer assessment and academic performance

Peer assessment can be important in developing active and independent learners, as well as providing more and faster feedback in large classes, compared to marking done by tutors. In addition, the evaluative, critical stance required by students in order to assess their peers' work encourages the development of higher-order cognitive skills. Changing roles from being assessed to being an assessor can also improve students' ability to judge and improve on their own work. However, peer assessment does have potential problems and there is some debate as to the appropriate academic level at which to implement it, the kinds of feedback that are given and the ways in which students respond. In addition, there is little evidence that peer assessment has an impact on academic performance. This research reports the results of an online peer assessment exercise for a macroeconomics essay conducted in a large Economics 1 class at Rhodes University. Of the 800 students, about half participated in the peer assessment exercise. Data were collected from students via a formal course evaluation. In addition, a sample of 50 essays was evaluated in terms of the relationship between peer marks and final (tutor) marks received and the impact that peer assessment had on the quality of the final essay submitted. An Ordinary Least Squares regression was used to investigate the impact of peer assessment participation on marks. Results showed that peer marks tended to ‘bunch’ in the 60–68% range, indicating the reluctance of peers to give very high or low marks. In general, peers gave more useful feedback on technical aspects, such as presentation and referencing (which were also the categories in which students most often made improvements), than on content. Regression analysis showed that peer assessment participation was not a significant determinant of final essay mark, but that economics ability and English language proficiency were.     

The effect of peer assessment on project performance of students at different learning levels

Peer assessment has been increasingly integrated in educational settings as a strategy to foster student learning. Yet little has been studied about how students at different learning levels may benefit from peer assessment. This study examined how peer-assessment and students’ learning levels influenced students’ project performance using a two-way factorial design. One hundred and thirty teacher education students participated in this quasi-experimental study. When working on a technology-integrated lesson plan project, the experimental group completed an online peer assessment process while the control group followed the discussion method. Students’ learning levels were measured and divided into low, average and high achieving according to the quality of their draft lesson plans. Data analysis suggested that the impact of peer assessment on students’ lesson plan project seemed to vary according to students’ learning levels. While low- and average-achieving students showed significantly improved performance right after the integration of a peer assessment model, the model seemed to have had less impact on the performance of high-achieving students. Significance, implications and limitations of findings are discussed.     

The quality of peer assessment in a wiki-based online context: a qualitative study

Peer assessment (PA) provides opportunities for authentic assessment, autonomy and collaboration. Several authors advocate that students can benefit from PA and put forward the effects of PA on the students’ learning outcomes. Questions concerning the validity and reliability of PA and PA competences are also addressed by different researchers. This qualitative study is part of a wider project that seeks to develop and test evaluation and assessment strategies in online contexts. In a doctoral module, PA was used for summative and formative purposes. Formative PA aimed to give feedback about the ongoing group work, but also to increase online interaction between the different groups of students. The main module task was to write a literature review, about a selected topic, using a wiki. Criteria and indicators to assess the literature review were negotiated with the students. Different criteria were used to assess the quality of PA, such as, the use of the negotiated criteria, the adequacy of the chosen vocabulary or the provision of constructive feedback. The results show that overall the quality of PA can be improved. Groups did not provide sufficient criticism, questions and suggestions for improvement.     

Uncertainty analysis for peer assessment: oral presentation skills for final year project

Peer assessment plays an important role in engineering education for an active involvement in the assessment process, developing autonomy, enhancing reflection, and understanding of how to achieve the learning outcomes. Peer assessment uncertainty for oral presentation skills as part of the FYP assessment is studied. Validity and reliability for complex assessments were conceptualised in the context of quantification, hierarchical system of uncertainties, aleatory uncertainty, and epistemic uncertainty. Systems as assessment methods were also conceptualised to define elements (i.e. examiner, examinee, and difficulty) as variables or/and constants, and to study the uncertainty behaviour of the peer assessment within the framework of the parametric uncertainty model. As a result, a singularity, at which the hierarchical structure of uncertainties breaks down, and its related behaviour were theoretically predicted and experimentally verified. Also, a validity paradox, when the ‘difficulty’ level is sufficiently lowered, was found to be possible and verified for oral presentation skills. It may be useful for understanding fundamental characteristics of the peer assessment in which epistemic uncertainties are dependent on how the assessment is close to the singularity point.     

Effects of integrating mobile technology-assisted peer assessment into flipped learning on students’ dance skills and self-efficacy

ABSTRACT

Dance courses aim to not only intensify learners’ dance skills, but also cultivate their personal dancing features, self-confidence, self-reflection ability, creative power, appreciation ability, to name just a few, in order to attain much higher level artistic performance. In the present study, an approach which integrates mobile peer assessment into flipped learning is proposed. Moreover, a 9-week experiment was conducted to explore the effects of the approach on students’ dance skills, self-efficacy, and learning satisfaction. The results indicated that the students learning with the integrated mobile peer assessment and flipped learning approach had better dance skills than those learning with the conventional flipped learning approach and traditional instruction. In terms of self-efficacy, the students learning with the conventional flipped learning approach outperformed those learning with the proposed approach and traditional instruction. In terms of learning satisfaction, the conventional flipped learning approach outperformed the traditional instruction. Discussion and suggestions are provided accordingly. It is suggested that teachers who want to incorporate flipped learning into dance courses may decide whether to combine peer assessment according to the teaching objectives.     

高职朋辈心理辅导员团体心理咨询技能培养的策略研究

朋辈心理辅导已成为高职院校心理健康教育过程中不可或缺的重要形式。加强朋辈心理辅导员的培训与指导,探索有效途径,提高其操作技能,发挥团体心理咨询的功能,具有重要意义。     

The role of anonymity in peer assessment

This quasi-experimental study aimed to examine the impact of anonymity and training (an alternative strategy when anonymity was unattainable) on students’ performance and perceptions in formative peer assessment. The training in this study focused on educating students to understand and appreciate formative peer assessment. A sample of 77 students participated in a peer assessment activity in three conditions: a group with participants’ identities revealed (Identity Group), a group with anonymity provided (Anonymity Group) and a group with identities revealed but training provided (Training Group). Data analysis indicated that both the Anonymity Group and Training Group outperformed the Identity Group on projects. In terms of perceptions, however, the Training Group appreciated the value of peer assessment more and experienced less pressure in the process than the other two groups.     

Student perceptions of peer assessment: an interdisciplinary study

Peer assessment provides students with multiple benefits during their learning process. The aim of our study is to examine students’ perception of peer assessment. Questionnaires were administered before and after the peer-assessment process to 416 students studying 11 different subjects in four different fields taught at the University of Girona. Results suggest that students have a positive predisposition towards this methodology, both before and after its implementation. Students perceive it as a both motivating and recommended methodology that facilitates the acquisition of learning at different levels. As for its limitations, students highlight the responsibility that comes with it and a certain amount of distrust in fellow students’ abilities to peer-assess.     

Peer assessment as a method of improving student engagement

To encourage increased student attendance and engagement in a third-year economics unit, the curriculum was redesigned to incorporate continuous assessment throughout the semester. A component of group project marks were allocated to peer assessment, in an attempt to address concerns about free-riding colleagues sharing a common mark. This study investigated the consistency of marks awarded to peers within teams, and the acceptance by students of marks awarded by peers. Students were asked to provide ratings and explanatory comments for each of their group peers. Focus groups were conducted to determine students’ acceptance of this strategy. Eighty student ratings were compared to determine consistency of assessment. Within groups, students who received higher marks from their peers generally awarded marks to their peers across a wider range, whereas students who received lower average grades often awarded the same mark to all team members. These results might indicate that students who were attending class regularly and/or contributing at a higher level were more discriminating in the marks awarded to their peers. Similarly, non-contributors (as identified by their peers) assigned the same or similar grades to each of their peers, possibly due to a lack of knowledge about their peers’ contributions.     

Peer assessment as a learning tool for enhancing student teachers' preparation

This study investigates how peer assessment contributes to enhancing student teachers' preparation during field experience. A semi-structured interview was conducted with 72 student teachers. The findings showed that the student teachers have positive beliefs about peer assessment. They think that it can be beneficial if some changes are made in the way it is employed in teacher education programs. In light of the study findings, the researchers put forth a number of suggestions and recommendations for employing peer assessment in teacher education programs in Jordan.     

Aiming for autonomy: formative peer assessment in a final-year undergraduate course

This article takes a dual focus on the theme of student formative peer assessment. On the one hand it offers a thorough literature review in this field, while on the other it unpacks a case study of curriculum design where peer assessment has been adopted. The practical example draws on recent changes made to a third- and final-year undergraduate research dissertation course in a UK architectural school. Although peer assessment worked quite well in this small setting, similar findings might be uncovered when scaled up to larger cohorts, as well as other disciplines and year groups. The research findings are certainly relevant for other contexts. Qualitative research from a focus group with six students on the course informs the empirical body of the paper. Through engaging with students’ reflections, as well as the relevant literature, and reflexively looking at the curriculum changes, this paper discusses some of the benefits and drawbacks that can arise from peer assessment in this context. Peer assessment is not without its challenges and hiccups, but, despite these, the benefit to autonomous critical thinking cannot be understated. And arguably peer assessment is especially germane for final-year undergraduates as they embark on employment or more advanced studies.