Assessment training effects on student assessment skills and task performance in a technology-facilitated peer assessment

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the training, students reviewed learning concepts, discussed marking criteria, graded example projects and compared their evaluations with the instructor’s evaluation. Data were collected in the form of initial and final versions of students’ projects, students’ scoring of example projects before and after the assessment training, and written feedback that students provided on peer projects. Results of data analysis indicate that the assessment training led to a significant decrease in the discrepancy between student ratings and instructor rating of example projects. In addition, the degree of student vs. instructor discrepancy was highly predictive of the quality of feedback that students provided to their peers and the effectiveness of revisions that they made to their own projects upon receiving peer feedback. Smaller discrepancies in ratings were associated with provision of higher quality peer feedback during peer assessment, as well as better revision of initial projects after peer assessment.     

Bias in student evaluations: Are my peers out to get me?

The use of peer evaluations to evaluate performance is commonplace, especially in higher education. Yet, researchers and educators have long expressed concerns about the accuracy of such ratings. While research has found student peer evaluations to be consistent with instructor evaluations, rater bias is inherent in all evaluations, and students are not exempt from this. This study examines task completion and identity threat to determine if experience and identity play a role in shaping student peer evaluations. A sample of MBA students evaluated their peers before and after completing a course presentation. Results showed that students’ peer evaluations were significantly “lower” or more critical after having completed the presentation themselves. This study illuminates the importance of experience and training in the implementation of performance evaluations.     

Learning to lead: Self‐ and peer evaluation of team leaders in the human structure didactic block

Increasing emphasis on leadership in medical education has created a need for developing accurate evaluations of team leaders. Our study aimed to compare the accuracy of self‐ and peer evaluation of student leaders in the first‐year Human Structure block (integrated gross anatomy, embryology, and radiology). Forty‐nine first‐year medical students at Mayo Medical School were assigned to learning teams of three or four members. Teams worked together on daily laboratory dissection, clinical projects, embryology presentations, and daily group quizzes. Student team leaders were responsible for leading laboratory dissection, reviewing radiographic findings, and organizing group assignments. Weekly electronic surveys were administered to evaluate team leaders on altruism, compassion, respect, integrity, responsibility, commitment to excellence, and self‐reflection. Results demonstrated that team leaders rated themselves lower than their peers rated them in multiple aspects of leadership. Peer evaluation of team leaders was statistically higher than self‐evaluation in all traits measured except respect. Female leaders were rated higher by their peers in the areas of responsibility and self‐reflection compared to male leaders. This study demonstrates the need for increased communication between team leaders and members, along with creation of a mutually respectful environment, to improve leader awareness of their abilities and foster team success. Anat Sci Educ 2:210–217, 2009. © 2009 American Association of Anatomists.     

How does student peer review influence perceptions,engagement and academic outcomes? A case study

Involving students in peer review has many pedagogical benefits, but few studies have explicitly investigated relationships between the content of peer reviews, student perceptions and assessment outcomes. We conducted a case study of peer review within a third-year undergraduate subject at a research-intensive Australian university, in which we examined: (1) students’ perceptions of the peer review process before and after peer review, (2) content of the peer reviews and what kinds of feedback were adopted and (3) the effect of participation in peer review on performance (grades) in the assessment task. Students overwhelmingly perceived peer review to be beneficial, and the opportunity to participate in peer review resulted in a significant improvement in the quality of work submitted for assessment. Students who benefited most from peer review were those of below-median performance, and the magnitude of benefit was related to the degree to which students engaged with the peer review process. Our study confirms that participation in peer review can lead to important improvements in performance and learning outcomes.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

Students’ peer relationships and their contribution to university adjustment: the need to belong in the university community

Whilst new friendships and an active social life are commonly discussed features of ‘being a student’, there is limited empirical research that has quantitatively studied the contribution that social factors play in students’ university experience. Research that has been conducted shows that belonging and social integration are important factors in successful transition to university, and subsequent retention. This article presents research into students’ social relationships at university, their attachment to the university, and how these elements link to university adjustment. Undergraduates (N = 135) completed questionnaires measuring their attachment to university peers, attachment to the university, experiences of problematic peer relationships and quality of adjustment to university life. Students who reported strong attachment to their peers also demonstrated higher levels of adjustment to university life and attachment to their university. Students who reported difficulties in their relationships with other students had lower levels of peer attachment and university adjustment. Attachment to university peers was the strongest predictor of university adjustment, followed by attachment to the university. The research highlights the role of social relationships in institutional belonging, and the importance of nurturing peer relationships and institutional affiliation to create a positive student experience.     

The impact of an elaborated assessee’s role in peer assessment

The purpose of this study was to investigate the effects of an elaborated assessee’s role on metacognitive awareness, performance and attitude in peer assessment. Two intact groups (a total of 82 students) were randomly assigned to a treatment condition (having back‐feedback activity) or a control condition (not having back‐feedback activity). The results indicated that, regarding metacognitive awareness, when participants played the elaborated assessee role, they tended to report higher metacognitive awareness in their learning process. Regarding performance, when participants played the elaborated assessee role, they tended to receive better scores in making a concept map. Regarding attitude, when participants played the elaborated assessee role, they reported greater motivation towards the peer assessment. The findings of this study suggest instructional implications for those who want to employ peer assessment as a learning method by showing the effectiveness of a well‐developed role design, specifically one that includes the back‐feedback activity.     

University Students’ Perceptions of and Attitudes Toward (Online) Peer Assessment

The use of peer assessment (PA) as an alternative form of evaluation method is reported to be helpful in learning and is increasingly being adopted in higher education settings. Through collecting data from 280 university students in Taiwan, a 20-item instrument was developed to investigate university students’ attitudes toward and perceptions of PA and online PA. Four subscales, Positive Attitudes, Online Attitudes, Understanding-and-Action, and Negative Attitudes, were extracted and reliabilities were calculated. Results revealed that participating students held positive attitudes toward the use of PA activities, but they viewed online PA as a technical tool to facilitate assessment processes, rather than as a learning aid. Male students had more positive attitudes toward PA than females did, and students with previous PA experiences had less negative attitudes toward PA. A majority of these students suggested that the PA score should be counted as a small part of the total course grade, and there was an effect of the perceived importance of PA score on students’ attitudes toward these four subscales.     

Dancing with the devil: formative peer assessment and academic performance

Peer assessment can be important in developing active and independent learners, as well as providing more and faster feedback in large classes, compared to marking done by tutors. In addition, the evaluative, critical stance required by students in order to assess their peers' work encourages the development of higher-order cognitive skills. Changing roles from being assessed to being an assessor can also improve students' ability to judge and improve on their own work. However, peer assessment does have potential problems and there is some debate as to the appropriate academic level at which to implement it, the kinds of feedback that are given and the ways in which students respond. In addition, there is little evidence that peer assessment has an impact on academic performance. This research reports the results of an online peer assessment exercise for a macroeconomics essay conducted in a large Economics 1 class at Rhodes University. Of the 800 students, about half participated in the peer assessment exercise. Data were collected from students via a formal course evaluation. In addition, a sample of 50 essays was evaluated in terms of the relationship between peer marks and final (tutor) marks received and the impact that peer assessment had on the quality of the final essay submitted. An Ordinary Least Squares regression was used to investigate the impact of peer assessment participation on marks. Results showed that peer marks tended to ‘bunch’ in the 60–68% range, indicating the reluctance of peers to give very high or low marks. In general, peers gave more useful feedback on technical aspects, such as presentation and referencing (which were also the categories in which students most often made improvements), than on content. Regression analysis showed that peer assessment participation was not a significant determinant of final essay mark, but that economics ability and English language proficiency were.     

‘Observation has set in’: comparing students and peers as reviewers of teaching

Peer review is a powerful method to enhance teaching in higher education. Peers, however, may not be the most relevant people in evaluating teaching success; as the most important stakeholders in learning, students’ evaluations need to be heard. Whilst some efforts to capture ‘the student voice’ are simplistic and may foster consumerist approaches, adopting ‘radical collegiality’ towards students may provide the benefits of peer review whilst avoiding some of its disadvantages. Here we describe the Students as Colleagues project, which trained student volunteers as evaluators of teaching. To assess the ability of students to provide useful reviews, we compared their evaluative feedback with that from academic peers, using a paired design and qualitative and quantitative data. Students gave significantly more positive comments, and just as many negative and directive comments, as academic peers. Student colleagues emphasised the positive personal (rather than professional) capacities of their reviewees, encouraged expressed vulnerability and drew on their broad experiences as students rather than from professional perspectives. Participation changed how students saw their abilities and helped ‘humanise’ both the reviewees and the university as a whole. Our results and standpoint theory suggest that students’ evaluative feedback is the most valuable perspective to inform teaching enhancement.     

Assessment for learning: a catalyst for student self-regulation

The development of students as lifelong, self-regulating learners is a valued outcome of higher education. To this end, the current project investigated how students in an undergraduate course experienced and responded to a teaching–learning environment where they were expected to take responsibility for their learning. The pedagogic environment of this course was grounded in strategies and activities associated with assessment for learning (AfL). As such, students were expected to participate in activities that provided them with opportunities to exercise control over their learning. Data gathering comprised individual semi-structured interviews and the collection of artefacts. Findings indicated goals helped students know where they were going; exemplars provided insights into what was expected and what constituted quality work; course activities elicited evidence of learning; dialogic interactions around these generated feedback about understandings and progress; the evaluation of exemplars developed evaluative knowledge, skill and expertise; peer review and feedback provided an authentic context for evaluation and monitoring of works-in-progress. It was concluded that, while each of the AfL strategies contributed to student self-regulation, the full impact of AfL as a catalyst for self-regulated learning was realised in the cumulative and recursive effect these strategies had on students’ thinking, actions and feelings.     

Peer assessment as a method of improving student engagement

To encourage increased student attendance and engagement in a third-year economics unit, the curriculum was redesigned to incorporate continuous assessment throughout the semester. A component of group project marks were allocated to peer assessment, in an attempt to address concerns about free-riding colleagues sharing a common mark. This study investigated the consistency of marks awarded to peers within teams, and the acceptance by students of marks awarded by peers. Students were asked to provide ratings and explanatory comments for each of their group peers. Focus groups were conducted to determine students’ acceptance of this strategy. Eighty student ratings were compared to determine consistency of assessment. Within groups, students who received higher marks from their peers generally awarded marks to their peers across a wider range, whereas students who received lower average grades often awarded the same mark to all team members. These results might indicate that students who were attending class regularly and/or contributing at a higher level were more discriminating in the marks awarded to their peers. Similarly, non-contributors (as identified by their peers) assigned the same or similar grades to each of their peers, possibly due to a lack of knowledge about their peers’ contributions.     

Homophily Among Peer Groups Members’ Perceived Self-Regulated Learning

The present study is one of the first examining whether peer group members hold similar levels of self-regulated learning. The study specifically addresses the potential homophily among group members’ regulative abilities (metacognition, environment regulation, effort regulation, peer learning, and help seeking) and whether group members’ regulative abilities predict affiliates’ academic performance. The study surveyed 9th-grade students from a Midwestern high school about their regulative abilities for mathematics. Results suggest that peer groups members’ effort regulation is similar among peer affiliates but not other regulative abilities. In addition, peer group members’ regulative abilities do not predict each others’ academic performance.     

Implementation of a peer‐led team learning instructional approach in an undergraduate organic chemistry course

This study focuses on the implementation of a peer‐led team learning (PLTL) instructional approach for all students in an undergraduate organic chemistry course and the evaluation of student outcomes over 8 years. Students who experienced the student‐centered instruction and worked in small groups facilitated by a peer leader (treatment) in 1996–1999 were compared with students who experienced the traditional recitation section (control) in 1992–1994. Quantitative and qualitative data show statistically significant improvements in student performance, retention, and attitudes about the course. These findings suggest that using undergraduate leaders to implement a peer‐led team learning model that is built on a social constructivist foundation is a workable mechanism for effecting change in undergraduate science courses. © 2002 Wiley Periodicals, Inc. J Res Sci Teach 39: 606–632, 2002     

The impact of formative peer feedback on higher education students’ academic writing: a Meta-Analysis

Peer feedback is frequently implemented with academic writing tasks in higher education. However, a quantitative synthesis is still lacking for the impact that peer feedback has on students’ writing performance. The current study conveyed two types of observations. First, regarding the impact of peer feedback on writing performance, this study synthesized the results of 24 quantitative studies reporting on higher education students’ academic writing performance after peer feedback. Engagement in peer feedback resulted in larger writing improvements compared to (no-feedback) controls (g?=?0.91 [0.41, 1.42]) and compared to self-assessment (g?=?0.33 [0.01, 0.64]). Peer feedback and teacher feedback resulted in similar writing improvements (g?=?0.46 [-0.44, 1.36]). The nature of the peer feedback significantly moderated the impact that peer feedback had on students’ writing improvement, whereas only a theoretically plausible, though non-significant moderating pattern was found for the number of peers that students engaged with. Second, this study shows that the number of well-controlled studies into the effects of peer feedback on writing is still low, indicating the need for more quantitative, methodologically sound research in this field. Findings and implications are discussed both for higher education teaching practice and future research approaches and directions.     

Role of peers in student academic achievement in exogenously formed university groups

We estimate the influence of classmates’ ability characteristics on student achievement in exogenously formed university student groups. The study uses administrative data on undergraduate students at a large selective university in Russia. The presence of high-ability classmates has a significant positive effect on individual grades in key economics and mathematics courses as well as on overall academic performance. While a simple linear-in-means model reveals moderate peer effects, non-linear specifications give strong evidence that students at the top of the ability distribution derive the greatest benefit from high-ability classmates. Less able students are not affected by peers and have no significant influence on peers’ outcomes.     

Oral presentations in higher education: a comparison of the impact of peer and teacher feedback

In this paper the authors analyse the competency of a sample of economics and business students in orally presenting academic content, as well as the changes that occurred after receiving feedback. The students’ presentations were videotaped, and a total of 96 were analysed at three stages: pre-test, post-test and follow-up. The participants were divided into two groups. In one, each student received feedback from a peer as part of a peer assessment with rubric activity. In the other, they received feedback from the teacher immediately after their presentation. In the post-test, the peer assessment with rubric students improved by 10% in the valuation of their presentation, while the teacher feedback students only improved by 5%. These results support the idea that undergraduates’ evaluations of their peers can be effective in improving oral presentation skills, especially when they are provided with some support instruments (videos and rubrics). However, the improvements in the peer assessment with rubric group were not maintained in the follow-up re-test. This suggests that a single session of peer assessment with rubric is insufficient to generalise any improvements in the said competency. The implications of these results with regard to the implementation of methods of formative peer assessment in higher education are discussed.     

Meta-assessment in a project-based systems engineering course

Project-based learning (PBL) facilitates significant learning, but it poses a major assessment challenge for assessing individual content knowledge. We developed and implemented an assessment approach and tool for a mandatory undergraduate systems engineering PBL-based course. We call this type of assessment student-oriented meta-assessment. Research participants included 131 undergraduate engineering students who carried out team projects requiring conceptual modelling of complex systems. Next, individual students assessed their peer team projects. Finally, the course staff assessed students individually, based on the content knowledge that was reflected in students’ comments to their peers, by using the new meta-assessment tool. The research findings validated the meta-assessment tool which can serve for assessing various PBL courses. Our contribution is twofold: a new approach for assessing project-based undergraduate engineering courses, and classification of meta-assessment methods into three types, where a student-oriented meta-assessment as one of the three meta-assessment methods is our newly introduced and validated method.     

Peer facilitated writing groups: a programmatic approach to doctoral student writing

Very few empirical studies have investigated programmes in which doctoral students act as peer facilitators in faculty writing groups. We report on the development of a centrally delivered doctoral student writing programme in which twenty student participants were mentored and provided with the resources to initiate their own faculty-based doctoral writing groups. ‘Legitimate peripheral participation’ was used as a conceptual lens to interpret the data collected during the establishment and evaluation of the programme. All student participants in the preparatory training course, which was developed in consultation with postgraduate students and research supervisors, went on to become doctoral writing peer facilitators of peer writing groups. Insights from seven of these showed how a well-structured and supportive programme harnessed the benefits of peer learning by bringing personal rewards to participants and building institutional capacity around doctoral research writing literacies.     

Peer‐mediated instruction: assisted performance in the primary classroom

Peer collaboration has been supported in research as an effective instructional strategy. However, education researchers lack a full understanding of the types of scaffolding that take place in peer groups. Little research is available that documents what happens during the small group interactions. This case study took place in one teacher’s multi‐age primary classroom in the southeastern region of the USA. It examined how elementary students provided scaffolding to one another during collaborative classroom activities. Specifically, this investigation explored the ways in which elementary students provide assistance to one another during academic tasks and examined the status of the children, in terms of more or less capabilities, during the collaborative events. Vygotsky asserts that for learning to occur, assistance must be provided by a more capable person. The research findings suggest that peers provide scaffolding for one another in various ways. In this investigation, these primary students utilized questioning, providing feedback, and instructing as the primary methods of scaffolding during the learning experiences. This study also supports assertions that when students are allowed to pose questions and provide feedback to one another during peer collaboration, they help to establish instruction in their zone of proximal development.