When peers agree,do students listen? The central role of feedback quality and feedback frequency in determining uptake of feedback

Prior research on the complex process of revision based upon peer feedback has focused on characteristics of each piece of feedback in isolation. Multipeer feedback allows for feedback to be repeated (or not), which could be a signal of feedback quality or be especially persuasive to peers. Separately, little research has examined how well peers select more impactful and accurate peer feedback in their revisions, whether repeated or not. We analyzed almost 2,000 peer comments received by 107 students in a secondary writing course in the US to determine whether feedback quality and feedback frequency predicted feedback implementation. Controlling for other feedback features and context factors, students were much more likely to implement feedback as both feedback quality and feedback frequency increased, surprisingly with no interaction (i.e., even low-quality comments were more likely to be implemented when repeated). However, low-quality comments often partially overlapped with high-quality comments, providing a potential explanation for the lack of an interaction. Finally, consideration of feedback frequency and feedback quality provides new insights into which feedback features are actually related to implementation. The results generally allay concerns about the blind-leading-the-blind in peer feedback as well as pushing for peer feedback arrangements that produce more overlapping comments.     

Investigating the impact of automated feedback on students’ scientific argumentation

ABSTRACT

This study investigates the role of automated scoring and feedback in supporting students’ construction of written scientific arguments while learning about factors that affect climate change in the classroom. The automated scoring and feedback technology was integrated into an online module. Students’ written scientific argumentation occurred when they responded to structured argumentation prompts. After submitting the open-ended responses, students received scores generated by a scoring engine and written feedback associated with the scores in real-time. Using the log data that recorded argumentation scores as well as argument submission and revisions activities, we answer three research questions. First, how students behaved after receiving the feedback; second, whether and how students’ revisions improved their argumentation scores; and third, did item difficulties shift with the availability of the automated feedback. Results showed that the majority of students (77%) made revisions after receiving the feedback, and students with higher initial scores were more likely to revise their responses. Students who revised had significantly higher final scores than those who did not, and each revision was associated with an average increase of 0.55 on the final scores. Analysis on item difficulty shifts showed that written scientific argumentation became easier after students used the automated feedback.     

The alignment of written peer feedback with draft problems and its impact on revision in peer assessment

Despite the wide use of peer assessment, questions about the helpfulness of peer feedback are frequently raised. In particular, it is unknown whether, how and to what extent peer feedback can help solve problems in initial texts in complex writing tasks. We investigated this research gap by focusing on the case of writing literature reviews in an academic writing course. The dataset includes two drafts from 21 students, sampled to represent a wide range of document qualities, and 84 anonymous peer reviews, involving 1,289 idea units. Our study revealed that: (1) at both substance and high prose levels, drafts of all quality levels demonstrated more common problems on advanced writing issues (e.g. counter-argument); (2) peer feedback was driven by difficulty of the problem rather than overall draft quality, peer comments were not well aligned with the relative frequency of problems, more comments were given to less difficult problems; (3) peer feedback had a moderate impact on revision, and importantly, receiving multiple comments on the same issue led to more repairs and improvement of draft quality, but consistent with the comments received, authors tended to fix basic problems more often. Implications for practice and research are drawn from these findings.     

The effect of assessment of process after receiving teacher feedback

This exploratory study investigated the effect of assessing both process and product compared to assessing written products alone. Two groups of students received teacher feedback over a one-year period. One group was assessed on their revisions in addition to the quality of final drafts, while a second group was assessed on the quality of final drafts alone. The uptake of feedback, in the form of revision attempts, and the success of those attempts were compared between the two groups using a Mann–Whitney U-test. In addition, improvement in writing performance was measured and compared between the two groups using one-way analysis of variance. It was found that the group assessed on process had significantly higher uptake of feedback than the group assessed on product alone. However, the increased uptake consisted of unsuccessful revision attempts, while the number of successful revisions was almost identical between the groups. It was also found that the group assessed on final product alone improved significantly more than the group assessed on process in terms of essay content. It is concluded that the assessment of feedback may not be necessary in some contexts, and suggestions are given for increasing the quality of learning achieved by teacher feedback.     

Passive,active, and constructive engagement with peer feedback: A revised model of learning from peer feedback

To deepen understanding of learning through peer feedback, the current study investigated the relationships between different peer feedback activities (organized into constructive vs active activities) and learning (i.e., transfer to new tasks), examining the nature of activities within provided feedback, received feedback, and revisions in response to feedback. Across five US high schools, 367 students in Advanced Placement classes participated, implementing common assignments and peer assessment rubrics. Provided/received comments and revisions in one assignment, and writing improvements observed in a second assignment were exhaustively coded and subjected to hierarchical model regression analyses. Results showed that constructive activities (providing explanations and making revisions after receiving explanations or providing suggestions) were consistently associated with learning, whereas passive (e.g., receiving feedback without making revisions) or active activities (e.g., implementing specific suggestions) were not. Further, the effects of received feedback on learning were mediated by the number of revisions. Theoretical and practical implications of the findings are discussed.     

The influence of social interaction on meaning construction in a virtual community

As proposed by social constructive theorists, meaningful learning and individual development were achieved through social interaction. To foster social interaction among students, this study formed an online learning community in which they played multiple roles as writers, editors and commentators. In playing different roles, they read peers' texts, edited peers' errors, evaluated peer editors' corrections and finally reconstructed their own texts. Results of this study showed that the multiple roles they played allowed them to have opportunities to view their own texts from others' perspectives. Based on these perspectives, they were more willing to acquire information from and contribute information to peers. All of this extensive information acquisition and contribution resulted in meaning construction of texts as active students improved their final drafts in both local revision (grammatical correction) and global revision (the style, organisation and development of a text) after receiving and evaluating feedback from peer editors. Their final drafts were very different from those of passive students whose first and final drafts were almost the same despite some grammatical revisions. This study suggests that, rather than relying only on an examination of students' final drafts, there may be benefits in teachers encouraging students to actively participate in social interaction by reading peers' texts, editing peers' errors and evaluating peer editors' corrections during text revisions.     

Feedback modes matter: Comparing student perceptions of digital and non-digital feedback modes in higher education

Assessment feedback is increasingly being provided in digital modes, from electronic annotations to digital recordings. Digitally recorded feedback is generally considered to be more detailed than text-based feedback. However, few studies have compared digital recordings with other common feedback modes, including non-digital forms such as face-to-face conversations. It is also unclear whether providing multiple feedback modes is better than a single mode. To explore these possibilities, an online survey asked 4514 Australian university students to rate the level of detail, personalisation and usability of the feedback comments they had most recently received. Of the students who received a single feedback mode only, electronic annotations and digital recordings were rated most highly on the three quality indicators. Students who received multiple modes were more likely to agree with all three indicators than those who received a single mode. Finally, students who received multiple modes were more likely to agree that the comments were detailed and usable when one of those modes was a digital recording. These findings enhance our understanding of feedback design, indicating that it is important to consider the strengths and weaknesses of particular modes, and the value of offering multiple modes.     

The effects of skill diversity on commenting and revisions

The use of peer assessment to evaluate students’ writing is one recommended method that makes writing assignments possible in large content classes (i.e., more than 75 students). However, many instructors and students worry about whether students of all ability levels are capable of helping their peers. We examine how ability pairing (e.g., high-ability student with high-ability student versus high-ability student with low-ability student) changes key characteristics of feedback to determine which pairings are likely to benefit students most. A web-based reciprocal peer-review system was used to facilitate the peer review of students’ writing of two papers. Over 1,100 comments given to writers from their peers were coded for several relevant categories: type of feedback, type of criticism, focus of problem, focus of solution, and implementation. Overall, creating peer-review groups such that students receive feedback from someone of a dissimilar ability appeared to be most beneficial. High-ability writers received similar kinds of feedback from high-ability versus low-ability peers. By contrast, the low-ability writers received more comments that identified problems focusing on substance issues from high-ability reviewers. In addition, the low-ability writers implemented a higher percentage of the comments from the high-ability reviewers.     

Assessment training effects on student assessment skills and task performance in a technology-facilitated peer assessment

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the training, students reviewed learning concepts, discussed marking criteria, graded example projects and compared their evaluations with the instructor’s evaluation. Data were collected in the form of initial and final versions of students’ projects, students’ scoring of example projects before and after the assessment training, and written feedback that students provided on peer projects. Results of data analysis indicate that the assessment training led to a significant decrease in the discrepancy between student ratings and instructor rating of example projects. In addition, the degree of student vs. instructor discrepancy was highly predictive of the quality of feedback that students provided to their peers and the effectiveness of revisions that they made to their own projects upon receiving peer feedback. Smaller discrepancies in ratings were associated with provision of higher quality peer feedback during peer assessment, as well as better revision of initial projects after peer assessment.     

Test Interpretation Competence: A Comparison of Microskills and Mental Practice Training

To study the effects of behavioral versus mental rehearsal in acquiring test interpretation skills, we randomly assigned students in a counseling-related testing course to one of two training groups or to a delayed-control (DC) group. The treatment groups received identical instructions, modeling, feedback, and review of target skills but practiced in either a microskills (MS) or mental practice (MP) mode. Ratings of videotaped interviews of the MS and MP trainees did not differ but were superior to the DC group's ratings on measures of general counseling and test interpretation skills. Those in the MS group had more confidence in and perceived greater benefits from their training than did those in the MP group. Implications are discussed, and a suggestion is made for combining MS and MP training methods.     

Peer-feedback and revision process in a wiki mediated collaborative writing

This study investigated the use of a wiki for collaborative writing among primary levels five (P5) and six (P6) students (n = 119) in a Chinese primary school in Hong Kong where English is taught as a second language (L2). Three classes of students and their English subject teachers participated in a three-month English language writing programme using a wiki. Quantitative and qualitative data were analyzed from activities recorded in the wiki system, including posted edits and comments, students’ group writings and student and teacher interviews. The wiki page history revealed information on the types of revisions that occurred, showing that different types of feedback elicited actual revisions, which may have resulted in better group writing. Findings from the study may shed light on how wikis can help provide support for students’ collaborative writing process with wikis, and how peer-feedback can influence this process.     

The bilateral benefits of providing and receiving peer feedback in academic writing across varying L2 proficiency

Although the effectiveness of peer review has been examined, few have tested the joint benefits of providing and receiving feedback features and quality in L2 contexts. The present study investigated variation in key features and quality of feedback provided and received by high and low L2 proficiency students and its benefits on revision in the authentic setting of students experiencing both roles of providing and receiving peer feedback in English academic writing. Analyses on two drafts from 50 students, 1356 idea units of anonymous implementable peer feedback and back-evaluation ratings revealed that the combined effect of providing and receiving feedback on revision was prominent for all students. But the bilateral benefits of providing and receiving feedback features were more significant for high proficiency students, while providing feedback features and quality benefited low proficiency students more. The study implies that students need more bilateral training with both providing and receiving feedback.     

From feedback to revisions: Effects of feedback features and perceptions

Although the effects of peer feedback have been studied from a number of perspectives, much remains to be learned about what leads students to act (or not) on their peers’ feedback in revisions. The present study examined the relationship between peer feedback features, student perceptions as potential mediators (understanding versus agreement with the feedback), and the likelihood of students’ implementation of the feedback. Peer feedback, back-evaluation comments, and revisions from 185 US high school students were analyzed. Investigated feedback features included four cognitive features (identification, explanation, solution, suggestion), and two affective features (mitigating praise, hedges). Logistic regression analyses revealed that: (1) both understanding and agreement with feedback predicted implementation; (2) presence of solutions predicted understanding of feedback; (3) mitigating praise predicted agreement of the problem; and (4) explanation and hedges predicted implementation separately from perception effects. Theoretical and practical implications of the findings are discussed.     

The effects of a competency‐oriented learning environment and tutor feedback on students’ reflection skills

This study investigates the effects of a competency‐oriented course and the effects of feedback training on students’ reflection skills. Thirty‐one nursing students enrolled in a conventional course with lectures and assignments following a traditional test. Subsequently, they enrolled in a competency‐oriented course, which included more performance‐oriented tasks based on competencies and a performance‐based assessment. In both courses, half of the students received feedback on their assignments from tutors who were trained in feedback skills (experimental group), while the other half of the students received feedback from non‐trained tutors (control group). After each course, all students wrote a reflection report. Results show that students’ reflection reports after the competency‐oriented course were of a higher quality than the reports after the conventional course. Contrary to our expectations, the quality of the reflection reports of the control group (students supervised by non‐trained tutors) was significantly higher than the quality of reflection reports of the experimental group (students supervised by trained tutors). This study shows that a competency‐oriented design of courses enhances students’ reflection skills, but that the training in feedback skills needs to be reconsidered.     

Students' Early Mathematical Representation Knowledge: The Effects of Emphasizing Single or Multiple Perspectives of the Rational Number Domain in Problem Solving

This study examined changes in 26 fourth-grade students' early conceptions of rational number representations as a function of receiving one of two curricular interventions. The first group of 12 students received a curriculum that emphasized constructing knowledge through extended problem solving with a single perspective of the rational number domain based on part-whole relations. A second group of 14 students received a curriculum that emphasized a more conceptually diverse multiple perspective view of the domain through problem solving with operator and ratio relations. Analyses of the students' rational number knowledge before and after the interventions indicated that students in the single perspective group produced organizations of knowledge that more frequently diverged from a formal domain analysis than those produced by students in the multiple perspective group. Further, students in the single perspective group increased their focus on superficial surface features. Alternatively, students in the multiple perspective group demonstrated an increased focus on operations that more frequently reflected the underlying mathematical relation conveyed by the representation. The findings indicate that an early exposure to more diverse perspectives of rational numbers assists students in developing more interconnected and viable representation knowledge for rational numbers.     

Characterisation of pre-service teachers' attitude to feedback in a wiki-environment framework

This is a two-phase research process in a wiki environment. The first phase explored (1) the significance of peer-feedback on students’ academic performance on a specific task in a wiki environment, (2) three types of peer-feedback and (3) students’ constraints on offering meaningful feedback to their peers. The objective of phase two was to determine the reasons why students refrain from peer-feedback. Significant correlation exists between the willingness to give feedback and the willingness to receive feedback. Significant correlation also exists between the difficulty of giving feedback and the difficulty of receiving feedback. Female students tend to consider feedback as the teacher's responsibility more than male students. Religious beliefs did not have any significant effect on any of the parameters tested. The results indicate that traditional students tend to be more conservative regarding feedback. There are worrying implications to pre-service teachers’ refraining from giving feedback.     

Oral presentations in higher education: a comparison of the impact of peer and teacher feedback

In this paper the authors analyse the competency of a sample of economics and business students in orally presenting academic content, as well as the changes that occurred after receiving feedback. The students’ presentations were videotaped, and a total of 96 were analysed at three stages: pre-test, post-test and follow-up. The participants were divided into two groups. In one, each student received feedback from a peer as part of a peer assessment with rubric activity. In the other, they received feedback from the teacher immediately after their presentation. In the post-test, the peer assessment with rubric students improved by 10% in the valuation of their presentation, while the teacher feedback students only improved by 5%. These results support the idea that undergraduates’ evaluations of their peers can be effective in improving oral presentation skills, especially when they are provided with some support instruments (videos and rubrics). However, the improvements in the peer assessment with rubric group were not maintained in the follow-up re-test. This suggests that a single session of peer assessment with rubric is insufficient to generalise any improvements in the said competency. The implications of these results with regard to the implementation of methods of formative peer assessment in higher education are discussed.     

The Effects of Modeling,Feedback, and Experiential Methods on Counselor Empathy

This study compares the effects of three supervisory techniques—modeling, feedback, and experiential intervention—in teaching counseling students to communicate empathic understanding. The students participated in three simulated counseling interviews during which their responses to standard clients were rated on the degree of empathic understanding conveyed. After each of the first two interviews students in the experimental groups were given one of the three supervisory treatments; students in the control group received no supervision. The ratings indicated that the modeling method of supervision was more effective than the feedback method and the feedback method was more effective than the experiential method in helping the counselors communicate empathy. No significant differences were found between students receiving experiential supervision and those in the control group. The interaction between method and sex was significant in that male students responded better than female students to the feedback method.     

Learning from reviewing peers’ concept maps in an inquiry context: Commenting or grading,which is better?

In peer assessment, both receiving feedback and giving feedback (reviewing peers’ products) have been found to be beneficial for learning. However, the different ways to give feedback and their influence on learning have not been studied enough. This experimental study compared giving feedback by writing comments and by grading, to determine which contributes more to the feedback providers’ learning. Secondary school students from Russia (n = 51) and the Netherlands (n = 42) gave feedback on concept maps during a physics lesson. The lesson was given in an online inquiry learning environment that included an online lab. Students gave feedback in a special Peer Assessment tool, which also provided assessment criteria. Findings indicate that post-test knowledge scores were higher for students from the commenting group. The difference between the groups was largest for the low prior knowledge students. Possible educational implications and directions for further research are discussed.     

Influences on peer evaluation in a group project: an exploration of leadership,demographics and course performance

Collaborative learning strategies are widely used in higher education to deepen learning, promote team-building skills and achieve course learning objectives. Using peer evaluation is an important strategy to ensure that engaged and active students are rewarded for their efforts, and to discourage loafing within groups. However, less is known about what biases may influence students’ peer evaluations. In this paper, we investigate what variables students may (consciously or unconsciously) use to evaluate their peers. We explore the role of sex, race, course performance and group leadership on peer evaluation. We also investigate whether these variables correlate with students’ final course grade. We found that students who reported being leaders in groups were evaluated higher than peers who reported being followers, and that course performance positively correlated with peer evaluations. White students received higher peer evaluations than students of colour. This difference reflects trends in group leadership and course performance, with more white students than students of colour reporting being leaders in groups and receiving higher grades.