Introducing a scalable peer feedback system for learning teams

In the current research, we report on a peer feedback system for supporting student teamwork skill development during post-secondary education. The peer feedback system is part of a larger suite of assessments at ITPmetrics.com. This is a free assessment-based system that allows team members to provide round-robin ratings of each member’s effectiveness in the team on five dimensions (communication; commitment; foundation of knowledge, skills and abilities; emphasising high standards; and focus), as well as provide anonymous written feedback to supplement the numeric scores. Team members have access to a dashboard where they can complete the assessment and store assessment reports generated from their peers’ ratings. We summarise data from student learning teams suggesting that peer ratings are reliable (internally consistent, unidimensional and inter-rater correlated; n?>?85,000). Recommendations for implementation are provided with respect to pre-briefing, debriefing, and assessment schedules.     

Exploring the value of peer feedback in online learning for the provider

This paper reviews studies of peer feedback from the novel perspective of the providers of that feedback. The possible learning benefits of providing peer feedback in online learning have not been extensively studied. The goal of this study was therefore to explore the process of providing online peer feedback as a learning activity for the provider. We concluded that (1) providing online peer feedback has several potential learning benefits for the provider; (2) when providing online peer feedback, students use different cognitive processes; (3) the cognitive processes and the potential learning benefits can be realised when students use specific elements in the feedback they provide.     

A GEARed peer feedback model and implications for learning facilitation

This study proposes a model to view and analyse college students’ written peer feedback after they assess each other’s formative task assignment and provide comments. The model was developed by: (a) revisiting learning-oriented assessment and feedback theories, (b) reviewing and summarising existing peer feedback analysis approaches as detailed in published empirical studies, and (c) analysing a small set of primary learner peer feedback data. In the proposed GEARed model of peer feedback, each complete piece of comment on one improvable unit may have four cognitive elements, i.e. identification of the Gap between performance and goal, Explanation of the gap, gap-bridging Aspirations, and being Resourceful to reach beyond the current knowledge/skill level. Two social-affective-interactive elements also play a part, i.e. being encouraging to the peer feedback receiver and doubtful toward the provider self. Each feedback comment may contain any or all of these incremental elements rather than being assigned only to one category. It is argued that the absent or unresolved aspects in feedback leave room for the receiver to reflect and may suggest instructional actions for teachers. Finally, different patterns of GEARed component combinations are hypothesised and their implications for learning and teaching are discussed.     

Passive,active, and constructive engagement with peer feedback: A revised model of learning from peer feedback

To deepen understanding of learning through peer feedback, the current study investigated the relationships between different peer feedback activities (organized into constructive vs active activities) and learning (i.e., transfer to new tasks), examining the nature of activities within provided feedback, received feedback, and revisions in response to feedback. Across five US high schools, 367 students in Advanced Placement classes participated, implementing common assignments and peer assessment rubrics. Provided/received comments and revisions in one assignment, and writing improvements observed in a second assignment were exhaustively coded and subjected to hierarchical model regression analyses. Results showed that constructive activities (providing explanations and making revisions after receiving explanations or providing suggestions) were consistently associated with learning, whereas passive (e.g., receiving feedback without making revisions) or active activities (e.g., implementing specific suggestions) were not. Further, the effects of received feedback on learning were mediated by the number of revisions. Theoretical and practical implications of the findings are discussed.     

The effect of peer feedback for blogging on college students' reflective learning processes

Reflection is an important prerequisite to making meaning of new information, and to advance from surface to deep learning. Strategies such as journal writing and peer feedback have been found to promote reflection as well as deep thinking and learning. This study used an empirical design to investigate the interaction effects of peer feedback and blogging on college students' reflective thinking skills and their learning approaches. Forty-four first- and second-year undergraduate students participated in the study. Students kept blogs each week throughout a whole semester. Two journals were sampled at the beginning and end of the semester for each student. A repeated measure one-way ANOVA suggested that students' reflective thinking level increased significantly over time; however, peer feedback was found to negatively affect students' reflective thinking skills. The result of the study suggests more carefully designed uses in the future.     

Designing and implementing peer formative feedback within online learning environments

With increasing need to achieve appropriate balance between learning support and self-regulation within the context of online learning, formative feedback has been identified as a viable means to achieve meaningful engagement. Specifically, this study sought to establish how peer–peer formative feedback was facilitated in an online course and to what extent this engaged students in meaningful learning experiences. This case study entailed an in-depth investigation into the design and implementation of an online course in a New Zealand university. The studied course was part of a postgraduate programme in continuing (in-service) teacher education. The study adopted a case study methodology with a bias on qualitative techniques. Online observations, analysis of the archived course discourse and interviews were utilised as sources of data. The data from multiple sources were subsequently triangulated to corroborate the evidence. The findings indicate that peer formative feedback promoted active learners’ participation and meaningful engagement. The findings further showed that opportunities for dialogic peer formative feedback promoted learning support and self-regulation.     

Instructional scaffolds for learning from formative peer assessment: effects of core task,peer feedback,and dialogue

Formative peer assessment is an instructional method that offers many opportunities to foster students’ learning with respect to both the domain of the core task and students’ assessment skills. The contributions to this special issue effectively address earlier calls for more research into instructional scaffolds and the implementation of dialogic features in formative peer assessment. However, open issues remain regarding the role of assessment criteria, the benefit of formative peer assessment for transferable knowledge and skills, the role of metacognitive and cognitive processes in the provision and reception of peer feedback, and the proposed benefit of more interactive forms of formative peer assessment. Addressing the latter issue in particular, a framework of three dimensions of increasing interactivity is proposed in order to guide future research. These three dimensions comprise the learner’s engagement with the core task (low interactivity), the provision and reception of peer feedback (medium interactivity), and the learner’s engagement with argumentation, tutoring, and co-construction in dialogue with peers (high interactivity).     

Effects of elicited reflections combined with tutor or peer feedback on self-regulated learning and learning outcomes

This study investigated the effects of students' reflections, combined with suggestive feedback, on the development of self-regulated learning and learning outcomes. Suggestive feedback alerts students that further reflection is advisable, without being very directive. Forty-nine students participated in a regular web-based distance education course on work psychology. In two experimental conditions students were prompted to reflect on their learning process, and received feedback on their reflections, either from peer students or a tutor. These conditions were compared to a control condition in which students studied the regular course without reflection and feedback. In line with the hypothesis, results showed that reflection combined with feedback positively affected students' self-regulated learning. In addition, students in the condition with tutor feedback outperformed students in both other conditions on learning outcomes.     

The development of academics’ feedback literacy: experiences of learning from critical feedback via scholarly peer review

Abstract

The emerging literature related to feedback literacy has hitherto focused primarily on students’ engagement with feedback, and yet an analysis of academics’ feedback literacy is also of interest to those seeking to understand effective strategies to engage with feedback. Data from concept map-mediated interviews and reflections, with a team of six colleagues, surface academics’ responses to receiving critical feedback via scholarly peer review. Our findings reveal that feedback can be visceral and affecting, but that academics employ a number of strategies to engage with this process. This process can lead to actions that are both instrumental, enabling academics to more effectively ‘play the game’ of publication, as well as to learning that is more positively and holistically developmental. This study thus aims to open up a dialogue with colleagues internationally about the role of feedback literacy, for both academics and students. By openly sharing our own experiences we seek to normalise the difficulties academics routinely experience whilst engaging with critical feedback, to share the learning and strategies which can result from peer review feedback, and to explore how academics may occupy a comparable role to students who also receive evaluation of their work.     

Assessing postgraduate student perceptions and measures of learning in a peer review feedback process

Peer review feedback, developed to assist students with increasing the quality of group reports and developing peer review skills, was added to a master’s level Climate Change Policy and Planning unit. A pre- and post-survey was conducted to determine whether students found the process a valuable learning opportunity: 87% of students responding to the post-review survey indicated they saw benefit in both developing and receiving peer review feedback. Peer review design minimising the potential for plagiarism resulted in low levels of student concern surrounding plagiarism. Marks allocated by students were valid (consistent with the lecturer) and were generally reliable; however, distribution of marks had a higher variance where group report quality was low. Training on completing the peer review template could result in more reliable feedback. Consistent with the literature, the feedback developed during the peer review process focused on style and presentation preferentially to content. Seven out of eight group reports improved between report submissions, with all groups responding to peer review feedback. Peer review feedback and changes made to group reports indicated students had limited capacity to judge the quality of referencing and critical analysis.     

Improving the effectiveness of feedback by use of assessed reflections and withholding of grades

We wished to improve levels of student engagement with feedback within the context of our postgraduate masters-level programme, and therefore evaluated the use of two interventions: assessed reflections on feedback and grade-withholding. In questionnaires students reported more engagement with feedback after the interventions, with 77% in favour of using reflections, though only 57% favoured grade-withholding, with feelings of frustration and anxiety about the grade cited as factors. Overall class grades improved over the two years in which reflections were used, with the greatest gains made by students generating the most insightful reflections. Additional gains in the second year of intervention may have been attributable to improved implementation or the introduction of grade-withholding, or a combination of both. Overall we demonstrated clear improvement in feedback utilisation and achievement associated with our interventions.     

Formative self-and peer assessment for improved student learning: the crucial factors of design,teacher participation and feedback

Self and peer-assessment are becoming central aspects of student-centred assessment processes in higher education. Despite increasing evidence that both forms of assessment are helpful for developing key capabilities in students, such as taking more responsibility for their own learning, developing a better understanding of the subject matter, assessment criteria and their own values and judgements, and developing critical reflection skills, both forms of assessment are still not the norm at universities. This paper provides the findings of a two-year study of formative self and peer-assessment at an Australian university. The study supports other research showing that students tend to regard formative self and peer-assessment as beneficial for gaining more insights about the assessment process and for improving their own work. We argue that self and peer-assessment requires careful design and implementation for it to be an effective tool for formative assessment processes; and that the development of students’ capacities for giving feedback, and the continuous and timely involvement of the teacher, are central aspects for successful self and peer-assessment. The move to self and peer-assessment is not simple for teachers and students but is worthwhile and necessary for twenty-first century higher education.     

Transferable skills for online peer learning

Efforts to enhance learning through peer interaction in an electronic forum are now commonplace. However, facilitation and moderation of such a forum by academic staff can be of limited effectiveness and very time‐consuming. The skills required by peer learners to effectively manage such distributed discourse for themselves have rarely been clearly identified. This paper compares the social, organizational and cognitive characteristics of effective peer learning interactions in face‐to‐face and online environments (with particular reference to peer tutoring and peer assessment). This leads to a specification of transferable skills for online peer learning, at the macro and micro levels, with commentary on areas needing particular attention. The implications for assessment of such online distributed discourse process skills are explored with reference to professional practice and future research.     

The development of student feedback literacy: the influences of teacher feedback on peer feedback

Abstract

Although the concept of student feedback literacy has drawn increasing attention in higher education, empirical research on this matter is still in its infancy. In the area of peer feedback, little research has investigated the role of teacher follow-up feedback on peer feedback in the development of student feedback literacy. To address the research gap, a multiple-case study of three Chinese master’s students enrolled in an academic writing course was conducted, drawing on the students’ drafts with peer feedback, teacher written feedback on that peer feedback, semi-structured interviews, retrospective verbal reports, observation field notes and class documents. Three students’ epistemological and practical knowledge about, attitudes towards, and self-efficacy beliefs in peer feedback were found to improve at different paces and to different degrees. However, considerable individual variations were observed with two high-achieving, highly motivated participants becoming more feedback-literate than their under-achieving, minimally motivated peer. Teacher feedback on peer feedback was found to have distinct impacts on individual students, depending on learner factors including language ability, beliefs and motivation. These findings suggest that teacher feedback on peer feedback, if consistently provided and compatible with learner factors, can scaffold both cognitive and social-affective aspects of student feedback literacy.     

Self-regulation of peer feedback quality aspects through different dimensions of experience within prior peer feedback assignments

Although peer review is a widely-used pedagogical technique, its value depends upon the quality of the reviews that students produce, and much research remains to be done to systematically study the nature, causes, and consequences of variation in peer review quality. We propose a new framework that conceptualizes five larger dimensions of peer review quality and then present a study that investigated three specific peer review quality constructs in a large dataset and further explored how these constructs change through different types of self-regulation peer reviewing experiences. Peer review data across multiple assignments were analyzed from 2,092 undergraduate students enrolled in one of three offerings of a biology course at a large public research university in the United States. Peer review quality was measured in terms of comment amount, comment accuracy, and rating accuracy; the measures of reviewing experience focused upon self-regulated learning factors such as practice, feedback, others’ modeling, and relative performance. Meta-correlation (for testing reliability, separability, and stability) and meta-regression (as a time-series analysis for testing the relationship of changes across assignments in reviewing quality with experiences as reviewer and reviewee) are used to establish the robustness of effects and meaningful variation of effects across course offerings and assignments. Results showed that there were three meaningful review quality constructs (i.e., were measured reliably, separable, and semi-stable over time). Further, all three showed changes in response to previous reviewer and reviewee experiences, but only feedback helpfulness, in particular, showed effects of all four examined types of self-regluation experiences (practice, feedback, others’ modeling, and relative performance). The findings suggest that instructors can improve review quality by providing comment prompt scaffolds that lead to longer comments as well as by matching authors with similarly performing reviewers.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

A case study of using peer feedback in face-to-face and distance learning classes among pre-service teachers

This study investigated a feedback exchange activity for engaging pre-service teachers and the nature of such feedback in two undergraduate classes, a distance learning (DL) and a face-to-face (F2F) class. The research question asked if the nature of peer feedback was different between F2F and DL class formats. Students’ work samples were collected during class time to examine feedback function ratings. Participants also completed an online questionnaire to examine the nature of expectations of the feedback given. For both the nature of expectations and feedback function ratings, results for chi-square analyses demonstrated no statistically significant difference between the two modes of instruction (F2F versus DL), and the peer feedback categories. However, provision of feedback was significantly favoured in the F2F class over the DL class. Implications for course curriculum and future research are discussed.     

A comparison of peer and tutor feedback

We report on a study comparing peer feedback with feedback written by tutors on a large, undergraduate software engineering programming class. Feedback generated by peers is generally held to be of lower quality to feedback from experienced tutors, and this study sought to explore the extent and nature of this difference. We looked at how seriously peers undertook the reviewing task, differences in the level of detail in feedback comments and differences with respect to tone (whether comments were positive, negative or neutral, offered advice or addressed the author personally). Peer feedback was also compared by academic standing, and by gender. We found that, while tutors wrote longer comments than peers and gave more specific feedback, in other important respects (such as offering advice) the differences were not significant.     

Improving assessment of learning and for learning

The focus of this article is recent work by the Assessment Reform Group (ARG) on the role of teachers' judgements in the summative use of assessment. A brief overview of the early work of the ARG is followed by discussion of the desirable properties of assessment for summative uses. The work of the ARG's Assessment Systems for the Future project provided evidence and arguments concerning the validity, reliability, impact and cost of tests and of summative assessment by teachers. Whilst there is ample evidence that the teachers' judgements are more valid than, and equally reliable as, tests, there is a danger of unwanted impact on teaching as long as results are used for ‘high stakes’ evaluation of teachers and schools. Implications for policy include an end to the practice of using the results of pupils' summative assessment, however they are derived, as the sole basis for target setting and school accountability.     

Improving self-regulation, learning strategy use, and achievement with metacognitive feedback

Comprehension of science topics occurs when learners meaningfully generate relationships and conceptions about what they read. In this generation process, learners’ cognitive and metacognitive regulation is one of the most critical factors influencing learning. However, learners are not always successful in regulating their own learning, especially in computer-based learning environments (CBLEs) where they are alone. Based on this rationale, the present study was designed to examine the effects of two scaffolding strategies—generative learning strategy prompts and metacognitive feedback—on learners’ comprehension and self-regulation while learning the human heart system in a CBLE. Participants were 223 undergraduate student volunteers. Structural Equation Modeling (SEM) was employed to conceptualize and empirically test a model that explains mediating processes among variables. Results revealed that the combination of generative learning strategy prompts with metacognitive feedback improved learners’ recall and comprehension by enhancing learners’ self-regulation and better use of highlighting and summarizing as generative learning strategies.