Introducing a scalable peer feedback system for learning teams

In the current research, we report on a peer feedback system for supporting student teamwork skill development during post-secondary education. The peer feedback system is part of a larger suite of assessments at ITPmetrics.com. This is a free assessment-based system that allows team members to provide round-robin ratings of each member’s effectiveness in the team on five dimensions (communication; commitment; foundation of knowledge, skills and abilities; emphasising high standards; and focus), as well as provide anonymous written feedback to supplement the numeric scores. Team members have access to a dashboard where they can complete the assessment and store assessment reports generated from their peers’ ratings. We summarise data from student learning teams suggesting that peer ratings are reliable (internally consistent, unidimensional and inter-rater correlated; n?>?85,000). Recommendations for implementation are provided with respect to pre-briefing, debriefing, and assessment schedules.     

A GEARed peer feedback model and implications for learning facilitation

This study proposes a model to view and analyse college students’ written peer feedback after they assess each other’s formative task assignment and provide comments. The model was developed by: (a) revisiting learning-oriented assessment and feedback theories, (b) reviewing and summarising existing peer feedback analysis approaches as detailed in published empirical studies, and (c) analysing a small set of primary learner peer feedback data. In the proposed GEARed model of peer feedback, each complete piece of comment on one improvable unit may have four cognitive elements, i.e. identification of the Gap between performance and goal, Explanation of the gap, gap-bridging Aspirations, and being Resourceful to reach beyond the current knowledge/skill level. Two social-affective-interactive elements also play a part, i.e. being encouraging to the peer feedback receiver and doubtful toward the provider self. Each feedback comment may contain any or all of these incremental elements rather than being assigned only to one category. It is argued that the absent or unresolved aspects in feedback leave room for the receiver to reflect and may suggest instructional actions for teachers. Finally, different patterns of GEARed component combinations are hypothesised and their implications for learning and teaching are discussed.     

Assessing postgraduate student perceptions and measures of learning in a peer review feedback process

Peer review feedback, developed to assist students with increasing the quality of group reports and developing peer review skills, was added to a master’s level Climate Change Policy and Planning unit. A pre- and post-survey was conducted to determine whether students found the process a valuable learning opportunity: 87% of students responding to the post-review survey indicated they saw benefit in both developing and receiving peer review feedback. Peer review design minimising the potential for plagiarism resulted in low levels of student concern surrounding plagiarism. Marks allocated by students were valid (consistent with the lecturer) and were generally reliable; however, distribution of marks had a higher variance where group report quality was low. Training on completing the peer review template could result in more reliable feedback. Consistent with the literature, the feedback developed during the peer review process focused on style and presentation preferentially to content. Seven out of eight group reports improved between report submissions, with all groups responding to peer review feedback. Peer review feedback and changes made to group reports indicated students had limited capacity to judge the quality of referencing and critical analysis.     

Grade one peer and teacher feedback on student writing

This article examines a grade one teacher's support for her students’ writing development through formal peer and teacher feedback. The teacher modelled and provided examples of effective feedback and good writing in whole-class and small-group lessons and in her own one-on-one verbal feedback on student writing. She allocated time for the students to participate in formal peer-feedback sessions, in turn giving feedback to the students on the suggestions to one another during these sessions. Students gave more content-oriented than conventions-oriented feedback to each other. They revised the content and writing conventions of their writing in response to 90% of the feedback they received from their peers and teacher.     

Comparing student learning experiences of in-text commentary and rubric-articulated feedback: strategies for formative assessment

This study compares students’ experiences of two types of criteria-based assessment: in-text commentary and rubric-articulated feedback, in an assessment design combining the two feedback channels. The main aim is to use students’ responses to shed light on how feedback strategies for formative assessment can be optimised. Following action research methodology, the study discusses key categories of student responses from three sources: reflective texts, a questionnaire, and interviews. Results show that different functions were attributed to the two feedback channels: in-text commentary to lower-order concerns related to language proficiency, and rubric-articulated feedback to higher-order concerns related to an overview of writing achievement. We argue that the different functions have the potential of creating a sufficiently balanced assessment design with the possibility of serving both short-term and continuous learning goals. On the other hand, some students found it difficult to navigate between the two feedback channels. The article therefore ends with a ‘lessons learned’ section where we list possible ways in which the current assessment design can be improved for optimal use of the synergy effects emanating from a combination of in-text commentary and rubric-articulated feedback for formative purposes.     

The development of student feedback literacy: enabling uptake of feedback

Student feedback literacy denotes the understandings, capacities and dispositions needed to make sense of information and use it to enhance work or learning strategies. In this conceptual paper, student responses to feedback are reviewed and a number of barriers to student uptake of feedback are discussed. Four inter-related features are proposed as a framework underpinning students’ feedback literacy: appreciating feedback; making judgments; managing affect; and taking action. Two well-established learning activities, peer feedback and analysing exemplars, are discussed to illustrate how this framework can be operationalized. Some ways in which these two enabling activities can be re-focused more explicitly towards developing students’ feedback literacy are elaborated. Teachers are identified as playing important facilitating roles in promoting student feedback literacy through curriculum design, guidance and coaching. The implications and conclusion summarise recommendations for teaching and set out an agenda for further research.     

Designing and implementing peer formative feedback within online learning environments

With increasing need to achieve appropriate balance between learning support and self-regulation within the context of online learning, formative feedback has been identified as a viable means to achieve meaningful engagement. Specifically, this study sought to establish how peer–peer formative feedback was facilitated in an online course and to what extent this engaged students in meaningful learning experiences. This case study entailed an in-depth investigation into the design and implementation of an online course in a New Zealand university. The studied course was part of a postgraduate programme in continuing (in-service) teacher education. The study adopted a case study methodology with a bias on qualitative techniques. Online observations, analysis of the archived course discourse and interviews were utilised as sources of data. The data from multiple sources were subsequently triangulated to corroborate the evidence. The findings indicate that peer formative feedback promoted active learners’ participation and meaningful engagement. The findings further showed that opportunities for dialogic peer formative feedback promoted learning support and self-regulation.     

Exploring the value of peer feedback in online learning for the provider

This paper reviews studies of peer feedback from the novel perspective of the providers of that feedback. The possible learning benefits of providing peer feedback in online learning have not been extensively studied. The goal of this study was therefore to explore the process of providing online peer feedback as a learning activity for the provider. We concluded that (1) providing online peer feedback has several potential learning benefits for the provider; (2) when providing online peer feedback, students use different cognitive processes; (3) the cognitive processes and the potential learning benefits can be realised when students use specific elements in the feedback they provide.     

Assessment for learning: a catalyst for student self-regulation

The development of students as lifelong, self-regulating learners is a valued outcome of higher education. To this end, the current project investigated how students in an undergraduate course experienced and responded to a teaching–learning environment where they were expected to take responsibility for their learning. The pedagogic environment of this course was grounded in strategies and activities associated with assessment for learning (AfL). As such, students were expected to participate in activities that provided them with opportunities to exercise control over their learning. Data gathering comprised individual semi-structured interviews and the collection of artefacts. Findings indicated goals helped students know where they were going; exemplars provided insights into what was expected and what constituted quality work; course activities elicited evidence of learning; dialogic interactions around these generated feedback about understandings and progress; the evaluation of exemplars developed evaluative knowledge, skill and expertise; peer review and feedback provided an authentic context for evaluation and monitoring of works-in-progress. It was concluded that, while each of the AfL strategies contributed to student self-regulation, the full impact of AfL as a catalyst for self-regulated learning was realised in the cumulative and recursive effect these strategies had on students’ thinking, actions and feelings.     

Comparative efficacy of group and individual feedback in gross anatomy for promoting medical student professionalism

Professionalism is a core competency of medical training that requires students to develop the skills of providing and receiving feedback. Our study evaluated the effectiveness of delivering feedback in a group setting compared with an individual setting. The first‐year class of Mayo medical students (n = 49) enrolled in gross anatomy (in dissection teams), completed weekly anonymous evaluations of themselves and their teammates regarding seven aspects of professionalism (altruism, compassion, respect, honesty/integrity, responsibility, commitment to excellence, and self‐reflection). Professionalism scores from these surveys were calculated using a six‐point Likert scale. Students were also asked to comment on strengths and possible areas for improvement on each peer. At the midpoint of the course, peer comments and professionalism scores were shared with students in debriefing sessions either individually or with their team. Analysis of preintervention and postintervention professionalism scores indicated that the students receiving feedback in a one‐on‐one setting (student and instructor) were more likely to demonstrate higher scores on subsequent evaluations as compared with those students receiving feedback in a group setting (all team members and one instructor). Our findings suggest that providing feedback to first‐year medical students on an individual basis is the best way to improve professional attitudes and behaviors. Anat Sci Educ 3: 64–72, 2010. © 2010 American Association of Anatomists.     

Assessment training effects on student assessment skills and task performance in a technology-facilitated peer assessment

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the training, students reviewed learning concepts, discussed marking criteria, graded example projects and compared their evaluations with the instructor’s evaluation. Data were collected in the form of initial and final versions of students’ projects, students’ scoring of example projects before and after the assessment training, and written feedback that students provided on peer projects. Results of data analysis indicate that the assessment training led to a significant decrease in the discrepancy between student ratings and instructor rating of example projects. In addition, the degree of student vs. instructor discrepancy was highly predictive of the quality of feedback that students provided to their peers and the effectiveness of revisions that they made to their own projects upon receiving peer feedback. Smaller discrepancies in ratings were associated with provision of higher quality peer feedback during peer assessment, as well as better revision of initial projects after peer assessment.     

Teamwork skills assessment for cooperative learning

Teamwork skills are required at work, but teacher efforts in many countries to track achievement within this context have been hindered by lack of assessment tools and input from students. The Teamwork Skills Inventory relies on peer and self-evaluation to establish accountability, identify competencies, and detect learning needs. Twenty-five items state the criteria students refer to in reporting observations about whether individual members of their cooperative learning group attend to teamwork, seek and share information, communicate with teammates, think critically and creatively, and get along with teammates. Ways to prepare students for authentic assessment and to process anonymous feedback from peers are discussed. A field test of 303 high school students and teachers determined validity and reliability. Students were able to recognize team skills as well as deficits of peers, and gender differences in competencies were acknowledged. A portfolio record of team skills enables teachers across subjects to apply united interventions.     

A Dialogic Feedback Index measuring key aspects of feedback processes in online learning environments

Research on feedback shows the dialogic perspective as one that potentially promotes learning improvement. This article presents the Dialogic Feedback Index (DFI), a measure that contributes to establishing to what extent feedback design is potentially dialogic. Performing the study in an online context enabled us to evaluate the potential of the index to measure the institutional situation in relation to dialogic feedback. The applied case allowed us to identify four teaching and learning contexts whose designs range from low to high dialogic context. We were also able to establish three components which had a great impact in differentiating one context from another. Specifically, it was found that designing teaching and learning contexts with high levels of dialogic potential of feedback involves: (1) providing feedback before the task is completed, (2) incorporating peer feedback into the process and (3) allowing resubmission once the feedback has been received. The DFI is intended as an applied tool for teaching staff and institutions. With it they can determine the level of dialogic potential of feedback and plan how to improve it.     

Peer assessment as a learning tool for enhancing student teachers' preparation

This study investigates how peer assessment contributes to enhancing student teachers' preparation during field experience. A semi-structured interview was conducted with 72 student teachers. The findings showed that the student teachers have positive beliefs about peer assessment. They think that it can be beneficial if some changes are made in the way it is employed in teacher education programs. In light of the study findings, the researchers put forth a number of suggestions and recommendations for employing peer assessment in teacher education programs in Jordan.     

Investigating secondary school students’ unmediated peer assessment skills

The purpose of this study was to investigate secondary school students’ unmediated peer assessment skills. Specifically, 36 seventh graders, without receiving any kind of support, were anonymously assigned to reciprocally assess their peers’ science web-portfolios. Additionally, students’ attitudes towards and intentions about the use of unsupported reciprocal peer assessment were examined. Three data sources were used, namely, interviews, screen-video captured data, and the peer assessors’ feedback produced. Findings showed that the students have positive attitudes towards unsupported reciprocal peer assessment and that they intend to implement it again. It was also found that students have the skills, at least the beginnings, needed for the implementation of peer assessment. Specifically, they were found to be able to define and use their own assessment criteria, whose overall validity and reliability, however, were found to be low. Finally, the feedback they produced included grades, positive and negative judgments, as well as suggestions for changes.     

Peer review in the classroom: student perceptions,peer feedback quality and the role of assessment

Abstract

Peer review in the classroom can enhance numerous employability skills such as critical appraisal, writing skills, reflection practices and collaborative experiences. This study takes place over two years and discusses the implementation of a repeating blind peer review cycle across a single semester for final year chemistry students enrolled on a compulsory employability module. The feedback cycle promotes personal reflection through the use of mini-reflective questionnaires. The process was assessed by academic tutors at the resubmission stage and/or the peer feedback stage where the quality of peer feedback was directly assessed. The research investigates the quality of peer feedback, the importance of assessment and student perceptions of what is most useful. Methods include directed content analysis of feedback produced, student opinions and a focus group. Students were capable of offering useable feedback across a range of assessment criteria but tend to focus on the important criteria best aligned to the particular assignment. A range of motivational factors and tactics were noticed but students tended to find reviewing the most beneficial.     

The motivational paradox of feedback: teacher and student perceptions

The notion that future performance can be affected by information about previous performance is often expressed in terms of ‘closing the gap’. Feedback has long been recognised as a mechanism through which teaching and learning may be influenced. The current wave of support in the United Kingdom for assessment for learning echoes these sentiments. This paper examines the feedback strategies employed by two experienced literacy practitioners in England. Using data gathered from field observations, interviews and documentary sources, the paper presents evidence of espoused practice associated with feedback, demonstrating that whilst teachers may claim that they make effective use of some feedback strategies to support pupils’ learning and motivation, that this is not supported by empirical data. The paper also identifies that whilst some teachers aim to mark every piece of pupils’ written work for perceived motivational benefits; such a strategy can undermine pupils’ intrinsic motivation and lead to a culture of over-dependency, whereby the locus of control with regard to feedback lies solely with the teacher. The paper concludes by exploring some possible implications for practice with regard to the provision of written feedback in particular.     

Effects of feedback on collaborative writing in an online learning environment

The need for supporting student writing has received much attention in writing research. One specific type of support is feedback—including peer feedback—on the writing process. Despite the wealth of literature on both feedback and academic writing, there is little empirical evidence on what type of feedback best promotes writing in online environments. This article reports on research that tries to determine what type of feedback best improves the quality of collaborative writing and what the effects of feedback are on student learning in an environment based on asynchronous written communication. The results reveal that concerning the type of feedback, epistemic feedback or epistemic and suggestive feedback best improve the quality of collaborative writing performance. The nature of the feedback-giver (whether teacher feedback or teacher and peer) makes a difference to the final text only when the feedback is epistemic, or epistemic and suggestive.     

The student experience of piloting multi-modal performance feedback tools in health and social care practice (work)-based settings

The aim of this study was to evaluate newly developed performance feedback tools from the student perspective. The tools were innovative in both their mode of delivery and the range of stakeholders they involved in the feedback process. By using the tools in health and social care settings, students were able to engage in interprofessional assessment of common competences and obtain performance feedback from a range of stakeholders not commonly involved in work-based learning; these included peers and service users. This paper discusses the ways in which the performance feedback tools were developed by a collaborative programme and compares their delivery, across a wide range of professions and work-based settings, in paper-based, web-based and mobile formats. The tools were evaluated through a series of profession-specific focus groups involving 85 students and 7 professions. The data were analysed thematically and reduced to three key categories: mode of delivery, assessment tool dynamics and work-based issues. These will be discussed in detail. The students agreed that the structured way of capturing and documenting feedback from several sources would support their practice placement learning. The reflective nature of the tools and the capacity for guiding reflection was also welcomed. The concepts of gaining service user, peer and/or interprofessional feedback on performance were new to some professions and evoked questions of reliability and validity, alongside appreciation of the value they added to the assessment process.     

Enhanced student learning in accounting utilising web-based technology,peer-review feedback and reflective practices: a learning community approach to assessment

Higher education is becoming a major driver of economic competitiveness in an increasingly knowledge-driven global economy. Maintaining the competitive edge has seen an increase in public accountability of higher education institutions through the mechanism of ranking universities based on the quality of their teaching and learning outcomes. As a result, assessment processes are under scrutiny, creating tensions between standardisation and measurability and the development of creative and reflective learners. These tensions are further highlighted in the context of large undergraduate subjects, learner diversity and time-poor academics and students. Research suggests that high level and complex learning is best developed when assessment, combined with effective feedback practices, involves students as partners in these processes. This article reports on a four-phase, cross-institution and cross-discipline project designed to embed peer-review processes as part of the assessment in two large, under-graduate accounting classes. Using a social constructivist view of learning, which emphasises the role of both teacher and learner in the development of complex cognitive understandings, we undertook an iterative process of peer review. Successive phases built upon students’ feedback and achievements and input from language/learning and curriculum experts to improve the teaching and learning outcomes.