Investigating Feedback on Practice Among Teachers: Coherence of Observed and Perceived Feedback

Despite that benefits of feedback in student learning are reported in much research, little has been reported regarding the use of feedback from teachers to other teachers—a key tool in professional development. In this study, we triangulated data from videotaped peer coaching sessions, questionnaires, and interviews regarding 12 primary school teachers in four peer groups in the Netherlands. We focused our research on two issues: the interplay of observed feedback dimensions and elements and perceptions of that feedback. Feedback dimensions were generally effective and the influence of the elements on the dimensions mostly aligned with the expectations. Teachers generally perceived feedback as effective. Moreover, effective observed feedback was perceived as effective. Findings indicate that peer coaches should stimulate coached teachers to become goal directed, specific, detailed, and neutral (neither positive nor negative) by using feedback elements so as to optimize feedback processes.     

Small group discussion as a key component in online assessment training for enhanced student learning in web-based peer assessment

The purpose of this study is to examine the effect of online assessment training, with synchronous group discussion as a key component, on subsequent web-based peer assessment results. Participants included 81 college students, mostly women, taking a business writing class. After initial submission of a draft counter-offer letter, they completed online assessment training by engaging in a consensus-seeking, synchronous group discussion of the rubric-based ratings they gave to sample counter-offer letters. They then engaged in web-based assessment of randomly assigned peer counter-offer letters and upon receiving peer feedback, submitted revision of their own counter-offer letter. The discussion groups were randomly assigned to either anonymous (using a pseudo name) or identified conditions (using real name). Findings indicate that the quality of student counter-offer letters improved after online assessment training, and improved even more after web-based peer assessment. There were no significant differences between anonymous discussion groups and identified groups.     

Sources of Instructional Feedback, Job Satisfaction, and Basic Psychological Needs

This study examined college instructors’ utilization and perceived value of sources of instructional feedback (institutional student ratings, consultation with an instructional specialist, soliciting feedback from students, self-assessment, self-observation, peer/administrator observation, and peer coaching). We examined relationships between the utilization of each source of feedback with job satisfaction and psychological needs satisfaction. We solicited instructors (N?=?126) via email. Results revealed self-assessment was the most utilized source of instructional feedback, and instructor-solicited feedback from students was perceived as the most useful. Job satisfaction was significantly correlated with basic psychological needs. We discuss qualitative findings, implications, and suggestions for future research.     

Understanding student engagement with peer feedback on master’s theses: a Macau study

While research on peer feedback in second language (L2) writing at university/college level has proliferated, scholars have conducted less research on peer feedback in L2 academic and scholarly writing. Drawing upon multiple sources of data including semi-structured interviews, stimulated recalls, online interviews via WeChat, first and revised drafts of master’s theses, peer written feedback, audio-recordings of oral peer feedback conferences and the finalised master’s theses, this case study examines how three master’s students affectively, behaviourally and cognitively engaged with peer feedback on drafts of their thesis in a Master of Education programme at a Macau university. The findings show that the relationship between sub-constructs within each dimension and the relationships between the three dimensions of student engagement were interconnected in a dynamic and complex way. While there were individual differences among the three students’ engagement, their affective engagement could promote or negatively influence their behavioural and cognitive engagement. This study builds our understanding of the multifaceted nature of master’s students’ engagement with peer feedback. It provides pedagogical implications about how peer feedback can enhance the quality of master’s students’ academic writing and sheds light on how to guide students to establish scholarly learning communities in higher education.     

同伴互评在翻译教学中的应用效果及其教学法意义

本研究旨在探讨同伴互评在翻译教学中的应用效果。研究对象是20 名英语专业高年级的学生。研究者首先对受试进行一定的培训,随后收集同伴互评的相关数据,此后随机挑选6名 学生进行访谈。结果表明：同伴互评行之有效;同伴评论的利用率很高;修改后的译文质量有显著提高;翻译的方向对同伴互评的效果无明显影响;学习者对互评机制持积极态度。将同伴互评应用到翻译教学中来,有利于培养学习者的自主 学习与合作学习的能力,强化校改意识并提高翻译质量,以及促成我国的翻译教学向以过程为取向过渡。     

Introducing a scalable peer feedback system for learning teams

In the current research, we report on a peer feedback system for supporting student teamwork skill development during post-secondary education. The peer feedback system is part of a larger suite of assessments at ITPmetrics.com. This is a free assessment-based system that allows team members to provide round-robin ratings of each member’s effectiveness in the team on five dimensions (communication; commitment; foundation of knowledge, skills and abilities; emphasising high standards; and focus), as well as provide anonymous written feedback to supplement the numeric scores. Team members have access to a dashboard where they can complete the assessment and store assessment reports generated from their peers’ ratings. We summarise data from student learning teams suggesting that peer ratings are reliable (internally consistent, unidimensional and inter-rater correlated; n?>?85,000). Recommendations for implementation are provided with respect to pre-briefing, debriefing, and assessment schedules.     

Latent trait modelling of rater accuracy in formative peer assessment of English-Chinese consecutive interpreting

Despite the increasing popularity of peer assessment in tertiary-level interpreter education, very little research has been conducted to examine the quality of peer ratings on language interpretation. While previous research on the quality of peer ratings, particularly rating accuracy, mainly relies on correlation and analysis of variance, latent trait modelling emerges as a useful approach to investigate rating accuracy in rater-mediated performance assessment. The present study demonstrates the use of multifaceted Rasch partial credit modelling to explore the accuracy of peer ratings on English-Chinese consecutive interpretation. The analysis shows that there was a relatively wide spread of rater accuracy estimates and that statistically significant differences were found between peer raters regarding rating accuracy. Additionally, it was easier for peer raters to assess some students accurately than others, to peer-assess target language quality accurately than the other rating domains, and to provide accurate ratings to English-to-Chinese interpretation than the other direction. Through these findings, latent trait modelling demonstrates its capability to produce individual-level indices, measure rater accuracy directly, and accommodate sparse data rating designs. It is therefore hoped that substantive inquiries into peer assessment of language interpretation could utilise latent trait modelling to move this line of research forward.     

The impact of formative peer feedback on higher education students’ academic writing: a Meta-Analysis

Peer feedback is frequently implemented with academic writing tasks in higher education. However, a quantitative synthesis is still lacking for the impact that peer feedback has on students’ writing performance. The current study conveyed two types of observations. First, regarding the impact of peer feedback on writing performance, this study synthesized the results of 24 quantitative studies reporting on higher education students’ academic writing performance after peer feedback. Engagement in peer feedback resulted in larger writing improvements compared to (no-feedback) controls (g?=?0.91 [0.41, 1.42]) and compared to self-assessment (g?=?0.33 [0.01, 0.64]). Peer feedback and teacher feedback resulted in similar writing improvements (g?=?0.46 [-0.44, 1.36]). The nature of the peer feedback significantly moderated the impact that peer feedback had on students’ writing improvement, whereas only a theoretically plausible, though non-significant moderating pattern was found for the number of peers that students engaged with. Second, this study shows that the number of well-controlled studies into the effects of peer feedback on writing is still low, indicating the need for more quantitative, methodologically sound research in this field. Findings and implications are discussed both for higher education teaching practice and future research approaches and directions.     

A modified peer rating system to recognise rating skill as a learning outcome

The peer rating system used here advances the quantitative literacy goals outlined in the social sciences. We instituted a mid-semester intervention to teach rating skills and used an index to track longitudinal changes of skill mastery over the course of the semester. Seventy-four students in five advanced research classes followed the procedure of the existing peer rating system by completing reading assignments, writing reflections online, engaging in class discussions, rating their peers’ reflections and receiving feedback on their group effort. Peer ratings were then compared with each other and also with the instructor ratings to derive individualised indices of reliability and validity. These technical indicators enabled two rounds of assessment before and after a class-wide intervention. An omnibus test across the five classes showed a significant improvement in rating quality due to the intervention. Our courses not only met a quantitative learning outcome but also promised vocational competence.     

The bilateral benefits of providing and receiving peer feedback in academic writing across varying L2 proficiency

Although the effectiveness of peer review has been examined, few have tested the joint benefits of providing and receiving feedback features and quality in L2 contexts. The present study investigated variation in key features and quality of feedback provided and received by high and low L2 proficiency students and its benefits on revision in the authentic setting of students experiencing both roles of providing and receiving peer feedback in English academic writing. Analyses on two drafts from 50 students, 1356 idea units of anonymous implementable peer feedback and back-evaluation ratings revealed that the combined effect of providing and receiving feedback on revision was prominent for all students. But the bilateral benefits of providing and receiving feedback features were more significant for high proficiency students, while providing feedback features and quality benefited low proficiency students more. The study implies that students need more bilateral training with both providing and receiving feedback.     

To Feedback or Not to Feedback in Student Self-assessment

Extensive research (Black & Wiliam, 1998) notes the importance of feedback for learning. It is posited that even final-year undergraduate students will benefit from feedback and would thus value tutor feedback as an integral part of the student self-assessment process. At a British university, students were trained to carry out self-assessment of two types: firstly, self-assessment prior to peer and tutor feedback and secondly, self-assessment which integrated feedback as part of the process. Students overwhelmingly favoured the latter. In addition, analysis of student errors showed that, contrary to expectations, the majority of students' errors in translation were principally caused by their native language. Furthermore, discussion and questions on criteria revealed that students' expectations of their grades were closely related to the amount of time and effort which they had invested in their work. This was external to the actual standard of their work and their awareness of this.     

国内外同伴互评研究综述：2004-2013

自20世纪70年代起,同伴互评就作为过程写作法的重要环节受到国内外学者广泛关注。其相关研究包括同伴互评本质、外语写作教学中的实际应用效果、与教师评价的对比等内容。其最新研究成果意义深远,为在英语写作教学中深入开展同伴互评提供了理论指导。因此,梳理相关文献就显得尤为迫切和重要。     

The development of student feedback literacy: the influences of teacher feedback on peer feedback

Abstract

Although the concept of student feedback literacy has drawn increasing attention in higher education, empirical research on this matter is still in its infancy. In the area of peer feedback, little research has investigated the role of teacher follow-up feedback on peer feedback in the development of student feedback literacy. To address the research gap, a multiple-case study of three Chinese master’s students enrolled in an academic writing course was conducted, drawing on the students’ drafts with peer feedback, teacher written feedback on that peer feedback, semi-structured interviews, retrospective verbal reports, observation field notes and class documents. Three students’ epistemological and practical knowledge about, attitudes towards, and self-efficacy beliefs in peer feedback were found to improve at different paces and to different degrees. However, considerable individual variations were observed with two high-achieving, highly motivated participants becoming more feedback-literate than their under-achieving, minimally motivated peer. Teacher feedback on peer feedback was found to have distinct impacts on individual students, depending on learner factors including language ability, beliefs and motivation. These findings suggest that teacher feedback on peer feedback, if consistently provided and compatible with learner factors, can scaffold both cognitive and social-affective aspects of student feedback literacy.     

Designing student peer assessment in higher education: analysis of written and oral peer feedback

The nature of written and oral peer feedback will be described as it occurred in seven writing courses, each with a different design of peer assessment. In all courses, peer feedback was focused on evaluation, which is one of the four feedback functions. Feedback on structure was hardly provided. Relating feedback to design features, we suggest that feedback is adequate when (1) peer assessment has a summative (on the basis of a writing product) as well as a formative character (during the writing process); (2) the assessment is performed in small feedback groups; (3) the written feedback is orally explained and discussed with the receiver.     

The validity and students’ experiences of peer assessment in a large introductory class of gene technology

Although studies have examined the validity of peer assessment, research including students’ own experiences of peer assessment is scarce. The present study aims to improve assessment practices in a context with a highly traditional assessment culture. The aim is first to examine the validity of peer assessment by analysing the compatibility of student and teacher evaluation and explore the differences between minor and major students’ evaluations. Second, the study examines students’ experiences of peer assessment. Peer assessment was implemented in a large bioscience course with 79 student participants. After the peer assessment, the students provided feedback. The results indicate that student subject understanding can be supported through a proper assessment practice. Peer assessment was successful in an introductory class with minor and major students, and most students experienced it as supportive of their learning.     

Assessment training effects on student assessment skills and task performance in a technology-facilitated peer assessment

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the training, students reviewed learning concepts, discussed marking criteria, graded example projects and compared their evaluations with the instructor’s evaluation. Data were collected in the form of initial and final versions of students’ projects, students’ scoring of example projects before and after the assessment training, and written feedback that students provided on peer projects. Results of data analysis indicate that the assessment training led to a significant decrease in the discrepancy between student ratings and instructor rating of example projects. In addition, the degree of student vs. instructor discrepancy was highly predictive of the quality of feedback that students provided to their peers and the effectiveness of revisions that they made to their own projects upon receiving peer feedback. Smaller discrepancies in ratings were associated with provision of higher quality peer feedback during peer assessment, as well as better revision of initial projects after peer assessment.     

Sex does not matter: gender bias and gender differences in peer assessments of contributions to group work

This paper considers the possibility of gender bias in peer ratings for contributions to team assignments, as measured by an online self-and-peer assessment tool. The research was conducted to determine whether peer assessment led to reliable and fair marking outcomes. The methodology of Falchikov and Magin was followed in order to test their finding that gender has no discernable impact on peer ratings. Data from over 1500 participants at two universities enrolled in four different degree programmes were analysed. The research indicates an absence of gender bias in six case studies. The research also found that women received significantly higher ratings than men.     

Humor and competence in school-aged children

Multiple aspects of humor were evaluated in children between the ages of 10 and 14 and related to several areas of competence manifested at school. Humor measures assessed appreciation (including mirth, subjective ratings, and response sets), comprehension, and production, while competence measures included teacher ratings of classroom behavior, peer reputation, and achievement. Humor was related to competence in several ways consistent with previous theory and research: (1) through the manifestation of intellectual ability both in humor behaviors and in competent functioning; (2) through the role of mastery motivation enhancing both types of functioning; and (3) through peer relations, resulting from the effects of humor on peer acceptance or the effects of peer relations on humor behaviors. Ideas for further research relating humor to social competence, social cognition, and mastery motivation are discussed.