How often do students working in two‐person teams report that work was shared equitably?

There are many reasons to assign group projects but determining the grade for each individual working in a group can be problematic. Self and peer assessments of contributions to a group project can be used to adjust individual grades. Most studies of such assessments have considered teams with three to seven members. This study documents the degree of congruence between self and peer assessments of contributions to a project assigned to teams of two people. The majority of students (88.6%) assigned equal credit to themselves and to their partner. Only 16.2% of students had their grades adjusted by the assessment process and most of those adjustments were small. These findings may alleviate concerns of instructors and students about the fairness of assigning individual grades to students working on a project in teams of two.     

Peer assessment accounting for student agreement

Peer assessment has been gaining significant ground as a means of fostering an environment of accountability and responsibility for group projects in higher education. A popular peer assessment method assigns individual grades via a linear relationship to peer assessment scores. Using this method, a group member who receives his or her group’s average peer assessment score is assigned the instructor‐given group grade; the group member who receives a total peer assessment score of zero is assigned an individual grade of zero. Unfortunately, applying this method often requires time‐consuming exploratory data analysis because of the potential for the method to yield a high spread of grades, including grades over 100%. The present study proposes a modification, which accounts for the degree of student agreement. When the modified procedure results in a change from the group grade, the direction of change is always the same, but with smaller magnitude, requiring less subsequent data analysis and instructor time demands. Easy‐to‐modify SAS code for applying the method is presented in the Appendix.     

Evaluating the quality of peer and self evaluations as measures of student contributions to group projects

ABSTRACT

Educators in higher education commonly use peer and self evaluations to help assess student performance on group projects. Although these evaluations provide multiple benefits, many educators are wary of using them due to concerns about their quality. This study addresses three questions debated in the literature regarding the quality of these assessments. How much do students differentiate among peer contributions through their ratings? How reliable are peer ratings? How much agreement exists between peer and self ratings? Although these questions have been addressed to varying degrees in past work, their answers have been far from settled. While many studies focus on just one of the questions, this study’s data make it possible to address all three questions for the same group of students as well as examine each question by student performance level. The evaluations assessed in this study were completed by a large number of students under conditions associated with obtaining more valid and reliable ratings. Overall, the results provide support for using peer and self evaluations to help assess student contributions to group projects. Peer ratings were largely reliable as group members generally agreed on the scores given to their peers. In addition, most students differentiated among group member contributions through their ratings. Students also tended to rate themselves higher than their peers rated them. This study has implications for how peer and self evaluations can be most effectively used by educators to measure student performance in group work.     

Assessing the reliability of self‐ and peer rating in student group work

Peer and self‐ratings have been strongly recommended as the means to adjust individual contributions to group work. To evaluate the quality of student ratings, previous research has primarily explored the validity of these ratings, as indicated by the degree of agreement between student and teacher ratings. This research describes a Generalizability Theory framework to evaluate the reliability of student ratings in terms of the degree of consistency among students themselves, as well as group and rater effects. Ratings from two group projects are analyzed to illustrate how this method can be applied. The reliability of student ratings differs for the two group projects considered in this research. While a strong group effect is present in both projects, the rater effect is different. Implications of this research for classroom assessment practice are discussed.     

Peer assessment in group projects accounting for assessor reliability by an iterative method

This study proposes an advanced method to factor in the contributions of individual group members engaged in an integrated group project using peer assessment procedures. Conway et al. proposed the Individual Weight Factor (IWF) method for peer assessment which has been extensively developed over the years. However, most methods associated with IWF use a simple average algorithm which still has limitations in effectively reflecting abnormal situations. Therefore, this study proposes an iterative method for measuring a fair IWF by considering each assessor's reliability. The proposed algorithm considers each assessor's reliability which is evaluated by an analysis of variance of the scores given by the assessor. This relative reliability factor is used to improve the fairness of the IWF and reduce the effect of unfair or unreliable assessors' scores. A comparison with previous methods demonstrates that the proposed method resulted in substantially more fair grades being given to individual students for group projects.     

Monte Carlo evaluations of methods of grade distribution in group projects: simpler is better

Group projects are widely used in higher education, but they can be problematic if all group members are given the same grade for a project to which they might not have contributed equally. Most scholars recommend addressing these problems by awarding individual grades, computing some kind of individual weighting factor (IWF) from peer and (sometimes) self-assessments, which is then multiplied by the group grade to generate an individual grade. Several variants of the IWF method have been proposed, sometimes with complex algorithms. However, theory suggests they are inaccurate and their accuracy has not been evaluated. This article uses Monte Carlo experiments to assess the accuracy of the original IWF method and variants proposed in the past decade. Findings show that the earlier, simpler methods work best and that self-assessments should definitely be avoided.     

Implementation and outcomes of online self and peer assessment on group based honours research projects

Honours research projects in the School of Civil, Environmental and Mining Engineering at the University of Adelaide are run with small groups of students working with an academic supervisor in a chosen area for one year. The research project is mainly self-directed study, which makes it very difficult to fairly assess the contribution of individual students to a group-based research project. Until recently, a paper-based method of self and peer assessment (SPA) for the honours research projects was used to assess the projects. This was unsatisfactory for both students and academic staff, who found that the same mark was often awarded by the peers to each student without feedback. In 2010, therefore, an online SPA tool, SPARK, together with a set of newly developed assessment criteria, were used for SPA by the honours students. The new SPA criteria are based on teamwork and research skills, and the new rating scales reflect standard academic grades. Surveys of the students confirmed that they found the criteria and scales much more suitable for the assessment of group work and design projects.     

Making a Difference: Using peers to assess individual students' contributions to a group project

Overcoming the potential dilemma of awarding the same grade to a group of students for group work assignments, regardless of the contribution made by each group member, is a problem facing teachers who ask their students to work collaboratively together on assessed group tasks. In this paper, we report on the procedures to factor in the contributions of individual group members engaged in an integrated group project using peer assessment procedures. Our findings demonstrate that the method we used resulted in a substantially wider spread of marks being given to individual students. Almost every student was awarded a numerical score which was higher or lower than a simple group project mark would have been. When these numerical scores were converted into the final letter grades, approximately one-third of the students received a grade for the group project that was different from the grade that they would have received if the same grade had been awarded to all group members. Based on these preliminary findings we conclude that peer assessment can be usefully and meaningfully employed to factor individual contributions into the grades awarded to students engaged in collaborative group work.     

Moderation of peer assessment in group projects

It is shown here that a grade distribution scheme commonly used to moderate peer assessments where self assessment is excluded is based on a false premise and will give an erroneous ranking in the situation where the best performer in a student group ranks the second best performer much higher than the other group members. An alternative to normalisation is proposed based on the idea that the rank order of peer grades should be preserved and should as far as possible be consistent between assessors. It is shown that the method correctly recovers the rank order of students within the group for all cases examined, while still eliminating biases that can result from differences in marking standards in the group. It is suggested that the approach could also be used to check for bias when self assessment is included.     

School entrance age, social acceptance, and self-perceptions in kindergarten and 1st grade

Concerns about the effect of school entrance age have generally focused on academic achievement. The effect of school entrance age on the social acceptance and self-perceptions of kindergarten and 1st-grade students was examined in two studies. In Study 1, the social acceptance and competence of 476 children was assessed in kindergarten and first grade through peer nominations and ratings, teacher ratings, and report card grades. In Study 2, a subgroup of 116 students was interviewed in kindergarten and first grade to assess their perceptions of their school adjustment, loneliness at school, cognitive and physical competence, and peer and maternal acceptance. Few differences were found related to school entrance age. Teachers' ratings and peer nominations generally described initial social problems for the youngest children which were overcome by first grade. There were no differences in self-reported school adjustment, loneliness, perceptions of competence, or acceptance related to school entry age.     

Neurocognitive and Temperamental Systems of Self‐Regulation and Early Adolescents’ Social and Academic Outcomes

ABSTRACT— The aim of the current study was to examine the role of individual differences in neurocognitive and temperamental systems of self‐regulation in early adolescents’ social and academic competence. Measures used in the study included the Attention Network Test, the Early Adolescence Temperament Questionnaire, a peer‐reported Social Status Questionnaire, a self‐reported measure of Schooling Skills, and information on grades obtained by the students in a variety of school subjects (n= 69 12‐year olds). Results showed that efficiency of the neurocognitive network of executive attention is related to academic outcomes, particularly in mathematics, as well as to aspects of social adjustment. Temperamental effortful control appears to be a significant predictor of all dimensions of school competence assessed in this study and mediates the relationship between social adjustment and poor schooling outcomes. These data suggest that individual differences in systems of self‐regulation are central to understanding processes of learning and social adjustment in the school.     

A longitudinal quantitative investigation into the concurrent validity of self and peer assessment applied to English-Chinese bi-directional interpretation in an undergraduate interpreting course

One of the productive lines of research on self-assessment (SA) and peer assessment (PA) concerns their concurrent validity with respect to a criterion measure. However, similar research has rarely been conducted for spoken-language interpreting. This article therefore reports on a longitudinal study that investigated the validity of self and peer ratings on three performance dimensions of English-Chinese consecutive interpretation (i.e., information completeness, fluency of delivery, and target language quality), taking teachers’ ratings as a yardstick. Major findings include: although the students as a group were unable to replicate teachers’ ratings, they were able to rank-order their performances in a fairly accurate manner and improved their SA and PA accuracy over time. Interpreting directionality seems to moderate the correlational strength of self/teacher ratings and peer/teacher ratings. These results are discussed in relation to previous literature, and pedagogical suggestions are provided to improve SA and PA for bi-directional interpretation.     

Psychosocial development in girls of high ability

Gender differences in young highly able children's psychosocial development were investigated using child and teacher ratings of prosocial behavior and peer acceptance. Developmental patterns were addressed by studying two groups – children in the primary grades (1 and 2) and children in grades 3 to 7. No gender differences were found on teacher or child ratings in the younger group. In the older group, gender differences favoring girls were found on teacher ratings of prosocial behavior and peer acceptance. Comparisons of child and teacher ratings by gender showed that the older boys viewed themselves as demonstrating more social skills and as being more accepted by peers than their teachers did. Implications for education and counselling are discussed.     

A robust approach for mapping group marks to individual marks using peer assessment

Group work can form a substantial component of degree programme assessments. To satisfy institutional and student expectations, students must often be assigned individual marks for their contributions to the group project, typically by mapping a single holistic group mark to individual marks using peer assessment scores. Since the early 1990s, various mapping methods that use self- and peer ratings have been developed. They are based on (normalised) individual weighting factors, partial scaling of the group mark, inter-rater agreement corrections or parabolic functions. We show that no single existing method can be successfully applied to most practical peer assessment scenarios such as different marking scale interpretations, intra-group ranking errors, biased free-riders and marks exceeding 100%. We present a combined analytical mapping method that incorporates the benefits of existing mechanisms, but alleviates their weaknesses with minimum computational effort and tutor input. The robustness of the method is illustrated through problematic assessment examples and empirically evaluated in multiple group work environments involving a total of 243 students and five different disciplines.     

Assessing contributions to group assignments

We report the use of a combination of self‐ and peer‐assessment in an undergraduate social psychology laboratory course. Students worked in small groups on a self‐directed empirical project that they each wrote up independently as a laboratory report. Marks for the written assignment were moderated by a contribution index measure based on the self‐ and peer‐assessment measures. Our analyses indicated that: (i) students took the peer‐assessment process seriously, clearly differentiating between group members on the contributions questionnaires; (ii) students show a self‐bias, rating their own contribution to the group task higher than that of other group members; (iii) for a large majority of students the contribution index resulted in very little moderation of the final assignment marks; (iv) there was a strong correlation between the contribution index and the overall assignment score. Implications for the assessment of group work are considered.     

Emotions and Behaviors in the Head Start Classroom: Associations Among Observed Dysregulation, Social Competence, and Preschool Adjustment

Effective regulatory skills are essential in busy preschool classroom environments where children must maintain some control over their emotions and behavior to interact effectively with peers and teachers. Regulatory abilities can play a crucial role in a child's successful adjustment to preschool. We investigated whether individual differences in dysregulation (emotional and behavioral) as observed in the naturalistic classroom context were associated with peer social competence and teacher ratings of classroom adjustment in a sample of low-income preschoolers. Naturalistic observational methods were used to assess dysregulated emotions and behaviors in Head Start classrooms. Findings demonstrate that although displays of observed dysregulation were relatively brief, about one-quarter of children showed high levels of dysregulation, and individual differences in dysregulated behavior predicted teacher-rated classroom adjustment and peer conflict. Research results are discussed with regard to implications for classroom practice and prevention.     

The peer group as a context: mediating and moderating effects on relations between academic achievement and social functioning in Chinese children

The purpose of the study was to examine, in a large sample of Chinese children and adolescents at 9, 13, and 16 years of age, the contextual effects of the peer group on relations between academic achievement and social functioning. Data on informal peer groups, social functioning and academic achievement were collected from multiple sources. It was found that peer groups were highly homogenous on academic achievement. Hierarchical linear modeling analysis revealed that academic achievement and social adjustment were associated at both the within-group individual level and the group level. Moreover, group academic performance moderated the relations between academic achievement and social adjustment such as peer acceptance, social competence, and leadership, suggesting that individual-level relations might be enhanced or exacerbated by group academic norms.     

The architecture of peer assessment: do academically successful students make good teammates in design assignments?

This paper considers the relationship between architecture and construction management students’ overall academic abilities (as measured by Weighted Average Marks [WAMs]), their peer ratings for contributions to team design assignments (as measured by an online Self-and-Peer-Assessment [SAPA] tool), and their specific abilities as building designers (as measured by grades in individual design assignments). The research was conducted to determine whether a student’s prior academic achievements might indicate how well they will work in teams. The research demonstrates a statistically significant relationship between WAMs and SAPA ratings indicating that academically successful students more often than not make good teammates. However, the study also highlights that when peers are assessing contributions to teamwork they are assessing skills and qualities in their teammates other than overall academic ability or the ability to design well. Whilst this study is largely located within the field of design, the findings are relevant to any group work where teachers aim to design assessment that unravels group and individual contribution.     

Peer assessment in a project-based engineering course: comparing between on-campus and online learning environments

As the number of participants in online distance learning courses increases, peer assessment is becoming a popular strategy for evaluating open assignments and for breaking the social isolation surrounding distance education. Yet, the quality and characteristics of peer assessment in massive online courses has received little attention. Hence, this study was set to examine peer feedback quality and grading accuracy in a project-based course. The study applied a sequential exploratory mixed methods design. It included 339 participants who studied the same engineering course, but in three different modes: on-campus (n = 77), small private online course (n = 110), and massive open online course (MOOC) (n = 152). Content analysis of feedback comments identified four categories: reinforcement, statement, verification and elaboration, arranged in an ascending scale of cognitive ability. The findings indicated that the MOOC participants provided more feedback comments and volunteered to assess more projects than their counterparts did. However, the on-campus students provided higher quality feedback and their peer grading was better correlated with the grades assigned by the teaching assistants.     

Learning to lead: Self‐ and peer evaluation of team leaders in the human structure didactic block

Increasing emphasis on leadership in medical education has created a need for developing accurate evaluations of team leaders. Our study aimed to compare the accuracy of self‐ and peer evaluation of student leaders in the first‐year Human Structure block (integrated gross anatomy, embryology, and radiology). Forty‐nine first‐year medical students at Mayo Medical School were assigned to learning teams of three or four members. Teams worked together on daily laboratory dissection, clinical projects, embryology presentations, and daily group quizzes. Student team leaders were responsible for leading laboratory dissection, reviewing radiographic findings, and organizing group assignments. Weekly electronic surveys were administered to evaluate team leaders on altruism, compassion, respect, integrity, responsibility, commitment to excellence, and self‐reflection. Results demonstrated that team leaders rated themselves lower than their peers rated them in multiple aspects of leadership. Peer evaluation of team leaders was statistically higher than self‐evaluation in all traits measured except respect. Female leaders were rated higher by their peers in the areas of responsibility and self‐reflection compared to male leaders. This study demonstrates the need for increased communication between team leaders and members, along with creation of a mutually respectful environment, to improve leader awareness of their abilities and foster team success. Anat Sci Educ 2:210–217, 2009. © 2009 American Association of Anatomists.