Peer assessment of oral presentations: effects of student gender,university affiliation and participation in the development of assessment criteria

Peer assessment provides a useful mechanism to develop many positive qualities in students studying in higher education (HE). Potential influences on peer‐awarded marks include student qualities such as gender, HE background (e.g. university affiliation) and participation in the development of the assessment criteria. Many studies that have investigated peer assessment have placed great emphasis on marks from a single tutor, or very few tutors, from a single university. This study examined grades awarded by 11 tutors (affiliated with four universities) to oral presentations delivered on a residential field course by second‐year undergraduate students from two universities studying environmental or biological disciplines. Student assessors awarded marks of fairly high precision (correlating strongly with tutor grades) but averaged 5% higher than their tutors (i.e. of only moderate accuracy). Marginally higher marks (circa 1.6%) were awarded by student assessors to speakers studying at the same university. Gender influences were detected as males tended to grade other male speakers very slightly more highly than female speakers. Marks from females were unaffected by speaker gender. Students who participated in the development of the assessment criteria did not achieve higher grades for their presentations. However, when these ‘participants’ were assessing, they awarded lower marks than their peers (i.e. closer to, but not as low as, those awarded by tutors). Lower marks were also awarded during the middle of sessions, possibly resulting from factors associated with motivation and attention of speakers and markers. Overall, the potential biases in marking by naive assessors examined in this study may influence the validity of marks generated by peer assessment schemes, but the experience of this type of assessment had positive effects on those involved.     

Relationships between student characteristics and self‐, peer and tutor evaluations of oral presentations

There are many influences on how assessors grade themselves and others. Oral presentations are useful for exploring such factors in peer, self‐ and tutor marked assessments, being rapidly completed and assessed, commonly used in HE and very difficult to anonymize. This opportunistic study examined the effects of gender and level of attainment on the triangulation of marks awarded to student presenters. Grades generated by peer assessment were associated more strongly with tutor‐awarded marks than those from self‐assessment. For self‐assessment there was a strong effect of gender (female students undervalued their performance compared with tutor grades). Peer assessment produced higher marks than from tutors, perhaps because of the close‐knit community developed during residential courses. For tutor marks, the greatest variability was at the lower end of the scale, whereas peer assessors were most variable when marking students who self‐evaluated or peer assessed highly. Students awarded a narrower range of marks to peers compared with tutors, but when self‐assessing used a larger range. Presentations by students who admitted to little sleep the night before received lower grades from both peers and tutors, but this was not reflected by self‐assessments, suggesting they were unaware of their poorer performances. Sessions with fewer talks (four rather than seven) reduced the ‘dip’ in marks previously observed in the middle of sessions. Findings are discussed in the context of bias in this mode of assessment.     

Making a Difference: Using peers to assess individual students' contributions to a group project

Overcoming the potential dilemma of awarding the same grade to a group of students for group work assignments, regardless of the contribution made by each group member, is a problem facing teachers who ask their students to work collaboratively together on assessed group tasks. In this paper, we report on the procedures to factor in the contributions of individual group members engaged in an integrated group project using peer assessment procedures. Our findings demonstrate that the method we used resulted in a substantially wider spread of marks being given to individual students. Almost every student was awarded a numerical score which was higher or lower than a simple group project mark would have been. When these numerical scores were converted into the final letter grades, approximately one-third of the students received a grade for the group project that was different from the grade that they would have received if the same grade had been awarded to all group members. Based on these preliminary findings we conclude that peer assessment can be usefully and meaningfully employed to factor individual contributions into the grades awarded to students engaged in collaborative group work.     

Implementation and outcomes of online self and peer assessment on group based honours research projects

Honours research projects in the School of Civil, Environmental and Mining Engineering at the University of Adelaide are run with small groups of students working with an academic supervisor in a chosen area for one year. The research project is mainly self-directed study, which makes it very difficult to fairly assess the contribution of individual students to a group-based research project. Until recently, a paper-based method of self and peer assessment (SPA) for the honours research projects was used to assess the projects. This was unsatisfactory for both students and academic staff, who found that the same mark was often awarded by the peers to each student without feedback. In 2010, therefore, an online SPA tool, SPARK, together with a set of newly developed assessment criteria, were used for SPA by the honours students. The new SPA criteria are based on teamwork and research skills, and the new rating scales reflect standard academic grades. Surveys of the students confirmed that they found the criteria and scales much more suitable for the assessment of group work and design projects.     

Evaluating student assessments: the use of optimal foraging theory

The concepts of optimal foraging theory and the marginal value theorem are used to investigate possible student behaviour in accruing marks in various forms of assessment. The ideas of predator energy consumption, handling and search times can be evaluated in terms of student behaviour and gaining marks or ‘attainment’. These ideas can be used to examine student responses to dealing with assessments by examining a marks awarded/time-on-task curve. The non-linear, cumulative mark gain, as a Gompertz function, has implications for how students tackle continuously assessed projects as well as examination questions. The attainment of a student can be viewed in these general terms, as well as in specific aspects such as question ‘difficulty’ and mark gain in an examination answer. Prospect theory, from econometrics and psychology, can also be used to suggest ways in which students might tackle problems in examinations. The implications of this analysis are considered with respect to setting questions, criterion referencing of assignments and dealing with ‘troublesome knowledge’. The ideas can also be used to assist problems regarding mark fidelity and integrity as well as mark comparability.     

Using statement banks to return online feedback: limitations of the transmission approach in a credit-bearing assessment

Electronic marking tools that incorporate statement banks have become increasingly prevalent within higher education. In an experiment, printed and emailed feedback was returned to 243 first-year students on a credit-bearing laboratory report assessment. A transmission approach was used, students being provided with comments on their work, but no guidance as to how they should use these remarks. A multiple-choice question test, undertaken before and after the return of feedback, was used to measure learning. Although returned comments included model answers, test scores showed no overall enhancement, even when students’ marks for their laboratory reports were initially hidden. A negative and significant (p = .010) linear trend between relative test scores and test date suggests that even modest improvements in subject knowledge were lost over time. Despite this, students could accurately guess their mark based on emailed feedback alone, estimated and awarded marks being linearly correlated (p < .001). It is concluded that statement banks organised according to published assessment criteria can ultimately help students to appreciate how their work was graded. However, students should be encouraged to produce a structured response to received feedback if self-assessment is to occur.     

Sozialverhalten und Schulleistungen

Prior research indicates that social behavior contributes to school achievement. The underlying mechanisms, however, have received little research attention to date. To investigate peer acceptance as mediating the influences of prosocial and antisocial behavior on school grades, this prospective study draws on 2387 ninth graders. All students in a class rated their peers in terms of prosocial and antisocial behavior, class teachers rated their students’ peer acceptance in class, and grades in German and mathematics were collected from students’ report cards. Prior report card grades, gender and socioeconomic status were controlled for. Structural equation modeling shows direct positive paths between prosocial behavior and both peer acceptance and grades; antisocial behavior negatively predicted both peer acceptance and grades. Prosocial behavior contributed to better grades via higher peer acceptance, whereas concerning antisocial behavior, the mediating effect was statistically not significant. Results are discussed with regard to their practical relevance and implications for future research.     

Quantifying participation in,and the effectiveness of,remediating assessment in a university mathematics module

In Maths for Business, a large first-year mathematics module, the continuous assessment component comprises 10 weekly quizzes which combine to contribute 40% of the final module mark. If students did not receive the full five marks on their weekly quiz, they were provided with the opportunity to resubmit their corrected weekly quiz with an explanation of their error(s) for one additional mark. We refer to this process as ‘remediation’. Of the students who had the opportunity to remediate, ～70% did. Through examining learning management system data, we show that the remediation process encouraged students to access module resources. Furthermore, by using a Bayesian hierarchical model to account for students’ level of participation, achievement and prior knowledge, we show that participation in the remediation process positively impacted the final examination marks of moderate to high-achieving students (based on initial continuous assessment marks). However, participation in the remediation process provided limited benefit to low-achieving students. We conjecture this is because these students had not achieved a level of understanding whereby participation in the remediation process could progress their knowledge.     

Student explanations of their science teachers’ assessments,grading practices and how they learn science

The current paper draws on data generated through group interviews with students who were involved in a larger ethnographic research project performed in three science classrooms. The purpose of the study from which this data was generated, was to understand science teachers’ assessment practices in an upper-secondary school in Sweden. During group interviews students were asked about their conceptions of what were the assessment priority of teachers, why the students were silent during lecturing and their experiences regarding peer- and self-assessments. The research design and analysis of the findings derives from what students told us about their assessments and learning sciences experiences. Students related that besides the results of the written test, they do not know what else teachers assessed and used to determine their grades. It was also found that students did not participate in the discussion on science because of peer-pressure and a fear of disappointing their peers. Student silence is also linked with student conceptions of science learning and student experiences with methodologies of teaching and learning sciences.     

Assessing ‘inexperienced’ students' ability to self‐assess: exploring links with learning style and academic personal control

The study sought to establish the level of students' self‐assessment skill—particularly inexperienced students—and to examine the relationship between self‐assessment skill and learning style, student perceptions of academic locus of control and academic self‐efficacy. Students were asked to evaluate and provide estimated marks for their own work, were which compared with tutors' actual marks. Students also completed measures of learning style, academic locus control and academic self‐efficacy. Comparisons of student estimated and tutor marks indicated a good level of self‐assessment skill in the majority of students. A significant minority of students did however fail to exhibit such skills. There was also some evidence of a tendency for students to underestimate their performance. While both strategic and deep approaches to learning were shown to be positively correlated with tutor mark, only surface approach was negatively correlated with students' estimated mark, suggesting that surface learners are inclined to provide lower evaluations of their own performance. Deep approach was also correlated with accuracy of student self‐assessment skill, suggesting that deep learning is associated with self‐assessment competency. No clear or convincing associations between self‐assessment skill and perceptions of academic locus of control or academic self‐efficacy were identified. Findings suggest that while self‐assessment skill undoubtedly develops, becoming more effective during students' academic career, inexperienced students do have the capacity for self‐evaluation and should therefore be included in self‐assessment activities. In the light of findings related to learning style and the heterogeneous nature of student groups, student monitoring and skill development are proposed in order to allow the integration of self‐assessment into the learning and assessment process.     

Disciplinary differences and other variations in assessment cultures in higher education: exploring variability and inconsistencies in one university in England

This article argues that differing disciplinary assessment cultures are likely to be an important factor in explaining differences in student marks and grades both within and between higher education institutions. Using institution-wide data on undergraduate student marks over the last five years in one UK higher education institution we demonstrate variability in the distribution of marks in terms of the ‘distance travelled’. This issue was further explored via interviews with senior teaching-active staff. We suggest that the distribution of marks is likely to reflect different disciplinary assessment cultures as well as complexity in the process of marking and assessment. These findings signify that it will be highly challenging, if not impossible, to establish nationally comparable learning gain measures using student mark data because of the underlying inconsistencies in the process of awarding marks. In the current higher education context, with the ongoing implementation of the Teaching Excellence Framework, it remains important to debate and further investigate these issues with all stakeholders, including students.     

Peer and Teacher Assessment of the Oral and Written Tasks of a Group Project

Undergraduate students, and their class teachers, assessed the performance of their peers in three oral and written tasks as part of a group project. The two sets of marks awarded by peers and teachers were subsequently compared to find out whether the students were competent to assess their peers alongside their class teachers and whether this competence, or lack of it, was partly determined by the nature of the task being assessed. A number of statistical tests were run to establish the levels of agreement, the ranges, differences and relationship between peer and teacher assessments. The results have led us to conclude that the peer assessments are not sufficiently reliable to be used to supplement teacher assessments. Students’ competencies in peer assessment do not appear to be dependent on the nature of the task being assessed, but there is some evidence that practical experience of assessing a particular task type can lead to an improvement in students’ assessment skills when they assess a similar task. The paper also discusses possible improvements in peer assessment procedures based on the experiences gained.     

Teachers' Assessment Practices: Preparation,Isolation, and the Kitchen Sink

A sample of 143 midwestern elementary and secondary school teachers from a variety of practice settings responded to a survey and provided comments regarding their assessment practices The purpose of the survey was to collect background (demographic) information on the teachers and information on several assessment-related practices, including frequency with which teachers assign routine class assignments, types of marks used to report student performance, frequency and grading of major assignments and tests, source of classroom tests, kinds of marks used, methods used to combine marks, meaning of grades, teachers' knowledge and perceptions regarding district grading policies, and teachers' awareness of the grading policies of their peers. Interviews with the teachers provided additional insights into their practices. Results indicated that teachers' assessment practices were highly variable and unpredictable from characteristics such as practice setting, gender, years of experience, grade level, or familiarity with assessment policies in their school district. Teachers generally claim to consider and incorporate a variety of objective and subjective factors when assigning grades on assignments, assessments, and report cards, synthesizing diverse kinds of information about achievement in ways that tend to maximize the likelihood that students will achieve high grades. Only about one half of the teachers surveyed indicated that they were aware of their districts' policies on grading; most were not aware of the assessment practices of their colleagues. Many teachers seemed to have individual assessment policies that reflected their own individualistic values and beliefs about teaching. Recommendations for making grades more meaningful ways of communicating about student performance are suggested.     

Group in‐course assessment promotes cooperative learning and increases performance

The authors describe and evaluate a method to motivate medical students to maximize the effectiveness of dissection opportunities by using In‐Course‐Assessments (ICAs) to encourage teamwork. A student's final mark was derived by combining the group dissection mark, group mark for questions, and their individual question mark. An analysis of the impact of the ICA was performed by comparing end of module practical summative marks in student cohorts who had, or had not, participated in the ICAs. Summative marks were compared by two‐way ANOVA followed by Dunnets test, or by repeated measures ANOVA, as appropriate. A cohort of medical students was selected that had experienced both practical classes without (year one) and with the new ICA structure (year two). Comparison of summative year one and year two marks illustrated an increased improvement in year two performance in this cohort. A significant increase was also noted when comparing this cohort with five preceding year two cohorts who had not experienced the ICAs (P <0.0001). To ensure that variation in the practical summative examination was not impacting on the data, a comparison was made between three cohorts who had performed the same summative examination. Results show that students who had undertook weekly ICAs showed significantly improved summative marks, compared with those who did not (P <0.0001). This approach to ICA promotes engagement with learning resources in an active, team‐based, cooperative learning environment. Anat Sci Educ 7: 224–233. © 2013 American Association of Anatomists.     

Curriculum‐based Continuous Assessment: a framework for concepts,procedures and policy

To many persons used to the ‘once and for all’ nature of leaving examinations for certification and selection of students, the concepts associated with continuous assessment (CA) are confusing. To some, CA is a summative ‘mark’ to be passed forward to certify or select a student. To others, CA is the physical pages in exercise books which students complete. To yet others, CA is diagnosis and formative evaluation of student learning. This paper presents a conceptual framework for organising and relating the many confusing concepts associated with continuous assessment practices. Curriculum‐based continuous assessment is shown to have two major components: formative and summative continuous assessment of student learning. The nature of these components and their interrelationships are discussed. The paper discusses using continuous assessment results for official summative evaluation purposes such as reporting to parents and incorporating continuous assessment grades into leaving and certification decisions. Among the issues discussed are: (a) weighting of CA marks; (b) record‐keeping and reporting of CA results; (c) in‐service and pre‐service teacher training in CA; and (d) building school‐to‐school comparability and credibility into the CA process.     

Detecting Gender Bias in Peer Marking of Students’ Group Process Work

Concerns relating to the reliability of teacher and student peer assessments are discussed, and some correlational analyses comparing student and teacher marks described. The benefits of the use of multiple ratings are elaborated. The distinction between gender differences and gender bias is drawn, and some studies which have reported gender bias are reviewed. The issue of ‘blind marking’ is addressed. A technique for detecting gender bias in cases where student raters have awarded marks to same and opposite sex peers is described, and illustrated by data from two case studies. Effect sizes were found to be very small, indicating an absence of gender bias in these two cases. Results are discussed in relation to task and other contextual variables. The authors conclude that the technique described can contribute to the good practice necessary to ensure the success of peer assessment in terms of pedagogical benefits and reliable and fair marking outcomes.     

Dancing with the devil: formative peer assessment and academic performance

Peer assessment can be important in developing active and independent learners, as well as providing more and faster feedback in large classes, compared to marking done by tutors. In addition, the evaluative, critical stance required by students in order to assess their peers' work encourages the development of higher-order cognitive skills. Changing roles from being assessed to being an assessor can also improve students' ability to judge and improve on their own work. However, peer assessment does have potential problems and there is some debate as to the appropriate academic level at which to implement it, the kinds of feedback that are given and the ways in which students respond. In addition, there is little evidence that peer assessment has an impact on academic performance. This research reports the results of an online peer assessment exercise for a macroeconomics essay conducted in a large Economics 1 class at Rhodes University. Of the 800 students, about half participated in the peer assessment exercise. Data were collected from students via a formal course evaluation. In addition, a sample of 50 essays was evaluated in terms of the relationship between peer marks and final (tutor) marks received and the impact that peer assessment had on the quality of the final essay submitted. An Ordinary Least Squares regression was used to investigate the impact of peer assessment participation on marks. Results showed that peer marks tended to ‘bunch’ in the 60–68% range, indicating the reluctance of peers to give very high or low marks. In general, peers gave more useful feedback on technical aspects, such as presentation and referencing (which were also the categories in which students most often made improvements), than on content. Regression analysis showed that peer assessment participation was not a significant determinant of final essay mark, but that economics ability and English language proficiency were.     

Self‐assessment and the disadvantaged student: potential for encouraging self‐regulated learning?

The Science Foundation Programme (SFP) at the University of KwaZulu Natal, Pietermaritzburg attempts to address past educational inequalities by providing disadvantaged matriculants with the skills, resources and self‐confidence needed to embark on their tertiary studies. Students entering the Programme typically adopt a surface approach to learning with emphasis being placed on high score achievement which results in a mark‐driven attitude towards assessment. Students also lack the metacognitive skills associated with a deep approach to learning. Within this mark‐driven culture, it is important to attempt to move students away from such a superficial approach to learning and assessment. Worldwide, self‐assessment practice has been gaining recognition, and it has been linked to the adoption of a deep approach to learning; self‐regulated learning and the development of metacognitive skills. In the biology module of the Programme, students are given two essay assignments, tasks that are routinely performed very poorly. In attempt to improve SFP students’ essay writing abilities, a self‐assessment scheme to accompany the essay tasks was instituted, hoping that this would get the students to engage with the assessment criteria. It was intended that students would not only fulfil the requirements of the task better, but also achieve the valuable skill of self‐assessment. The results provide overwhelming evidence that SFP students cannot accurately self‐assess. Their naivety and inexperience in fulfilling assessment criteria was also revealed. Overall, marks awarded by staff were significantly lower than those awarded by students, and there was no evidence to suggest that any one category of students based on academic ability were better able to self‐assess than another. Marker inconsistencies were revealed, but this was shown to have no effect on findings. In spite of these results, a case for pursuing self‐assessment is made and proposals for improving student self‐assessment practice in the SFP are suggested.     

Social acceptance of students with Down syndrome and students without disability

We investigated the influence of teacher feedback on the social acceptance of peers with intellectual disabilities and peers without disabilities. A computer task was administered to 601 students in grades 3 and 4. Twenty-six per cent of the students attend an inclusive school; the others are in regular schools without students with special educational needs. Participants are introduced to ‘new’ virtual classmates, one student with Down syndrome (DS), and one control student with no obvious disability. Additionally, teacher feedback and feedback about fun playing with the new classmates is given. Social acceptance is evaluated by asking if one would like to sit next to him/her. Both feedbacks showed a strong effect. The child with DS was less socially accepted than the child without disability. No difference regarding the social acceptance of the students with DS was found between students from inclusive and regular classes. Students from regular classes rate the social acceptance of the student without disabilities significantly higher than students from inclusive classrooms.