Developing Procedures for Implementing Peer Assessment in Large Classes Using an Action Research Process

Peer assessment has been used successfully in higher education, with important benefits reported in terms of student learning. However, most of the literature has focused on its use with small groups of students taught by staff who are committed to the peer assessment process. This paper reports the development of peer assessment procedures for use in large classes, using a cyclical process of action, reflection and refined action. The project was carried out in three phases and after each phase changes were made to the procedures in response to student and staff feedback. The development of procedures is discussed in relation to assessment tasks, assessment criteria, anonymity, procedural guidelines, distribution systems, marking procedures and tutor remarking. Although there are specific difficulties associated with the use of peer assessment in large classes, this study suggests that these are outweighed by the learning benefits for students. Based on the findings of this study, recommendations are made for ways in which peer assessment might be successfully applied in large classes.     

SPARK, a confidential web–based template for self and peer assessment of student teamwork: benefits of evaluating across different subjects

Students often enjoy learning in teams and developing teamwork skills, but criticise team assessment as unfair if there is equal reward for unequal contributions. This paper describes the design, implementation and evaluation in four subjects of the Self and Peer Assessment Resource Kit (SPARK), a web–based template which aims to improve learning from team assessment tasks and make the assessment fairer for students. Students benefit because the web–based template improves confidentiality and the potential for accurate assessment of relative contributions. Academics benefit through the potential for improving student learning from teamwork tasks, and saving time by automating the process of calculating self and peer adjustments of assessment grades, especially attractive for large enrolments. Benefits accrue to the institution and wider academic community because the template suits a range of group assessment situations. Based on experiences gained over five years of developing, evaluating and implementing SPARK, this paper aims to illustrate the potential benefits of the template to potential users and more critically, to use what was learned from implementing the template across a range of subjects to alert others to key issues for evaluating and disseminating educational technology innovations.     

Feedback,feedforward: evaluating the effectiveness of an oral peer review exercise amongst postgraduate students

Assessment for learning approaches, such as peer review exercises may improve student performance in summative assessments and increase their satisfaction with assessment practices. We conducted a mixed methods study to evaluate the effectiveness of an oral peer review exercise among post-graduate students. We examined: (1) final assessment grades among students who did and did not take part in the peer review exercise; (2) student perceptions of the impact of the peer review exercise; and (3) student understanding of, and satisfaction with, this new assessment practice. We found that students who took part in the exercise had a significantly higher mean grade in a subsequent summative oral presentation assessment than students who did not take part in the exercise. Students gained a better understanding of assessment and marking criteria and expressed increased confidence and decreased anxiety about completing the subsequent summative assessment. Assessment for learning improves academic attainment and the learning experience in postgraduate students.     

Examining Student Satisfaction with Group Projects and Peer Assessment

The efficiency and effectiveness of learning outcomes in groupwork and in peer assessment is well attested in the higher education literature. However, there is little evidence that any quantitative measures of student satisfaction have been undertaken with the group process and peer assessment. In this paper a peer assessment method is explained and a study detailed which was undertaken on a cohort of 261 students to measure student satisfaction of the assessment process. It was found that there were high levels of student satisfaction with groupwork and in the adopted assessment method. Further analysis revealed that there were no effects on the levels of satisfaction of students with the independent variables gender and age. Yet, there was a fairly significant difference in the levels of satisfaction of students having work experience favouring those without. Finally, there was found to be a substantial difference in the levels of satisfaction between Australian and international students with international students expressing higher values. It is recommended that this study be extended to other subjects and disciplines.     

But is it fair? Developing students’ understanding of grading complex written work through peer assessment

Peer assessment of long written tasks poses particular problems as these tasks typically involve complex learning and solving ill‐structured problems which require divergent responses. Marking reliability of this kind of writing task is difficult to achieve. The author illustrates this through an evaluation of two implementations of peer assessment, involving 81 students, in a UK university. In these implementations, all peer assessor grades were returned to students (not just mean grades). In this way students were exposed to subjectivity in marking. The implementations were evaluated through questionnaires, focus groups, observations of lectures and tutor interview. While students reported a better understanding of quality in student writing as a result of their experience, many complained that peer assessors’ marks were not ‘fair’. The article draws on recent research on the reliability of tutor marking to argue that marking judgements are subjective and that peer assessment offers the opportunity to explore subjectivity in marking, creating an opportunity for dialogue between tutors and students.     

Calibrating assessment literacy through benchmarking tasks

Abstract

In calibration tasks students assess exemplar texts using criteria against which their own work will be assessed. Typically, these tasks are used in the context of training for peer assessment. Little research has been conducted on the benefits of calibration tasks, such as benchmarking, as learning opportunities in their own right. This paper examines a dataset from a long-running benchmarking task (～500 students per semester, for four semesters). We investigate the relationship of benchmarking performance to other student outcomes, including ability to self-assess accurately. We show that students who complete the benchmarking perform better, that there is a relationship between benchmarking performance and self-assessment performance, and that students appreciate the support for learning that benchmarking tasks provide. We discuss implications for teaching and learning flagging the potential of calibration tasks as an under-explored tool.     

Collaborative learning and teaching: Reflections on a facilitated journey

Abstract

This report summarises the first year of an innovative and multilayered teaching and learning collaborative experience within the context of an undergraduate unit in early childhood. The facilitative influence is two‐fold: (a) generic information skills training (GIST) project which is a joint initiative between the Institute of Early Childhood (IEC), the Library and the Centre for Flexible Learning to teach information literacy skills which are contextualised within the unit presentation mode, its requirements and assessment tasks, and (b) redesign of the unit from a former third‐year level to a first‐year level to incorporate the notion of problem‐based, collaborative teaching and learning. Many challenges were faced by those involved in the unit in its first year, mainly class size and heterogeneity of student levels, incorporation of technology into learning and teaching, and flow of information for smooth operations for students, the teaching team and Library staff. Multilevel collaborations were central to this experience, and examples of student‐student, student‐teacher, and teacher‐staff collaborations are provided. The first of the 2‐year cycle of the GIST project has generated substantive data, as has the implementation of the unit in an innovative format. The complexity and challenges presented in the unit seemed to account for student withdrawal of about 25% in the first month; final evaluations are obtained from 292 students. Survey results presented here are both from the withdrawn students, as well as those who completed the unit. Modifications to the delivery of the unit in its second year and ways of strengthening the communities of learners and teachers towards a common goal are briefly reviewed.     

The impact of two types of peer assessment on students' performance and satisfaction within a Wiki environment

The purpose of this study was to compare the effects of two peer assessment methods on university students' academic writing performance and their satisfaction with peer assessment. This study also examined the validity and reliability of student generated assessment scores. Two hundred and thirty-two predominantly undergraduate students were selected by convenience sampling during the fall semester of 2007. The results indicate that students in the experimental group demonstrated greater improvement in their writing than those in the comparison group, and the findings reveal that students in the experimental group exhibited higher levels of satisfaction with the peer assessment method both in peer assessment structure and peer feedback than those in the comparison group. Additionally, the findings indicate that the validity and reliability of student generated rating scores were extremely high. Using Wiki interactive software and providing an online collaborative learning environment to facilitate peer assessment added value to peer assessment.     

Introducing a scalable peer feedback system for learning teams

In the current research, we report on a peer feedback system for supporting student teamwork skill development during post-secondary education. The peer feedback system is part of a larger suite of assessments at ITPmetrics.com. This is a free assessment-based system that allows team members to provide round-robin ratings of each member’s effectiveness in the team on five dimensions (communication; commitment; foundation of knowledge, skills and abilities; emphasising high standards; and focus), as well as provide anonymous written feedback to supplement the numeric scores. Team members have access to a dashboard where they can complete the assessment and store assessment reports generated from their peers’ ratings. We summarise data from student learning teams suggesting that peer ratings are reliable (internally consistent, unidimensional and inter-rater correlated; n?>?85,000). Recommendations for implementation are provided with respect to pre-briefing, debriefing, and assessment schedules.     

Student attitudes to blogs: a case study of reflective and collaborative learning

The aim of this study is to evaluate the capacity of an e-learning tool (blog) to facilitate reflection among students as part of collaborative group learning. The paper provides insights into student attitudes towards blogs as an interactive and reflective learning tool. Additionally, the study highlights the differences between domestic and international student attitudes towards reflection as part of collaborative learning in groups. The results indicate that international students consistently viewed the use of e-learning tools more positively than domestic students. For educators, the results provide direction in terms of enhancements to assessment tasks incorporating e-learning, including the professional development of staff to address the rapidly changing learning environment.     

Peer versus staff tutoring in problem-based learning

Effects of student versus staff tutoring on student learning in a problem-based, health sciences curriculum were studied. Academic achievement of 334 tutorial groups guided by staff tutors was compared with achievement of 400 groups guided by student tutors. In addition, students rated their tutor's performance on four behaviors considered critical to facilitating student learning. Overall, students guided by a staff tutor achieved somewhat better. In terms of practical significance, the difference was, however, fairly small. Staff tutors were rated as more knowledgeable and their contributions as more relevant. In addition, they asked stimulating questions to a larger extent. However, an interaction effect was found between the ratings and the year of study: Peer tutors displayed the supportive behaviors more extensively in the first year, whereas staff tutors' ratings were higher as the curriculum advanced. These results were interpreted in terms of the cognitive congruence framework.Parts of this article have been presented to the Annual Meeting of the American Educational Research Association, Atlanta, GA, April, 1993.     

The effect of peer assessment on project performance of students at different learning levels

Peer assessment has been increasingly integrated in educational settings as a strategy to foster student learning. Yet little has been studied about how students at different learning levels may benefit from peer assessment. This study examined how peer-assessment and students’ learning levels influenced students’ project performance using a two-way factorial design. One hundred and thirty teacher education students participated in this quasi-experimental study. When working on a technology-integrated lesson plan project, the experimental group completed an online peer assessment process while the control group followed the discussion method. Students’ learning levels were measured and divided into low, average and high achieving according to the quality of their draft lesson plans. Data analysis suggested that the impact of peer assessment on students’ lesson plan project seemed to vary according to students’ learning levels. While low- and average-achieving students showed significantly improved performance right after the integration of a peer assessment model, the model seemed to have had less impact on the performance of high-achieving students. Significance, implications and limitations of findings are discussed.     

Introduction

Abstract

Drawing on a large government commissioned research study, the Queensland School Reform Longitudinal Study (QSRLS), the article confirms the existing research finding regarding the cenerality of teachers’ classroom practices (pedagogies and assessment) to student learning. On the basis of a literature review and classroom observations, analysis of assessment tasks and student work, the article identifies the models of productive pedagogies and productive assessment as being effective in terms of improving both social and academic outcomes for all students, especially students from disadvantaged backgrounds. In so doing, it stresses the need for aligning curriculum, pedagogies and assessment with desired student outcomes and also the need to create teacher professional learning communities so as to enhance whole school effects. However, the article also recognises the significance of family background in terms of students’ learning outcomes and thus argues the need for complementary and appropriate educational system funding and policies. Indeed, the article recognises that claims about teacher practices making a difference cannot be posed innocently and that a focus on individual teachers in policy is a double‐edged sword.     

The impact of stress in self‐ and peer assessment

While a large amount of interest has been shown in the use of peer and self‐assessment, few studies have considered the effect of stress on the students involved. None have considered whether the resultant stress itself might account for any noticeable improvements in student performance. The research presented in this paper addresses this question. An experimental design measured the effects of type of assessment and gender on student stress levels and performance. Results suggest that females are more stressed by self‐assessment than males and that being subjected to self‐ and peer assessment, while more stressful, leads to improved student performance in summative tasks.     

Assessment by Interview and Portfolio in a Graduate School Program

Assessment plays an integral role in teaching and learning in higher education and teachers have a strong interest in debates and commentaries on assessment as and for learning. In a 1-year graduate entry teacher preparation program, the temptation is to emphasize assessment in an attempt to ensure students “cover” everything as part of a robust preparation for the profession. The risk is that, for students, assessment drives curriculum, and time spent in the completion of assignments is no guarantee of either effective learning or authentic preparation for teaching. Interviews as assessment provide an opportunity for a learning experience as well as an authentic task, since students will shortly be interviewing for employment in a “real world” situation. This paper reports on a project experimenting with interview panels as authentic assessment with preservice early childhood teachers. At the end of their first semester of study, students enrolled in the Graduate Diploma of Education program at the Queensland University of Technology in Australia were required to participate in a panel interview where they were graded by a panel made up of three faculty staff and one undergraduate student enrolled in the 4-year Bachelor of Education program. Students and panel members completed a questionnaire on their experience after the interview. Results indicated that both students and staff valued the experience and felt it was authentic. Results are discussed in terms of how the assessment interview and portfolio presentation supports graduating students in their preparation for employment interviews, and how this authentic assessment task has benefits for both students and teaching staff.     

Students’ learning outcomes and peer rating accuracy in compulsory and voluntary online peer assessment

Peer assessment can be conducted online with rapid development of online learning technology. The current study was conducted empirically to investigate peer rating accuracy and student learning outcomes in online peer assessments, comparing compulsory and voluntary peer assessment. Section 1 (N?=?93) was assigned to the voluntary group and Section 2 (N?=?31) was assigned to the compulsory group. The results showed the voluntary group scored significantly higher than the compulsory group in the final task of the course, while there was no significant difference on the final task score increase. Students who participated in the voluntary group provided more accurate scores (i.e. peer rater accuracy) than those who participated in the compulsory group. The peer score leniency/severity rating, comparing peer assigned scores with the teacher assigned scores, were generally consistent with the peer rater accuracy results. The current study offers insights for researchers who are interested in studying the effect of online peer assessment activities. The results are also of interest for instructors who may want to conduct peer assessments in online courses and are choosing between compulsory and voluntary formats.     

Does assessment for learning make a difference? The development of a questionnaire to explore the student response

Assessment for learning is a widely used term and the concept forms the basis of many teaching innovations in higher education. However, the definitions and scope of assessment for learning vary considerably. We describe a conceptualisation of assessment for learning that encompasses current thinking in a holistic way and which has been trialled and extensively refined in practice. A student questionnaire is presented which has enabled us to explore the student experience. Results indicate that the overall student experience is more positive in modules where assessment for learning approaches are used and students are more likely to take a deep approach to learning. It also demonstrates that the student experience is centred on staff support and module design, feedback, active engagement and peer learning. The full questionnaire is made available and its wider use in evaluation, enhancement and research is encouraged.     

Staff student partnership in assessment: enhancing assessment literacy through democratic practices

In recent years, research and practice focused on staff and students working in partnership to co-design learning and teaching in higher education has increased. However, within staff–student partnerships a focus on assessment is relatively uncommon, with fewer examples evident in the literature. In this paper, we take the stance that all assessment can be oriented for learning, and that students’ learning is enhanced by improving their level of assessment literacy. A small study in a Scottish university was undertaken that involved a range of different adaptations to assessment and feedback, in which students were invited to become partners in assessment. We argue that a partnership approach, designed to democratise the assessment process, not only offered students greater agency in their own and their peers’ learning, but also helped students to enhance their assessment literacy. Although staff and students reported experiencing a sense of risk, there was immense compensation through increased motivation, and a sense of being part of an engaged learning community. Implications for partnership in assessment are discussed and explored further. We assert that adopting staff–student partnership in assessment and more democratic classroom practices can have a wide range of positive benefits.     

Accuracy of students' self‐assessment and their beliefs about its utility

The purpose of the two studies presented here was to evaluate the accuracy of students' self‐assessment ability, to examine whether this ability improves over time and to investigate whether self‐assessment is more accurate if students believe that it contributes to improving learning. To that end, the accuracy of the self‐assessments of 3588 first‐year students enrolled in a post‐secondary institution was studied throughout a semester during which each student made approximately 80 self‐assessments about his or her own learning process. These self‐assessments were then compared with multiple judgements by peers and tutors. The overall correlations between the scores of self‐, peer and tutor assessments suggest weak to moderate accuracy of student self‐assessment ability. The findings also reveal an ability effect; students judged as more academically competent were able to self‐assess with higher accuracy than their less competent peers. Comparing the accuracy of student self‐assessment averaged over four consecutive periods indicates that the accuracy does not improve over time. In a second study, a questionnaire aimed at eliciting student's beliefs about the effects of self‐assessment on their learning was administered to 936 first‐year students. Based on their responses, sub‐groups of students were identified: those who either believed in the usefulness of self‐assessment or did not. Results suggest that there is no significant association between student beliefs about the utility of self‐assessment and the accuracy of their self‐assessments.