Portfolio appraisal: In search of criteria

Two inherently contradictory forces are pushing for reform in portfolio assessment. On the one hand there is a felt need for creating more rigid standards that operate to promote uniformity of ratings in appraisal practice to certify achievement. However, on the other hand, critical questions are being raised about separating acclaimed portfolio goals aimed both at appraising achievement while also improving quality of student learning and development. The position of portfolio assessment, which is widely used nowadays in teaching and teacher education, comes into debate. In our study, we look for actual practices in portfolio appraisal in search of criteria used for rating the quality of portfolio materials. It is our interest to find out how appraisal criteria are selected and used to evaluate achievement or to improve the quality of development and learning. In the context of teacher education, we have examined both the espoused criteria of both assessors and collectors of portfolios as well as the actual appraisal practices by looking at the judgmental orientations and supervision styles used by portfolio assessors. In addition, we offered an authentic portfolio document to be rated by different assessors to gauge and compare their quality of rating and criteria use. The actual processes we detected point to a most common practice of employing judgmental, usually normative evaluations based on assessor dependent, more or less pre-decided criteria which permit a “checkbox” approach to appraisal.     

Explicitness of criteria in peer assessment processes for first-year engineering students

This article will outline the motives for the implementation of peer assessment as put into action at the first year of the Civil Engineering course of the University of Minho. The implementation of new assessment methods was a consequence of the successful implementation of peer assessment at other engineering courses. During the semester, three assessment moments took place, in which students assessed the work of their peers. Assessment criteria were defined in a negotiation process between students and teacher. The students subsequently graded the work of their colleagues and had to provide a clear justification for each given grade. The aim of this method is to involve students in the subject, enhance their motivation and deepen their learning. Advantages and disadvantages of peer assessment are discussed. The results that have been obtained so far draw attention to the nature of assessment criteria and the corresponding explicitness.     

Assessment matters: In search of the wizard of assessment

At first, the shift in measurement of educational success from bodies in seats to learning outcomes sent student affairs educators at California State University, Northridge, into a panic. Persistence, support, and a clear framework for understanding learning led to the discovery that there was a little bit of assessment wizard in all of them.     

Formative peer assessment in primary school: the effects of involving pupils in setting assessment criteria on their appraisal and feedback style

This experimental study is aimed at examining the effect of involving pupils in primary education in setting assessment criteria and standards on their appraisal of a peer’s work and their peer feedback style. It is expected that our intervention will lead to (1) a criteria-referenced appraisal and (2) more final vocabulary, which is associated with a more authoritative style. In total, 95 sixth grade pupils from The Netherlands were randomly assigned to one of two conditions. In the experimental group, the pupils first received an exemplar, which was followed by a group discussion about appropriate assessment criteria and standards. In the control group, the pupils provided peer feedback without this discussion. Their peer appraisal was measured with a questionnaire and feedback style was determined by analysing the written feedback using a person-oriented approach (cluster analysis). Results showed that the chance that pupils in the experimental group had an authoritative style was three times higher than in the control group. Theoretical and practical implications of the study are discussed.     

The quality of peer assessment in a wiki-based online context: a qualitative study

Peer assessment (PA) provides opportunities for authentic assessment, autonomy and collaboration. Several authors advocate that students can benefit from PA and put forward the effects of PA on the students’ learning outcomes. Questions concerning the validity and reliability of PA and PA competences are also addressed by different researchers. This qualitative study is part of a wider project that seeks to develop and test evaluation and assessment strategies in online contexts. In a doctoral module, PA was used for summative and formative purposes. Formative PA aimed to give feedback about the ongoing group work, but also to increase online interaction between the different groups of students. The main module task was to write a literature review, about a selected topic, using a wiki. Criteria and indicators to assess the literature review were negotiated with the students. Different criteria were used to assess the quality of PA, such as, the use of the negotiated criteria, the adequacy of the chosen vocabulary or the provision of constructive feedback. The results show that overall the quality of PA can be improved. Groups did not provide sufficient criticism, questions and suggestions for improvement.     

Assessing assessment quality: Criteria for quality assurance in design of (peer) assessment for learning - A review of research studies

The interest in assessment for learning (AfL) has resulted in a search for new modes of assessment that are better aligned to students’ learning how to learn. However, with the introduction of new assessment tools, also questions arose with respect to the quality of its measurement. On the one hand, the appropriateness of traditional, psychometric criteria is being questioned in the context of assessment for learning. On the other hand, it has been argued that new criteria need to be formulated to acknowledge the unique features of assessment for learning. The issue dealt with in this review is what quality criteria are specifically relevant to AfL. Studies using (peer) assessment for learning were evaluated with regard to use of quality criteria under two perspectives: their recognition of educational measurement criteria, as well as their consideration of student involvement in the assessment of learning. The selected studies were analyzed to determine what specific criteria were utilized in (successive parts of) the assessment cycle (i.e., the process of construction, administration and follow up of an assessment). Our results indicate that quality criteria are dissimilarly connected to the distinct steps of an assessment cycle, which holds as well for the attention given to student involvement in assessment for learning.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

The role of anonymity in peer assessment

This quasi-experimental study aimed to examine the impact of anonymity and training (an alternative strategy when anonymity was unattainable) on students’ performance and perceptions in formative peer assessment. The training in this study focused on educating students to understand and appreciate formative peer assessment. A sample of 77 students participated in a peer assessment activity in three conditions: a group with participants’ identities revealed (Identity Group), a group with anonymity provided (Anonymity Group) and a group with identities revealed but training provided (Training Group). Data analysis indicated that both the Anonymity Group and Training Group outperformed the Identity Group on projects. In terms of perceptions, however, the Training Group appreciated the value of peer assessment more and experienced less pressure in the process than the other two groups.     

Moderation of peer assessment in group projects

It is shown here that a grade distribution scheme commonly used to moderate peer assessments where self assessment is excluded is based on a false premise and will give an erroneous ranking in the situation where the best performer in a student group ranks the second best performer much higher than the other group members. An alternative to normalisation is proposed based on the idea that the rank order of peer grades should be preserved and should as far as possible be consistent between assessors. It is shown that the method correctly recovers the rank order of students within the group for all cases examined, while still eliminating biases that can result from differences in marking standards in the group. It is suggested that the approach could also be used to check for bias when self assessment is included.     

Challenges in operationalizing cultural practices in classroom and peer communities

This paper explores challenges encountered in a study of culturally relevant pedagogy that examines numeracy practices of under-served youth in secondary mathematics classrooms and in their peer communities at school. Taking a sociocultural stance toward studying under-served youths’ cultural practices as valuable resources for mathematics learning poses several methodological challenges, including identifying distinct cultural groups; adopting a view of mathematics as a social practice; and examining youth cultural practices to identify what counts as a viable example of practice relevant to mathematics practices and knowledge. Two dimensions of cultural practices are the focus: (1) communication, interaction, and language-use patterns that youth develop as they engage in the cultural practices of their peer communities; and (2) mathematical funds of knowledge that youth have developed over time. The complexities of identifying mathematical practices in youth peer communities and leveraging them in schooled mathematics are discussed, as are implications for further research.     

An inventory of peer assessment diversity

Since Topping published his literature review on peer assessment in 1998 Topping, K. 1998. Peer assessment between students in colleges and universities. Review of Educational Research, 68: 249–76. [Crossref], [Web of Science ®] [Google Scholar], the number of studies on this subject has doubled, if not tripled. However, along with this expansion, the diversity of peer assessment applications increased equally fast. Based on recent literature, this contribution focuses specifically on the diversity that has come to exist in order to update Topping's typology, adding eight variables, extending eight others with extra sub‐dimensions, and including the implementation factors. Five original variables were absorbed into larger entities, leaving 20 variables which were finally grouped into five clusters. For teachers or advisors, this inventory may be of interest as a source of inspiration or as a checklist of important decisions to make when working with peer evaluation. For researchers, it may be a guide towards covering the full scope of particularities when introducing their peer assessment design. Moreover, the framework developed in this paper might relieve the possible confusion originating from the use of a single term to cover a multitude of sometimes incompatible practices.     

Peer review in the classroom: student perceptions,peer feedback quality and the role of assessment

Abstract

Peer review in the classroom can enhance numerous employability skills such as critical appraisal, writing skills, reflection practices and collaborative experiences. This study takes place over two years and discusses the implementation of a repeating blind peer review cycle across a single semester for final year chemistry students enrolled on a compulsory employability module. The feedback cycle promotes personal reflection through the use of mini-reflective questionnaires. The process was assessed by academic tutors at the resubmission stage and/or the peer feedback stage where the quality of peer feedback was directly assessed. The research investigates the quality of peer feedback, the importance of assessment and student perceptions of what is most useful. Methods include directed content analysis of feedback produced, student opinions and a focus group. Students were capable of offering useable feedback across a range of assessment criteria but tend to focus on the important criteria best aligned to the particular assignment. A range of motivational factors and tactics were noticed but students tended to find reviewing the most beneficial.     

In search of inclusion

This article provides a context within which other contributions to this issue might be read. It examines the position of special educational needs (SEN) within the evolving continuum of education in Northern Ireland, specifically within the context of educational inclusion. It describes recent changes in educational policy and legislation which are likely to impact on the inclusion of children with special educational needs, examines inclusion within the mainstream schools and the role of special schools.     

Rethinking feedback practices in higher education: a peer review perspective

Peer review is a reciprocal process whereby students produce feedback reviews on the work of peers and receive feedback reviews from peers on their own work. Prior research has primarily examined the learning benefits that result from the receipt of feedback reviews, with few studies specifically exploring the merits of producing feedback reviews or the learning mechanisms that this activates. Using accounts of their experiences of peer review, this study illuminates students’ perceptions of the different learning benefits resulting from feedback receipt and feedback production, and, importantly, it provides insight into the cognitive processes that are activated when students construct feedback reviews. The findings show that producing feedback reviews engages students in multiple acts of evaluative judgement, both about the work of peers, and, through a reflective process, about their own work; that it involves them in both invoking and applying criteria to explain those judgements; and that it shifts control of feedback processes into students’ hands, a shift that can reduce their need for external feedback. The theoretical and practical implications of these findings are discussed. It is argued that the capacity to produce quality feedback is a fundamental graduate skill, and, as such, it should receive much greater attention in higher education curricula.     

Assessment criteria: reflections on current practices

This article reviews the findings of a small‐scale investigation into the criteria used by a number of SACWG departments for assessing final‐year project modules in business and history and other written history assignments. The findings provide the basis for a broader discussion of the issues relating to the formulation and use of assessment criteria. Assessment entails academics making professional judgements about the standards and quality of students' work. However, for the educational value of the work entailed in developing assessment criteria to be fully realized, there needs to be a higher level of shared understanding than currently exists (among students, tutors and other stakeholders) of the language in which criteria are couched and the ways in which criteria are applied.     

Student attitudes that predict participation in peer assessment

Peer assessment has been widely applied to actively engage students in learning to write. However, sometimes students resist peer assessment. This study explores reviewers’ attitudes and other underlying factors that influence students’ participation in online peer assessment. Participants were 234 Chinese undergraduates from two different academic backgrounds: engineering majors (n = 168) and English majors (n = 66). Gender, academic background and prior experience with peer assessment were all related to participation levels. Moreover, factor analyses revealed three attitudinal factors: (1) positive attitude (a general endorsement of the benefits of peer assessment), (2) interpersonal negative (concerns about the negative effects on interpersonal relationships), and (3) procedural negative (doubts about the procedural rationality of peer assessment). Among the attitudinal factors, procedural negative was negatively associated with participation, as expected. Interestingly, interpersonal negative was associated with greater participation, and positive attitude was associated with lower participation, in part because students worked hard on each review rather than doing many reviews superficially. Implications for instruction are discussed.     

浅谈大学生同伴互评作文

同伴互评英语作文是一种积极有效的评阅英语作文的辅助方法。本文就大学生英语写作现状,从理论上论述了在大学生中同伴互评作文的理论依据;从大学生的实际及其问卷调查角度论证了在大学生中进行同伴互评作文的可行性;阐述了同伴互评作文的优势及意义。