Academics' resistance to summative peer review of teaching: questionable rewards and the importance of student evaluations

This study draws from 30 semi-structured interviews with tenure-track faculty members in a research-intensive university to examine their lack of engagement in the summative peer review of teaching. Findings indicate that most academics in the study do not think peer review outcomes contribute meaningfully to decisions about career advancement and believe that, in comparison, student evaluation of teaching scores matter more. The findings suggest that faculty member resistance to summative peer reviews will persist unless academics are confident that the results will be seriously considered in decisions about tenure and promotion. This article also contends that senior administrators should provide further clarity about the purpose and use of peer review outcomes in high-stakes career decisions.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

The assessment of learning skills in nursing student selection: A scoping review

Abstract

Higher education student selection has significant societal, institutional and individual impacts. Thousands of applicants apply only for nursing, one of the major higher education disciplines. As the nursing profession is characterised by cognitive requirements, higher education institutions assess the learning skills of nursing applicants. However, there has been no comprehensive analysis of learning skills assessment for nursing student selection. The purpose of this scoping review was to describe the assessment of learning skills in undergraduate nursing student selection. Five databases were systematically searched, and 24 studies published between 2006 and 2016 were included. Learning skills were most commonly assessed using standardised tests in the areas of language and communication, reasoning, mathematics and natural sciences. Overall scores of onsite selection methods were found to best predict future academic performance. The results indicate that higher education institutions may benefit from comprehensive assessment of learning skills in their selection processes. This assessment should focus on a wider range of cognitive aptitudes, including reasoning skills. This review focussed on nursing education, but the results may benefit other higher education disciplines due to the generic nature of learning skills and similar cognitive requirements of higher education studies. The results support the development of more comprehensive and valid methods for assessing learning skills.     

Assessing postgraduate student perceptions and measures of learning in a peer review feedback process

Peer review feedback, developed to assist students with increasing the quality of group reports and developing peer review skills, was added to a master’s level Climate Change Policy and Planning unit. A pre- and post-survey was conducted to determine whether students found the process a valuable learning opportunity: 87% of students responding to the post-review survey indicated they saw benefit in both developing and receiving peer review feedback. Peer review design minimising the potential for plagiarism resulted in low levels of student concern surrounding plagiarism. Marks allocated by students were valid (consistent with the lecturer) and were generally reliable; however, distribution of marks had a higher variance where group report quality was low. Training on completing the peer review template could result in more reliable feedback. Consistent with the literature, the feedback developed during the peer review process focused on style and presentation preferentially to content. Seven out of eight group reports improved between report submissions, with all groups responding to peer review feedback. Peer review feedback and changes made to group reports indicated students had limited capacity to judge the quality of referencing and critical analysis.     

A new scholar's perspective on open peer review

This article argues that the pedagogical and scholarly benefits of open peer review far outweigh those of traditional double-blind peer review, but require a shift in our perspective of the function and value of peer review – from a gate-keeping process, toward a supportive, constructive process of collaboration between peers and mentors.     

Peer review in the classroom: student perceptions,peer feedback quality and the role of assessment

Abstract

Peer review in the classroom can enhance numerous employability skills such as critical appraisal, writing skills, reflection practices and collaborative experiences. This study takes place over two years and discusses the implementation of a repeating blind peer review cycle across a single semester for final year chemistry students enrolled on a compulsory employability module. The feedback cycle promotes personal reflection through the use of mini-reflective questionnaires. The process was assessed by academic tutors at the resubmission stage and/or the peer feedback stage where the quality of peer feedback was directly assessed. The research investigates the quality of peer feedback, the importance of assessment and student perceptions of what is most useful. Methods include directed content analysis of feedback produced, student opinions and a focus group. Students were capable of offering useable feedback across a range of assessment criteria but tend to focus on the important criteria best aligned to the particular assignment. A range of motivational factors and tactics were noticed but students tended to find reviewing the most beneficial.     

Teacher immediacy: reflections on a peer review of teaching scheme

Using a qualitative approach drawing on the experiences of four HE lecturers, this study provides an exploration of and insights into a peer review of teaching (PRT) scheme, which focused on teacher immediacy and communication skills. Within the United Kingdom, limited research has been undertaken in relation to teacher immediacy even though international research, particularly in the United States, suggests it enhances lecturer–student interaction.

The study identified three key factors that can enhance teaching through the development of immediacy skills. These were: voice and verbal qualities; body language; and location in the teaching space and environmental factors. This article suggests that, based on the evidence from the four lecturers, any PRT scheme that wishes to focus upon enhancing teacher effectiveness should consider addressing these themes in order to facilitate enhanced lecturer–student interaction.     

Four narratives about online international students: a critical literature review

In the current higher education context, offering online programmes is seen as an effective means to recruit more international students. However, supporting online international students studying at a distance is not a simple task for both universities and tutors. The problem mainly stems from a lack of theoretical understanding of online international students and their learning experiences. The present article, therefore, aims to address the gap, by systematically, yet critically reviewing relevant academic narratives about online international students. Our review reveals four types of narratives presented in the literature, describing online international students in particular ways: unspecified others with a rapid increase in their numbers, specific others with deficits, specific others as pedagogical resources and active participants in international learning communities. We discuss the merits and the drawbacks of each type of narratives for online educators seeking pedagogical suggestions about supporting online international students.     

Rethinking feedback practices in higher education: a peer review perspective

Peer review is a reciprocal process whereby students produce feedback reviews on the work of peers and receive feedback reviews from peers on their own work. Prior research has primarily examined the learning benefits that result from the receipt of feedback reviews, with few studies specifically exploring the merits of producing feedback reviews or the learning mechanisms that this activates. Using accounts of their experiences of peer review, this study illuminates students’ perceptions of the different learning benefits resulting from feedback receipt and feedback production, and, importantly, it provides insight into the cognitive processes that are activated when students construct feedback reviews. The findings show that producing feedback reviews engages students in multiple acts of evaluative judgement, both about the work of peers, and, through a reflective process, about their own work; that it involves them in both invoking and applying criteria to explain those judgements; and that it shifts control of feedback processes into students’ hands, a shift that can reduce their need for external feedback. The theoretical and practical implications of these findings are discussed. It is argued that the capacity to produce quality feedback is a fundamental graduate skill, and, as such, it should receive much greater attention in higher education curricula.     

A review of the characteristics of intermediate assessment and their relationship with student grades

In-course assessment, such as midterms, quizzes or presentations, is often an integral part of higher education courses. These so-called intermediate assessments influence students’ final grades. The current review investigates which characteristics of intermediate assessment relate to these grades. In total, 88 articles were reviewed that examined the relationship between intermediate assessment and student grades. Four main characteristics were identified: the use of feedback, whether the assessment is mandatory, who is the assessor, and the reward students get for participating. Results indicate that corrective feedback leads to the most positive results, but elaborate feedback may benefit lower achieving groups. No difference in results was found for mandatory versus voluntary intermediate assessments. Peer assessment seemed to be beneficial, and rewarding students with course credit improves grades more than other rewards. Three scenarios are presented on how teachers can combine the different characteristics to optimise their intermediate assessment.     

A systematic literature review of student engagement in software visualization: a theoretical perspective

ABSTRACT

Background and Context: In spite of the decades spent developing software visualization (SV), doubts still remain regarding their effectiveness. Furthermore, student engagement plays an important role in improving SV effectiveness as it is correlated with many positive academic outcomes. It has been shown that the existing SV has failed to engage students effectively.

Objective: Therefore, there is a need to understand the theories behind SV design from the engagement perspective to produce a roadmap for future tool construction. The aim of this study was to identify the theories have been used in literature to explain or construct student engage- ment with SV in computer science courses for novices.

Method: We performed a systematic literature review that identified a total of 58 articles published between 2011 and 2017, which were then selected for the study. However, among them, only 18 articles had discussed their theoretical framework.

Findings: The results of this study show a richness in the theoretical framework obtained from different disciplines, however, constructivism is still dominant in the computing education research (CER) domain. It is evidently clear from the findings that the theories generated from the CER domain are needed to enhance the effectiveness of SV.

Implications: As a result of this review, we suggest several design principles and engagement attributes to be considered while creating an engaging SV.     

Outcomes of synergetic peer assessment: first-year experience

Active participation in learning activities and reviewing assessment activity can facilitate learners engaged in these processes. This case study reports student experiences of the process of peer assessment with teacher guidance in a group project for a first-year nursing course with 153 students. Twenty groups of students were assigned roles in exploring one of the two scenarios: one involved analysing diabetes mellitus and the other considered colorectal carcinoma, in each case covering provision of acute nursing care in hospitals and preparing health promotion strategies in a community. Each group also provided comments to the group that worked on the other scenario, based on prearranged assessment criteria. After receiving comments from their peer group, each group was then allowed to revise their drafts prior to final submission. The teacher’s input would be offered, if the peer feedback was inaccurate or insufficient. A survey was conducted to explore the learning experiences, and three semi-structured focus group interviews were conducted to triangulate findings with the survey. The first-year students expressed the need for both peer feedback and the lecturer’s direct guidance.     

国外TESOL写作同伴互评研究综述

回顾并评述了国外TESOL写作中同伴互评相关论文及其研究成果,重点关注了同伴互评的效果、教师反馈和自我评测的比较以及对同伴互评的培训和具体实施3个方面,指出了国内未来研究的方向。     

Predicting academics' willingness to participate in peer review of teaching: a quantitative investigation

Peer review of teaching is a collegial process designed to help academics reflect on and improve their teaching practice. Considerable research supports the value of peer review of teaching. However, uptake of voluntary programs is typically low. Few studies have examined the predictors of engagement in voluntary peer review. This study surveyed 221 (106 female, 115 male) academics in the first year of implementation of a peer review of a teaching program at a large dual-sector university. Results supported the impact of four key variables on willingness to engage in peer review of teaching: the perceived benefits of the program, an individual's perceived need for support around teaching, perceived drawbacks of the program and career-related benefits. Level of teaching experience was related to willingness to engage: staff with moderate levels of teaching experience were most willing to participate in the program. Limitations of the study are noted and recommendations for peer-review programs are provided.     

An inventory of peer assessment diversity

Since Topping published his literature review on peer assessment in 1998 Topping, K. 1998. Peer assessment between students in colleges and universities. Review of Educational Research, 68: 249–76. [Crossref], [Web of Science ®] [Google Scholar], the number of studies on this subject has doubled, if not tripled. However, along with this expansion, the diversity of peer assessment applications increased equally fast. Based on recent literature, this contribution focuses specifically on the diversity that has come to exist in order to update Topping's typology, adding eight variables, extending eight others with extra sub‐dimensions, and including the implementation factors. Five original variables were absorbed into larger entities, leaving 20 variables which were finally grouped into five clusters. For teachers or advisors, this inventory may be of interest as a source of inspiration or as a checklist of important decisions to make when working with peer evaluation. For researchers, it may be a guide towards covering the full scope of particularities when introducing their peer assessment design. Moreover, the framework developed in this paper might relieve the possible confusion originating from the use of a single term to cover a multitude of sometimes incompatible practices.     

Teaching the writing of psychological reports through formative assessment: peer and teacher review

This paper describes the teaching of psychological report writing at Karolinska Institutet and the efforts to improve it through the introduction of formative assessment applied as peer and teacher review. 48 fourth-year psychology students were required to write a psychological report. Half of the students also participated in a non-mandatory peer review of a first draft. All students handed in a preliminary version of the report, had it reviewed by the teachers and made alterations before handing in a final version. The results showed that both peer review and teacher review were valuable methods in the teaching of psychological report writing.     

网络化审稿系统的概要设计

随着计算机和通讯技术的飞速发展,人类社会正在步入网络时代。高校学报作为学术前沿的窗口,必然要与时俱进,必然呈现高效、便捷、实时、共享、合作等新特性。传统的审稿模式必然要与网络化审稿模式相结合,这样才能优势互补。各家学报都具有共性和特性,所以审稿系统需考虑绝大多数学报的具体情况来建立,充分考虑各方需求。本文就作者、主编、编辑、审稿专家、管理员的各自需求对网络化审稿系统作概要的设计。     

提高科技期刊专家审稿质量的途径

为了使专家审稿的过程更加快速有效,从审稿专家的选取、审稿方式和方法的选择、审稿过程的控制等方面对提高科技期刊专家审稿质量的途径进行了分析,只有在正确的审稿专家选择原则的指导下,采用正确的审稿方式和方法,并对审稿过程进行合理的控制,才能使审稿专家对刊物有更加深入的了解,从而更加准确地把握稿件的学术质量。     

同伴互评机制中学生书面反馈的语类结构分析

文章运用实证的方法对大学英语写作教学中同伴互评的书面反馈内容进行了为期十八周的实证研究。研究结果表明学生的书面反馈可以分为三步即:内容评价、文章修改意见、以及鼓励期望等内容。学生反馈中存在的问题主要表现在误评、漏评等方面。     

The alignment of written peer feedback with draft problems and its impact on revision in peer assessment

Despite the wide use of peer assessment, questions about the helpfulness of peer feedback are frequently raised. In particular, it is unknown whether, how and to what extent peer feedback can help solve problems in initial texts in complex writing tasks. We investigated this research gap by focusing on the case of writing literature reviews in an academic writing course. The dataset includes two drafts from 21 students, sampled to represent a wide range of document qualities, and 84 anonymous peer reviews, involving 1,289 idea units. Our study revealed that: (1) at both substance and high prose levels, drafts of all quality levels demonstrated more common problems on advanced writing issues (e.g. counter-argument); (2) peer feedback was driven by difficulty of the problem rather than overall draft quality, peer comments were not well aligned with the relative frequency of problems, more comments were given to less difficult problems; (3) peer feedback had a moderate impact on revision, and importantly, receiving multiple comments on the same issue led to more repairs and improvement of draft quality, but consistent with the comments received, authors tended to fix basic problems more often. Implications for practice and research are drawn from these findings.