Students’ learning outcomes and peer rating accuracy in compulsory and voluntary online peer assessment

Peer assessment can be conducted online with rapid development of online learning technology. The current study was conducted empirically to investigate peer rating accuracy and student learning outcomes in online peer assessments, comparing compulsory and voluntary peer assessment. Section 1 (N?=?93) was assigned to the voluntary group and Section 2 (N?=?31) was assigned to the compulsory group. The results showed the voluntary group scored significantly higher than the compulsory group in the final task of the course, while there was no significant difference on the final task score increase. Students who participated in the voluntary group provided more accurate scores (i.e. peer rater accuracy) than those who participated in the compulsory group. The peer score leniency/severity rating, comparing peer assigned scores with the teacher assigned scores, were generally consistent with the peer rater accuracy results. The current study offers insights for researchers who are interested in studying the effect of online peer assessment activities. The results are also of interest for instructors who may want to conduct peer assessments in online courses and are choosing between compulsory and voluntary formats.     

Students’ behaviour in peer assessment: a multi-criteria clustering approach

Peer assessment (PA), as formative procedure, enhances learning by providing students with the opportunity to peer assess each other's work. However, since students exhibit different value systems (abilities, experiences, attitudes, cognitive styles, etc.) we propose a diagnostic procedure, which can be applied at the beginning of a course, in order to infer the most prevailing attitude among students. For this purpose, the proposed methodological framework, based on a multi-criteria clustering approach, identifies different assessment behaviours, in order to adopt the most common student assessment policy. To demonstrate the proposed method, an example is presented where two different PA data-sets are examined. The results clearly indicate that students exhibit different PA policies, nevertheless, a densely populated group is formed, the value system of which can be adopted thereafter.     

Is peer review an appropriate form of assessment in a MOOC? Student participation and performance in formative peer review

Many aspects of higher education must be reconceptualised for massive open online courses (MOOCs). Formative and summative assessment of qualitative work in particular requires novel approaches to cope with the numbers involved. Peer review has been proposed as one solution, and has been widely adopted by major MOOC providers, but there is currently little evidence about whether it is appropriate or under what conditions. Here, we examine student participation, performance and opinions of a peer review task in a biomedical science MOOC. We evaluate data from approximately 200 student topic summaries and 300 qualitative peer reviews of those summaries, and compare these to student demographic data (gender, age, employment status, education, national language) and to performance in multiple choice tests. We show that higher performance in the written topic summary correlated with both higher participation in the peer review task and with writing higher quality peer reviews. Qualitative analysis of student comments revealed that student opinion on the usefulness of the peer review task was mixed: some strongly believed it benefitted their learning, while others did not find it useful or did not participate. We suggest instructional design strategies to improve student participation and increase learning gain from peer review in the MOOC context.     

The impact of an elaborated assessee’s role in peer assessment

The purpose of this study was to investigate the effects of an elaborated assessee’s role on metacognitive awareness, performance and attitude in peer assessment. Two intact groups (a total of 82 students) were randomly assigned to a treatment condition (having back‐feedback activity) or a control condition (not having back‐feedback activity). The results indicated that, regarding metacognitive awareness, when participants played the elaborated assessee role, they tended to report higher metacognitive awareness in their learning process. Regarding performance, when participants played the elaborated assessee role, they tended to receive better scores in making a concept map. Regarding attitude, when participants played the elaborated assessee role, they reported greater motivation towards the peer assessment. The findings of this study suggest instructional implications for those who want to employ peer assessment as a learning method by showing the effectiveness of a well‐developed role design, specifically one that includes the back‐feedback activity.     

Does evaluating peer assessment accuracy and taking it into account in calculating assessor’s final score enhance online peer assessment quality?

Education and Information Technologies - Peer assessment has an important role in teaching and learning nowadays. However, the existing techniques tend to be limited due to a lack of a suitable...     

Student teachers’ engagement in Facebook-assisted peer assessment in an initial teacher education context: speaking 2.0

Although peer assessment (PA) has been explored and implemented in higher education, little is known about how it is carried out through such a virtual platform as Facebook in an initial teacher education (ITE) context. To fill this practical void, this article showcases the brief portrait of Facebook-assisted PA in an English as a foreign language (EFL) classroom. This in-practice article shows how Facebook as an online platform serves as virtual engagement in PA among English student teachers in the EFL speaking classroom. This virtual engagement not only created an interactive virtual speaking classroom, but also it encouraged students to become critical reflective language learners.     

University‐school collaboration and needs assessment in the training of student leadership and peer support in Hong Kong

One hundred seventy secondary schools in Hong Kong responded to a questionnaire designed to assess their needs for leadership training of students in different domains of school activities and their endorsement of university‐school collaboration in these activities. The questionnaire also assessed the extent and degree of support and involvement that schools intended to provide in training programs and in organized school activities for students to assume leadership roles in schools. Findings indicated that schools were in need of student leaders in different student activities, especially in those less related to academic concerns. It was stressed that in providing opportunities for student leaders to continue practicing their skills in school activities, other students might eventually benefit through participation in school activities and peer support programs organized by student leaders.     

Academics’ perceptions of the benefits and challenges of self and peer assessment in higher education

Despite compelling evidence of its potential effectiveness, uptake of self and peer assessment in higher education has been slower than expected. As with other assessment practices, self and peer assessment is ultimately enabled, or inhibited, by the actions of individual academics. This paper explores what academics see as the benefits and challenges of implementing self and peer assessment, through the analysis of interviews with 13 Australian academics. Thematic analysis of our qualitative data identified seven themes of benefits and five challenges. Our academics showed strong belief in the power of self and peer assessment as formative assessment, contrary to past literature which has focussed on the accuracy of students’ marking. This paper therefore brings insights as to not only what academics value about self and peer assessment but also identifies potential inhibitors in practice. Recommendations are made about improving the design and implementation of self and peer assessment in higher education.     

Using peer assessment of fieldwork to enhance students’ practical training

Fieldwork training is a key component of several practical disciplines. In this study, students’ peer assessment of fieldwork is explored as a method to improve their practical training. Peer assessment theories are first discussed. A framework for peer assessment of fieldwork is proposed, and the steps taken for preparation of students for this task are discussed. A developed marking, feedback and moderation tool of assessment are presented. Application of peer assessment in the field was investigated over a period of two years in one undergraduate unit in the geospatial discipline as an example. Reliability of peer assessment was estimated by measuring the difference between assessments carried out by groups of peer assessors, and its validity was measured by comparing students’ marks with those given by tutors. Results show that students have gained from the peer assessment process, mainly as a formative form of assessment, by better understanding and endeavouring to achieve the objectives of field tasks. Tutors use differences among assessments made by groups of students compared to tutors’ assessments to identify field components that need better explanation of their content and assessment criteria.     

The architecture of peer assessment: do academically successful students make good teammates in design assignments?

This paper considers the relationship between architecture and construction management students’ overall academic abilities (as measured by Weighted Average Marks [WAMs]), their peer ratings for contributions to team design assignments (as measured by an online Self-and-Peer-Assessment [SAPA] tool), and their specific abilities as building designers (as measured by grades in individual design assignments). The research was conducted to determine whether a student’s prior academic achievements might indicate how well they will work in teams. The research demonstrates a statistically significant relationship between WAMs and SAPA ratings indicating that academically successful students more often than not make good teammates. However, the study also highlights that when peers are assessing contributions to teamwork they are assessing skills and qualities in their teammates other than overall academic ability or the ability to design well. Whilst this study is largely located within the field of design, the findings are relevant to any group work where teachers aim to design assessment that unravels group and individual contribution.     

Exploring students’ emotional responses and participation in an online peer assessment activity: a case study

The STACI^N Project is a development, implementation, and research effort that examines the impact of systems thinking, simulation, and modeling in precollege curricula on student learning, classroom processes, and organizational change. Nearly 100 teachers have been trained to use systems thinking in their secondary school curricula. This article examines how systems thinking affects teachers and classroom processes. A two‐dimensional matrix is presented to characterize the modes of adaptation employed by the teachers as they use modeling and simulation in their courses. The dimensions include teacher proficiency with technology and the specific systems applications observed in classes. A discussion of issues related to the use of systems in precollege instruction also is included.     

Factors associated with students’ attitude change in online peer assessment – a mixed methods study in a graduate-level course

Abstract

Positive student attitude towards peer assessment can be demonstrated through high-quality peer assessment activities. Research findings on students’ attitudes before and after such activities were mixed. This study was conducted using a mixed-methods approach to investigate the underlying factors influencing students’ attitude change. Participants were students enrolled in an online graduate-level assessment course in the college of education in a university in the southeast United States in the fall of 2017 (N?=?31). Surveys and interviews were used to collect quantitative and qualitative data separately, both of which indicated that perceived accurate and specific feedback tended to help students shift to more positive attitudes towards online peer assessment. In addition, final task score improvement after the activity was correlated with positive attitude change. Qualitative results also identified other factors, such as communication with the peers’ work and logistic concerns. The current study offers insights for researchers and instructors to promote positive attitude changes in online peer assessment activities.     

The role of self-, peer and teacher assessment in promoting Iranian EFL learners’ writing performance

In the last decade, with the increased attention to learner-centred curricula, the topic of self-assessment and peer assessment has become of particular interest in testing and evaluation. The present study explores the role of self-assessment, and peer assessment in promoting writing performance of language learners. To do this, 157 intermediate TEFL (Teaching English as Foreign Language) students were assigned to five different treatments in five groups: four experimental groups and one control group. The first experimental group did journal writing as a self-assessment technique, the second group self-assessed their own writings, the third group employed peer assessment, and the fourth group had both self- and peer assessment. Moreover, there was teacher assessment in all experimental groups, except the fourth group, i.e., the self- and peer assessment group. In the control group, there was only teacher assessment. Also, at the beginning and end of the semester, all participants took a writing test. The design of the study was quasi-experimental, non-randomised control group, pre-test–post-test design. The results revealed that in the second and third groups, in which the students employed self-assessment and peer assessment, together with teacher assessment, we observed the maximum improvement in writing.     

Teachers’ reasons for using peer assessment: positive experience predicts use

Peer assessment (PA) is one of the central principles of formative assessment and assessment for learning (AfL) fields. There is ample empirical evidence as to the benefits for students’ learning when AfL principles are implemented. However, teachers play a critical role in mediating the implementation of intended policies. Hence, their experiences, beliefs, and attitudes towards PA are important factors in determining whether the policy is actually carried out. A survey of over 1500 primary, secondary, and higher education teachers in Spain elicited their beliefs and values around PA as well as other aspects of formative assessment; only 751 teachers provided complete responses to all PA items. Teachers reported occasional use of PA in their classrooms but with positive experience of it. The vast majority did not use anonymous forms of PA and half of the teachers considered the students were accurate when assessing peers. Confirmatory factor analysis and structural equation modeling were used to examine relationships of attitudes and beliefs to self-reported frequency of using of PA. The self-reported frequency of using PA was strongly predicted by teacher experience of PA which included positive reasons for using PA, rather than negative obstacles for avoiding, prior use, and beliefs that students should participate in assessment, and willingness to include PA in grading.     

Predicting pre-service teachers’ intention of implementing peer assessment for low-achieving students

Despite the benefits of peer assessment, many teachers are not willing to implement it, particularly for low-achieving students. This study used the theory of planned behaviour to predict pre-service teachers’ intention to use peer assessment for low-achieving students. A total of 229 pre-service teachers in Singapore participated in the survey which consists of eight factors about peer assessment in writing: belief of knowledge source, belief of ability, belief of constructivist assessment, attitude, subjective norm, perceived behaviour control, behavioural intention, and intention for low-achieving students. Structural equation modelling was used to test hypothesized relationships of the eight factors. This study found that pre-service teachers’ beliefs about knowledge source and constructivist assessment significantly influenced their attitude towards peer assessment of writing. In addition, the attitude and perceived behaviour control were significant factors in shaping the intention of peer assessment. Lastly, pre-service teachers’ intention of implementing peer assessment for low-achieving students was influenced by both the intention of peer assessment and the belief of ability.     

Global learning partnerships in the Café: peer feedback as a formative assessment tool for animation students

This paper reports on a global learning partnership using the Café: the collaborative application for education as an e-learning environment within the Facebook framework, for first-year animation students at the University of South Australia (USA) in Australia and Nanyang Technological University in Singapore. “The Café” has been designed based on five principles of user interface design – visibility, usability, relevance, accessibility, and interactivity – and aims to provide institutions with an established, structured, and dedicated e-learning environment that meets the needs of modern-day tertiary students and teaching staff. From July to November in 2013, 71 students participated within the e-learning environment. Students submitted work-in-progress imagery related to major assignments, and provided feedback and critiques to their local and global peers. A post-semester survey provided students with the opportunity to critically reflect on the learning experience. The results are discussed in light of the use of peer feedback as a formative assessment tool.     

Learners’ engagement online in peer help

ABSTRACT

Much is known about help seeking in face-to-face classrooms, but there is a limited understanding about how learners seek and provide help online. This study implemented a peer-help forum in an online technology course, and investigated students’ engagement in the forum, their perceptions, and the relationship between peer help and course performance. The findings suggested that students actively engaged in the peer help beyond the course requirements. Emerging themes are presented regarding the students’ online help-seeking and help-giving behaviors and their perceptions. In addition, help seeking was not significantly related to performance, but a significant interaction effect suggested that help seeking benefited the performance of a subgroup of the students – those who provided less help.     

Student perspectives on formative peer assessment: an attempt to deepen learning?

Peer assessment is a common form of shared learning in which students provide feedback on each others work. Peer assessment takes many forms; and involves students and tutors taking various roles at different stages of the process. This study explores the views and opinions of undergraduate students in relation to their perceptions and experiences of formative peer assessment introduced as a learning development opportunity for the first time. The study found that on the whole formative peer assessment was a positive experience in enhancing students learning and development. However, consideration needs to be taken to address individual learning styles, as a limited number of students found the process to be less useful. Consequently, when tutors are constructing peer assessment strategies they should be cognisant at the planning stage of the variety of learning styles that are evident in order to maximise the development opportunities this can bring to students.     

External peer review of assessment: an effective approach to verifying standards?

There is growing international concern to regulate and assure standards in higher education. External peer review of assessment, often called external examining, is a well-established approach to assuring standards. Australian higher education is one of several systems without a history of external examining for undergraduate programmes that is currently considering the approach. What can entrants to external examining at that level learn from the UK higher education system's long history of external examining? To that end, this paper reports on a mixed methods research project designed to investigate current practices in how academic standards are conceived, constructed and applied by external examiners and debates the implications of the findings for the development of external examining in other countries. The findings suggest that the potential of experienced peers in a subject discipline to provide the assurance of standards is limited. It concludes by presenting various possible enhancements that might be considered.