Factors associated with students’ attitude change in online peer assessment – a mixed methods study in a graduate-level course

Abstract

Positive student attitude towards peer assessment can be demonstrated through high-quality peer assessment activities. Research findings on students’ attitudes before and after such activities were mixed. This study was conducted using a mixed-methods approach to investigate the underlying factors influencing students’ attitude change. Participants were students enrolled in an online graduate-level assessment course in the college of education in a university in the southeast United States in the fall of 2017 (N?=?31). Surveys and interviews were used to collect quantitative and qualitative data separately, both of which indicated that perceived accurate and specific feedback tended to help students shift to more positive attitudes towards online peer assessment. In addition, final task score improvement after the activity was correlated with positive attitude change. Qualitative results also identified other factors, such as communication with the peers’ work and logistic concerns. The current study offers insights for researchers and instructors to promote positive attitude changes in online peer assessment activities.     

Assessment training effects on student assessment skills and task performance in a technology-facilitated peer assessment

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the training, students reviewed learning concepts, discussed marking criteria, graded example projects and compared their evaluations with the instructor’s evaluation. Data were collected in the form of initial and final versions of students’ projects, students’ scoring of example projects before and after the assessment training, and written feedback that students provided on peer projects. Results of data analysis indicate that the assessment training led to a significant decrease in the discrepancy between student ratings and instructor rating of example projects. In addition, the degree of student vs. instructor discrepancy was highly predictive of the quality of feedback that students provided to their peers and the effectiveness of revisions that they made to their own projects upon receiving peer feedback. Smaller discrepancies in ratings were associated with provision of higher quality peer feedback during peer assessment, as well as better revision of initial projects after peer assessment.     

Exploring the Structural Relationships between High School Students’ Scientific Epistemological Views and their Utilization of Information Commitments toward Online Science Information

The main purpose of this study was to examine the structural relationships between scientific epistemological views (SEVs) and information commitments (ICs) of high school students in Taiwan. Data were collected from 486 Taiwanese high school students via two self‐reporting instruments: one was the SEV questionnaire, including five scales for representing students’ views toward scientific knowledge; and the other was the ICs survey, involving six scales for exploring their evaluative standards and searching strategies of online science information. Structural equation modelling analysis was used to examine the relationships between the aspects of SEVs and ICs. The results of the measurement model confirmed that both the SEVs and ICs instruments had highly satisfactory validity and reliability. The structural equation modelling analysis further indicated that students’ SEVs guided their evaluative standards and searching strategy when dealing with Web‐based science information. For example, students who viewed scientific knowledge as more changeable and tentative significantly tended to adopt a more sophisticated evaluative standard, such as carefully inspecting the content of web sites for judging the usefulness. The findings in general suggested that students with more constructivist‐oriented SEVs might develop more advanced standards and searching strategy toward online scientific information to derive great benefit from Web‐based environments. Consequently, the role of SEVs should be highlighted as increasingly metacognitive engagement with online science information.     

Using online peer assessment in an Instructional Technology and Material Design course through social media

This study was designed to investigate the student teachers perceptions about and benefits and challenges of using Facebook as an online peer assessment tool for the student teachers’ works. The study group included 24 student teachers in science education department of a state university located in the southeast region of Turkey. A case study approach of the qualitative method was employed in the research. Semi-structured interviews were conducted to collect the data. The interviews were audio recorded, and records of all the interviews were transcribed into full text in Turkish. Collected data were analyzed using an emergent coding approach. Codes, then, were categorized to constitute themes and subthemes. The findings indicated that the student teachers were able to give objective feedback on their peers’ work and engaged more actively in class after participating in online peer assessment. Additionally, the students found it exciting and productive to use Facebook as a peer assessment tool in their learning.     

Peer assessment effects on the self-assessment process of first-year undergraduates

Peers carry potential in enhancing students’ self-assessment development, but few studies have explored how peer scaffolding is enacted in the process. This qualitative study explores peer assessment effects on the self-assessment process of 11 first-year undergraduates and the factors limiting peer influence. Drawing on the data from students’ journals, follow-up interviews, observations of in-class formative peer assessment activities and teacher interviews, we ascertained that peers could aid the self-assessment process by enriching student understanding of quality, refining subjective judgement and deepening self-reflection. Yet, peer influence could be reduced by distrust, tensions in feedback communication, competition and lack of readiness for peer learning. Implications for effective use of peers in supporting self-assessment development are discussed.     

Using a Dialogical Approach to Examine Peer Feedback During Chemistry Investigative Task Discussion

Peer feedback is an inherent feature of classroom collaborative learning. Students invariably turn to their peers for feedback when carrying out an investigative task, and this feedback is usually implicit, unstructured and may positively or negatively influence students’ learning when they work on a task. This study explored the characteristics of verbal peer feedback during a collaborative investigative chemistry task involving New Zealand Year 13 students. During the planning stage of the students’ investigation, the discussions of five pairs of students were recorded and then transcribed. Analysis of transcribed verbal data focused on interactions that involved peer feedback along two dimensions, interactive/non-interactive and dialogic/authoritative (Mortimer and Scott, 2003). The findings indicated that although students adopted a predominantly interactive/authoritative communicative approach, with peer feedback as confirmation or evaluation, they are also capable of a more interactive/dialogic exchange, characterised by elaborative peer feedback. We discuss how this dialogic perspective on peer feedback provides an alternative approach to the analysis and study of student–student interactions during science investigations. The findings should be interpreted in light of the limitations in terms of sample size, grouping and specificity of the coding scheme. Implications for teacher practice are discussed in relation to facilitating peer feedback discourse in the science classroom.     

Repeating Knowledge Application Practice to Improve Student Performance in a Large,Introductory Science Course

There is a tendency for lecture-based instruction in large introductory science courses to strongly focus on the delivery of discipline-specific technical terminology and fundamental concepts, sometimes to the detriment of opportunities for application of learned knowledge in evidence-based critical-thinking activities. We sought to improve student performance on evidence-based critical-thinking tasks through the implementation of peer learning and problem-based learning tutorial activities. Small-group discussions and associated learning activities were used to facilitate deeper learning through the application of new knowledge. Student performance was assessed using critical-thinking essay assignments and a final course exam, and student satisfaction with tutorial activities was monitored using online surveys. Overall, students expressed satisfaction with the small-group-discussion-based tutorial activities (mean score 7.5/10). Improved critical thinking was evidenced by improved student performance on essay assignments during the semester, as well as a 25% increase in mean student scores on the final course exam compared to previous years. These results demonstrate that repeated knowledge application practice can improve student learning in large introductory-level science courses.     

Promoting deep learning in a teacher education programme through self- and peer-assessment and feedback

The incorporation of self- and peer-assessment and feedback has significant potential as a pedagogical strategy to promote deep learning in project based coursework. This study examined the impact of a deeper approach to learning on pre-service teachers’ critical thinking and metacognitive skills. It also examined the impact on student learning outcomes within a project based module with a significant design element. Forty-seven students participated in the pilot of an online peer feedback system. Results suggest that the quality of students’ reflections through peer feedback and overall satisfaction with the module remained high despite students’ citing a preference for instructor feedback. The data also indicate that the incorporation of self- and peer-assessment and feedback resulted in higher quality learning outcomes and enhanced critical thinking skills.     

Learning through Teaching: Challenges and Opportunities in Facilitating Student Learning in Food Science and Nutrition by Using the Interteaching Approach

ABSTRACT: Interteaching is a new pedagogical strategy for classroom instruction that demonstrates great effective student learning outcomes in the field of psychology. It is a 20 to 30 min student‐to‐student discussion addressing the main points in a specified body of reading materials. Interteaching includes elements such as reciprocal peer tutoring, cooperative learning, and problem‐based learning. These elements have been well theorized and their effectiveness has been empirically documented. To date, little is known about the effects of interteaching on students' perceived learning outcomes in food science and nutrition. This case study describes how the interteach method was employed in an undergraduate nutrition and food science course with specific examples of the tools used, such as interteach preparation guides, the interteach record form, and the peer assessment form. Based on the continuous feedback provided by the students during the course work, several specific modifications were made from the conventional interteaching methods, including 4‐person interteaching instead of one‐on‐one interteaching, as well as the use of in‐class thought‐provoking “synthesis” questions. At the end of the course, we assessed the students' perceptions of interteaching, as well as the students' perceived learning outcomes. The method generally fostered critical thinking and enhanced their motivation, which led to their perceived learning. On the other hand, some of the students expressed challenges in learning from peers due to their peers' varied preparation levels, as well as tackling complex scientific concepts prior to lectures. Further investigation is needed to develop possible strategies for accommodating the challenges among students with different learning styles.     

A preliminary discourse analysis of constructivist-oriented mathematics instruction for a student with learning disabilities

Reform efforts in mathematics education arose, in part, in response to constructivist works on conceptual learning. However, little research has examined how students with learning disabilities (LD) respond to constructivist-oriented instruction in mathematics, particularly in moment-to-moment interactions. To understand the nature of constructivist-oriented mathematics instruction involving students with LD, the authors conducted a case study to analyze teacher–student interactions during constructivist-oriented small group instruction involving a student with LD. The student demonstrated, to a certain degree, the ability to reason mathematically when provided with appropriate opportunities and prompting. However, given the limited intervention time, his reasoning and problem solving did not seem to go beyond the semiconcrete level of operation, which may have inhibited his solving of complex word problems with large numbers. Findings indicate that more efforts are needed to support students, those with LD in particular, in their transitions from concrete or semiconcrete to abstract conceptual understanding and problem solving.     

Helping preservice teachers learn to assess writing: Practice and feedback in a Web-based environment

Writing is a highly valued skill that is often neglected in the classroom; one reason is that teachers often do not receive adequate training in writing assessment and instruction. Teachers, particularly preservice teachers, need practice making detailed assessments of student writing and to build their confidence for assessing student writing, but practical issues of time and resources often constrain the frequency and quality of training they receive. This mixed method study focused on the design and evaluation of an online tool for building preservice teachers’ writing assessment skills and self-efficacy for writing assessment. In the study, teacher education students interacted with actual 4th-graders’ writing samples via a Web-based critical thinking tool. They received scaffolded practice in assessing multiple student papers and justified their assessments using analytic criteria. After each paper, they received feedback that included access to expert assessments and those of their peers, along with both teacher and peer rationales for their ratings. Participants significantly improved in ability to accurately assess student writing using an analytic approach and in knowledge of the writing traits. They also showed significantly greater self-efficacy for assessing student writing and high levels of satisfaction with the Web-based tool and their overall learning experience.     

非实时在线同侪互评环境下学生作者读者意识变化研究

本研究旨在探讨非实时在线同侪互评（ ALN-based peer assessment）促进学生作者读者意识的可行性。借助非实时在线同侪互评,学生作者接收同侪评价的同时,也对同侪的写作实施评价与参考,使其具备作者与读者双重角色,实现在线写作社群的构建,为培养读者意识提供所需的读者群。研究结果显示,学生的写作收到同侪较高频次的显性评价与非显性拜访,写作初稿与终稿的比对结果表明写作质量获得提升,体现出较高的互评频率和互评效能,因而判断有效在线写作社群得以构建。试验后问卷调查表明,有效在线写作社群的构建和该社群中学生作者评价同侪写作的倾向与学生读者意识的提高相关。     

Student Peer Teaching: An Innovative Approach to Instruction in Science and Engineering Education

This paper reports the results of a pilot-study in a senior paper science and engineering class, of an innovative instructional method designed to foster student problem-solving and in-depth learning of material, namely, student peer teaching. We review related literature focusing on active learning methods in science and engineering education, describe the method of student peer teaching used in this pilot-study, present the evaluation method and results, and discuss implications for further development of this method of instruction. Results suggest that students were able to effectively teach significant curricular content. In addition, the method of student peer teaching served important purposes in helping students develop in-depth understanding and expertise in the issues related to their teaching session, as well as teaching and presentation skills which will be useful in their professional practice. However, students expressed concern that, while achieving in-depth learning of the content of their teaching session, they tended to focus on the content area of their teaching session at the expense of other content areas and may not have learned as well from other students as from the Professor. We recommend a modified structure for student peer teaching which incorporates cooperative learning methods; increased Professor involvement in class sessions, in the role of the mentor; and modifications in performance evaluation methods to ensure ongoing student monitoring of progress and self-assessment. The method of student peer teaching, in science and engineering education, combined with cooperative learning methods, is viewed as a major extension of cooperative learning methods, used in the service of preparation for professional careers.     

Learning from reviewing peers’ concept maps in an inquiry context: Commenting or grading,which is better?

In peer assessment, both receiving feedback and giving feedback (reviewing peers’ products) have been found to be beneficial for learning. However, the different ways to give feedback and their influence on learning have not been studied enough. This experimental study compared giving feedback by writing comments and by grading, to determine which contributes more to the feedback providers’ learning. Secondary school students from Russia (n = 51) and the Netherlands (n = 42) gave feedback on concept maps during a physics lesson. The lesson was given in an online inquiry learning environment that included an online lab. Students gave feedback in a special Peer Assessment tool, which also provided assessment criteria. Findings indicate that post-test knowledge scores were higher for students from the commenting group. The difference between the groups was largest for the low prior knowledge students. Possible educational implications and directions for further research are discussed.     

The effects of skill diversity on commenting and revisions

The use of peer assessment to evaluate students’ writing is one recommended method that makes writing assignments possible in large content classes (i.e., more than 75 students). However, many instructors and students worry about whether students of all ability levels are capable of helping their peers. We examine how ability pairing (e.g., high-ability student with high-ability student versus high-ability student with low-ability student) changes key characteristics of feedback to determine which pairings are likely to benefit students most. A web-based reciprocal peer-review system was used to facilitate the peer review of students’ writing of two papers. Over 1,100 comments given to writers from their peers were coded for several relevant categories: type of feedback, type of criticism, focus of problem, focus of solution, and implementation. Overall, creating peer-review groups such that students receive feedback from someone of a dissimilar ability appeared to be most beneficial. High-ability writers received similar kinds of feedback from high-ability versus low-ability peers. By contrast, the low-ability writers received more comments that identified problems focusing on substance issues from high-ability reviewers. In addition, the low-ability writers implemented a higher percentage of the comments from the high-ability reviewers.     

Peer review in the classroom: student perceptions,peer feedback quality and the role of assessment

Abstract

Peer review in the classroom can enhance numerous employability skills such as critical appraisal, writing skills, reflection practices and collaborative experiences. This study takes place over two years and discusses the implementation of a repeating blind peer review cycle across a single semester for final year chemistry students enrolled on a compulsory employability module. The feedback cycle promotes personal reflection through the use of mini-reflective questionnaires. The process was assessed by academic tutors at the resubmission stage and/or the peer feedback stage where the quality of peer feedback was directly assessed. The research investigates the quality of peer feedback, the importance of assessment and student perceptions of what is most useful. Methods include directed content analysis of feedback produced, student opinions and a focus group. Students were capable of offering useable feedback across a range of assessment criteria but tend to focus on the important criteria best aligned to the particular assignment. A range of motivational factors and tactics were noticed but students tended to find reviewing the most beneficial.     

Online peer assessment: effects of cognitive and affective feedback

This study reports the effects of online peer assessment, in the form of peer grading and peer feedback, on students’ learning. One hundred and eighty one high school students engaged in peer assessment via an online system—iLap. The number of grade-giving and grade-receiving experiences was examined and the peer feedback was coded according to different cognitive and affective dimensions. The effects, on both assessors and assessees, were analyzed using multiple regression. The results indicate that the provision by student assessors of feedback that identified problems and gave suggestions was a significant predictor of the performance of the assessors themselves, and that positive affective feedback was related to the performance of assessees. However, peer grading behaviors were not a significant predictor of project performance. This study explains the benefits of online peer assessment in general and highlights the importance of specific types of feedback. Moreover, it expands our understanding of how peer assessment affects the different parties involved.     

The impact of two types of peer assessment on students' performance and satisfaction within a Wiki environment

The purpose of this study was to compare the effects of two peer assessment methods on university students' academic writing performance and their satisfaction with peer assessment. This study also examined the validity and reliability of student generated assessment scores. Two hundred and thirty-two predominantly undergraduate students were selected by convenience sampling during the fall semester of 2007. The results indicate that students in the experimental group demonstrated greater improvement in their writing than those in the comparison group, and the findings reveal that students in the experimental group exhibited higher levels of satisfaction with the peer assessment method both in peer assessment structure and peer feedback than those in the comparison group. Additionally, the findings indicate that the validity and reliability of student generated rating scores were extremely high. Using Wiki interactive software and providing an online collaborative learning environment to facilitate peer assessment added value to peer assessment.     

Does assessment for learning make a difference? The development of a questionnaire to explore the student response

Assessment for learning is a widely used term and the concept forms the basis of many teaching innovations in higher education. However, the definitions and scope of assessment for learning vary considerably. We describe a conceptualisation of assessment for learning that encompasses current thinking in a holistic way and which has been trialled and extensively refined in practice. A student questionnaire is presented which has enabled us to explore the student experience. Results indicate that the overall student experience is more positive in modules where assessment for learning approaches are used and students are more likely to take a deep approach to learning. It also demonstrates that the student experience is centred on staff support and module design, feedback, active engagement and peer learning. The full questionnaire is made available and its wider use in evaluation, enhancement and research is encouraged.