Instructional scaffolds for learning from formative peer assessment: effects of core task,peer feedback,and dialogue

Formative peer assessment is an instructional method that offers many opportunities to foster students’ learning with respect to both the domain of the core task and students’ assessment skills. The contributions to this special issue effectively address earlier calls for more research into instructional scaffolds and the implementation of dialogic features in formative peer assessment. However, open issues remain regarding the role of assessment criteria, the benefit of formative peer assessment for transferable knowledge and skills, the role of metacognitive and cognitive processes in the provision and reception of peer feedback, and the proposed benefit of more interactive forms of formative peer assessment. Addressing the latter issue in particular, a framework of three dimensions of increasing interactivity is proposed in order to guide future research. These three dimensions comprise the learner’s engagement with the core task (low interactivity), the provision and reception of peer feedback (medium interactivity), and the learner’s engagement with argumentation, tutoring, and co-construction in dialogue with peers (high interactivity).     

Transferable skills for online peer learning

Efforts to enhance learning through peer interaction in an electronic forum are now commonplace. However, facilitation and moderation of such a forum by academic staff can be of limited effectiveness and very time‐consuming. The skills required by peer learners to effectively manage such distributed discourse for themselves have rarely been clearly identified. This paper compares the social, organizational and cognitive characteristics of effective peer learning interactions in face‐to‐face and online environments (with particular reference to peer tutoring and peer assessment). This leads to a specification of transferable skills for online peer learning, at the macro and micro levels, with commentary on areas needing particular attention. The implications for assessment of such online distributed discourse process skills are explored with reference to professional practice and future research.     

An empirical review of anonymity effects in peer assessment,peer feedback,peer review,peer evaluation and peer grading

Abstract

Peer assessment has proven to have positive learning outcomes. Importantly, peer assessment is a social process and some claim that the use of anonymity might have advantages. However, the findings have not always been in the same direction. Our aims were: (a) to review the effects of using anonymity in peer assessment on performance, peer feedback content, peer grading accuracy, social effects and students’ perspective on peer assessment; and (b) to investigate the effects of four moderating variables (educational level, peer grading, assessment aids, direction of anonymity) in relation to anonymity. A literature search was conducted including five different terms related to peer assessment (e.g., peer feedback) and anonymity. Fourteen studies that used a control group or a within group design were found. The narrative review revealed that anonymous peer assessment seems to provide advantages for students’ perceptions about the learning value of peer assessment, delivering more critical peer feedback, increased self-perceived social effects, a slight tendency for more performance, especially in higher education and with less peer assessment aids. Some conclusions are that: (a) when implementing anonymity in peer assessment the instructional context and goals need to be considered, (b) existent empirical research is still limited, and (c) future research should employ stronger and more complex research designs.     

Improving the effectiveness of peer feedback for learning

The present study examined the effectiveness of (a) peer feedback for learning, more specifically of certain characteristics of the content and style of the provided feedback, and (b) a particular instructional intervention to support the use of the feedback. A quasi-experimental repeated measures design was adopted. Writing assignments of 43 students of Grade 7 in secondary education showed that receiving ‘justified’ comments in feedback improves performance, but this effect diminishes for students with better pretest performance. Justification was superior to the accuracy of comments. The instructional intervention of asking assessees to reflect upon feedback after peer assessment did not increase learning gains significantly.     

Managing Uncertainty During Collaborative Problem Solving in Elementary School Teams: The Role of Peer Influence in Robotics Engineering Activity

This study investigated how interaction with peers influenced the ways students managed uncertainty during collaborative problem solving in a 5th-grade class. The analysis focused on peer responses to individuals’ attempts to manage uncertainty they experienced while engaged in collaborative efforts to design, build, and program robots and achieve assignment objectives. Patterns of peer response were established through discourse analysis of work sessions for 5 teams engaged in 2 collaborative projects. Three socially supportive peer responses and 2 unsupportive peer responses were identified. Peer interaction was influential because students relied on supportive social response to enact most of their uncertainty management strategies. This study provides a useful theoretical contribution to understanding the roles of peer interaction in collaborative problem solving. Conceptualizing collaborative problem solving as a process of negotiating uncertainties can help instructional designers shape tasks and relational contexts to facilitate learning.     

Collaboration and Peer Tutoring in Chemistry Laboratory Education

The aim of this study is to examine the effectiveness of collaborative learning with hints and peer tutoring with hints, and individual learning with hints in chemistry laboratory education in a secondary school. A total of 96 eleventh graders participated in this study. The study has a randomized pre‐test and post‐test design with a delayed post‐test. During the four‐week intervention, students were required to carry out eight lab tasks in total. The students filled in a 17‐item self‐assessment of learning gain questionnaire on the last day. Analyses of students’ learning achievements showed that students in both the collaborative learning and peer tutoring situations outperformed those learning individually with hints. The delayed post‐test, which was administered three months later, revealed that students who had been in the peer tutoring situation outscored those in the collaborative learning situation. Student self‐assessment questionnaires on learning gain provided further details in this regard.     

Grudges and gratitude: the social-affective impacts of peer assessment

Abstract

Students’ dissatisfaction with peer assessment has been widely documented. While most relevant literature places focus on the cognitive (content and uptake of feedback) or structural (feedback design) dimensions, students’ emotions in peer assessment have received scant attention. This study investigates the social-affective impacts of peer assessment by analysing students’ appeal letters addressed to their tutors, reflective posts in the online discussion forum and responses to a survey. A thematic analysis of data indicated three main aspects of students’ (dis)satisfaction: content, scores and process of peer assessment. The most negative emotion that students expressed was related to ‘disrespectful’ behaviour and attitudes of peer reviewers, whereas the feeling of appreciation was triggered by the helpful feedback attributes which were perceived as reflecting reviewers’ respect to others’ works. Students generally held mixed feelings toward peer assessment, valuing learning in the process of providing and receiving feedback but showing resistance to using peer assessment for summative purposes. The findings highlight the significance of respect in peer assessment and argue that the perceived lack of mutual respect seems to underpin the nature of students’ dissatisfaction. This study carries implications for nurturing students’ respectful attitudes and behaviour in and through peer assessment.     

Outcomes of synergetic peer assessment: first-year experience

Active participation in learning activities and reviewing assessment activity can facilitate learners engaged in these processes. This case study reports student experiences of the process of peer assessment with teacher guidance in a group project for a first-year nursing course with 153 students. Twenty groups of students were assigned roles in exploring one of the two scenarios: one involved analysing diabetes mellitus and the other considered colorectal carcinoma, in each case covering provision of acute nursing care in hospitals and preparing health promotion strategies in a community. Each group also provided comments to the group that worked on the other scenario, based on prearranged assessment criteria. After receiving comments from their peer group, each group was then allowed to revise their drafts prior to final submission. The teacher’s input would be offered, if the peer feedback was inaccurate or insufficient. A survey was conducted to explore the learning experiences, and three semi-structured focus group interviews were conducted to triangulate findings with the survey. The first-year students expressed the need for both peer feedback and the lecturer’s direct guidance.     

ZPD-activated languaging and collaborative L2 writing

Recent second/foreign language (L2) research has witnessed the application of sociocultural tenets to L2 classrooms. This study aimed to probe whether Iranian L2 learners’ engagement in ZPD-activated collaborative dialogue, or ‘languaging’, mediates their learning process and, specifically, their appropriate use of metadiscourse to address content, organisation and audience issues in writing. English-as-a-foreign-language-writing classes at two universities were assigned to four different instructional conditions, namely, ZPD-activated collaborative, ZPD-free collaborative, fine-tuned L2-input provision and prevalent teacher-fronted approaches. The data comprised metadiscourse-oriented writing test scores, weekly writings and audio-recorded in-class collaborative dialogues of the ZPD groups. The results demonstrated the ZPD-activated collaborative writing approach significantly facilitated the learners’ appropriate use of metadiscourse. Complementary interpretative analysis of the ZPD participants’ metadiscourse-related episodes showed that the initially-other-regulated (undistributed) writing tasks became progressively reciprocal with less-skilled peers displaying microgenetic use of L2 metadiscourse resources and dynamically assuming more participatory roles in regulating the writing tasks. The findings suggest engagement in ZPD-activated ‘languaging’ is crucial to shaping opportunities for L2 learners’ microgenetic learning and to their longer term cognitive development. In sum, the application of sociocultural tenets and notions has shown to be highly fruitful in explaining the social genesis of L2 development as a higher order psychological functioning.     

Patterns of peer scaffolding in technology-enhanced inquiry classrooms: application of social network analysis

The purpose of this study was to identify types and patterns of peer scaffolding that occur during inquiry-based learning (IBL) group activities. It employed a single instrumental case approach that integrated quantitative and qualitative analyses of data gathered from 21 students in a ninth grade biology course. A verbal analysis, a content analysis, and a social network analysis (SNA) were performed to identify patterns in group interactions and refine emergent themes. First, nine types of peer scaffolding were identified and found to serve the goals of direction maintenance, cognitive structuring, and simplification. Second, three different patterns related to the high, mixed, and low prior knowledge levels of each group were identified. The high prior knowledge group provided peer scaffolding that focused attention on considerations key to developing their arguments, and this scaffolding may have improved the group’s work. In the mixed prior knowledge group, the students with greater prior knowledge were likely to support those with less prior knowledge. Together, these findings indicate the way students are grouped may impact observable patterns in peer scaffolding. Identifying the difficulties that learners face and the assistance they seek could help instructional designers and teachers identify areas in which students need support during IBL group activities. This study informs educators and practitioners of effective strategies for designing and implementing peer scaffolding to assist inquiry activities in technology-enhanced classroom settings.

     

Differential effects of problem-solving demands on individual and collaborative learning outcomes

The effectiveness and efficiency of individual versus collaborative learning was investigated as a function of instructional format among 140 high school students in the domain of biology. The instructional format either emphasized worked examples, which needed to be studied or the equivalent problems, which needed to be solved. Because problem solving imposes a higher cognitive load for novices than does studying worked examples it was hypothesized that learning by solving problems would lead to better learning outcomes (effectiveness) and be more efficient for collaborative learners, whereas learning by studying worked examples would lead to better learning outcomes and be more efficient for individual learners. The results supported these crossover interaction hypothesis. Consequences of the findings for the design of individual and collaborative learning environments are discussed.     

The effects of autonomy support on student engagement in peer assessment

Although peer assessment is widely implemented in higher education, not all students are highly engaged in it. To enhance student engagement in peer assessment, we designed and developed a web-based tool, autonomy-supportive peer assessment (ASPA), to support students’ need for autonomy when they conducted peer assessment. Students’ sense of autonomy, and their behavioral, emotional, and cognitive engagement in peer assessment were examined via interviews and surveys. We also examined students’ academic performances, including their improvement from initial to revised essays and the quality of feedback they provided. Survey results indicated that the ASPA group (1) experienced a slightly higher sense of autonomy than the non-ASPA group, and (2) spent much more time on each evaluation criterion than the non-ASPA group. Interviews suggested that both the ASPA and non-ASPA groups were engaged in peer assessment. However, there were no statistically significant differences between the two groups in behavioral, emotional, cognitive engagement, and academic performances.     

大学英语自主学习形成性评估实证研究

自主学习要求学习者监控学习进程和评价学习效果,传统的终结性评估不适用于自主学习。构建了适用于自主学习的形成性评估模式,该模式由"学习档案袋评价""教师评价"及"学生自评和他评"构成。经过在两个教学班级一学期的教学实验,结果表明:通过对学生自主学习过程的监控和管理,形成性评估激发了学生语言学习的动机和兴趣,促进了学生学习策略形成,有效培养了学生自主学习能力。     

Group peer assessment for summative evaluation in a graduate-level statistics course for ecologists

Peer assessment is often used for formative learning, but few studies have examined the validity of group-based peer assessment for the summative evaluation of course assignments. The present study contributes to the literature by using online technology (the course management system Moodle^?) to implement structured, summative peer review based on an anchored rubric in an ecological statistics course taught to graduate students. We found that grade discrepancies between students and the instructor were fairly common (60% of assignments), relatively low in value (mean = 3.3 ± 2.5% on assignments that had discrepancies) and proportionally higher for criteria related to interpretation of statistical results and code quality and organisation than for criteria related to the successful completion of analysis or instructional tasks (e.g. fitting particular statistical methods, de-identification of one’s submission). Students reported that the peer assessment process increased their exposure to alternative ways of approaching statistical and computational problem-solving, but there were concerns raised about the fairness of the process and the effectiveness of the group component. We conclude with some recommendations for implementing peer assessment to maximise student learning and satisfaction.     

面向时代需求 实现范式转变——赖格卢特论信息时代视角下的教学理论演进

教学是帮助学习者构建内化的知识的各种行为。信息时代所出现的变革呼唤着教学新范式。教学新范式既保留了已经被实践证明行之有效的基本教学方法(讲解、示证和练习),同时也鼓励采用变通教学方法(问题教学、项目学习、模拟学习、辅导学习和认知学徒等)。信息时代的教学理论遵循着四项基本原理,这就是个性化与多样性,主动性与自导性,协作性与情感性、整体性与一体化,而技术在记录学习进步、规划学习蓝图、提供学习指导和评估学习效果等方面也起到了不可或缺的保障作用。     

Peer relations in peer learning

Over the last decades, much research on peer learning practices has been conducted. Quantitative, experimental designs focusing on problems of cause and effect dominate. Consequently, effects on achievement are well documented, as is the influence of different conditions on the effect rate. In spite of the general acknowledgment of the importance of peer learning and a large amount of research on collective learning practices, questions regarding the quality of peer interaction, and how peer relations influence learning, are not well elaborated. This paper complements the discussion on effect focusing on the processes of interaction between peers, and relates these to theoretical perspectives on learning as fundamentally social. Inspired by meta-ethnography an integrative analysis across seven qualitative studies was accomplished. The approach enabled an investigation of peer interactions in different educational settings. The analysis elaborates on how instructional designs and students’ relational knowledge mediate interaction in peer learning. The paper further discusses the potential of approaches synthesising qualitative studies as a tool in qualitative research.     

Examination involving students as peer examiners

The main interest in this article is students’ involvement in assessment as a part of growth towards self‐directedness in learning. In order to enhance students’ development of autonomy in learning, a project involving ‘older’ students as peer examiners for ‘younger’ students was designed and carried out. Students in the sixth semester in a PBL‐based Master’s program of Medical Biology participated, together with faculty, as examiners of fifth‐semester students. The examination and the assessment situation was carefully designed based on learning theories, empirical evidence and experiences underpinning student‐centred learning, especially in the form of PBL used at the faculty. The project was evaluated and analysed in order to understand students’ learning processes related to the responsibility for assessing peers. The situation of the peer examiners was interpreted based on their own experiences with statements from the students assessed and faculty involved in the assessment. Evaluations from six occasions, spring and fall, 2003–2005, were included in the study. The findings suggest that involving students in assessment as equal partners with faculty makes it is possible for students to apprehend the metacognitive competences needed to be responsible and autonomous in learning. The peer examiners experience motivation to learn about learning, they acquire tacit knowledge about assessment and they learn through being involved and trusted. The student‐centred educational context, which requires responsibility throughout the programme, is recognized as very important.     

Teaching domain-specific skills before peer assessment skills is superior to teaching them simultaneously

Instruction in peer assessment of complex task performance may cause high cognitive load, impairing learning. A stepwise instructional strategy aimed at reducing cognitive load was investigated by comparing it with a combined instructional strategy in an experiment with 128 secondary school students (mean age 14.0?years; 45.2% male) with the between-subjects factor instruction (stepwise, combined). In the stepwise condition, study tasks in Phase 1 were domain-specific and study tasks in Phase 2 had both domain-specific and peer assessment components. In the combined condition, these two components were present in all tasks in both phases. Final performance (i.e. speed and accuracy in domain-specific skills and peer assessment skills) showed no significant differences, but performance improved more from Phase 1 to Phase 2 in the stepwise condition than in the combined condition. The results suggest that, with complex study tasks, it might be beneficial to teach domain-specific skills before peer assessment skills.     

Peer assessment in MOOCs: The relationship between peer reviewers’ ability and authors’ essay performance

In a relatively short period of time, massive open online courses (MOOCs) have become a considerable topic of research and debate, and the number of available MOOCs is rapidly growing. Along with issues of formal recognition and accreditation, this growth in the number of MOOCs being developed increases the relevance of assessment quality. Within the context of a typical xMOOC, the current study focuses on peer assessment of essay assignments. In the literature, two contradicting theoretical arguments can be found: that learners should be matched with same‐ability peers (homogeneously) versus that students should be matched with different‐ability peers (heterogeneously). Considering these arguments, the relationship between peer reviewers’ ability and authors’ essay performance is explored. Results indicate that peer reviewers’ ability is positively related to authors’ essay performance. Moreover, this relationship is only established for intermediate and high ability authors; essay performance of lower ability authors appeared not to be related to the ability of their reviewing peers. Results are discussed in relation to the matching of learners and instructional design of peer assessment in MOOCs.     

Promoting learning by leveraging the collaborative nature of formative peer assessment with instructional scaffolds

Over the past two decades, formative peer assessment has become a popular instructional approach. Initially, it was more readily applied in higher education but has since expanded to other educational levels, including primary and secondary education. The popularity is understandable given the increased amount of feedback by multiple peers and enhanced awareness of performance criteria. Although it is increasingly acknowledged by the research community that formative peer assessment is inherently a social endeavour, the collaborative nature is simultaneously the least-explored mechanism. The contributions in this special issue address this gap conceptualising peer assessment and peer feedback as both an individual and a collaborative learning practice. Furthermore, we highlight core learning conditions: learner characteristics, domain and task characteristics, and, finally, instructional scaffolds.