Promoting learning by leveraging the collaborative nature of formative peer assessment with instructional scaffolds

Over the past two decades, formative peer assessment has become a popular instructional approach. Initially, it was more readily applied in higher education but has since expanded to other educational levels, including primary and secondary education. The popularity is understandable given the increased amount of feedback by multiple peers and enhanced awareness of performance criteria. Although it is increasingly acknowledged by the research community that formative peer assessment is inherently a social endeavour, the collaborative nature is simultaneously the least-explored mechanism. The contributions in this special issue address this gap conceptualising peer assessment and peer feedback as both an individual and a collaborative learning practice. Furthermore, we highlight core learning conditions: learner characteristics, domain and task characteristics, and, finally, instructional scaffolds.     

The differential effects of task complexity on domain-specific and peer assessment skills

In this study the relationship between domain-specific skills and peer assessment skills as a function of task complexity is investigated. We hypothesised that peer assessment skills were superposed on domain-specific skills and will therefore suffer more when higher cognitive load is induced by increased task complexity. In a mixed factorial design with the between-subjects factor task complexity (simple, n?=?51; complex, n?=?59) and within-subjects factor task type (domain-specific, peer assessment), secondary school students studied four integrated study tasks, requiring them to learn a domain-specific skill (i.e. identifying the six steps of scientific research) and to learn how to assess a fictitious peer performing the same skill. Additionally, the students performed two domain-specific test tasks and two peer assessment test tasks. The interaction effect found on test performance supports our hypothesis. Implications for the teaching and learning of peer assessment skills are discussed.     

Accuracy feedback improves word learning from context: evidence from a meaning-generation task

The present study asked whether accuracy feedback on a meaning generation task would lead to improved contextual word learning (CWL). Active generation can facilitate learning by increasing task engagement and memory retrieval, which strengthens new word representations. However, forced generation results in increased errors, which can be detrimental for learning if they are not corrected. The goal of this study was to determine whether immediate feedback on response accuracy would ameliorate this risk. The study was conducted using an intelligent tutoring system, which presents target words in multiple contexts and prompts users to generate a target word meaning after each context. One group of participants (feedback group) received immediate feedback based on Markov Estimation of Semantic Association (MESA), which estimates the distance between the learner response and the target word meaning. The control group did not receive feedback. Results from conventional (pre/post-test) measures showed greater gains in accuracy and confidence for the feedback group. Moreover, when contextual support was decreased mid-way through the training (from trial 3 to trial 4), MESA measures showed a corresponding drop in accuracy, but only for the No-Feedback group. These findings suggest that accuracy feedback can improve outcomes in CWL, particularly when there is an increased risk of errors. This strengthens the case for meaning generation as a tool to build high-quality lexical representations.     

合作学习模式下形成性评估在英语专业教学中的探索

有效评估对促进外语教学至关重要。结合问卷调查和个案研究,本研究旨在探讨如何在合作学习模式下将形成性评估应用于英语专业教学中。结果表明,形成性评估有利于端正学生学习态度,改进学习策略和提高学习效果;合作学习模式有助于促进形成性评估的实施,对完善评估方式有重要意义。     

Formative peer assessment in primary school: the effects of involving pupils in setting assessment criteria on their appraisal and feedback style

This experimental study is aimed at examining the effect of involving pupils in primary education in setting assessment criteria and standards on their appraisal of a peer’s work and their peer feedback style. It is expected that our intervention will lead to (1) a criteria-referenced appraisal and (2) more final vocabulary, which is associated with a more authoritative style. In total, 95 sixth grade pupils from The Netherlands were randomly assigned to one of two conditions. In the experimental group, the pupils first received an exemplar, which was followed by a group discussion about appropriate assessment criteria and standards. In the control group, the pupils provided peer feedback without this discussion. Their peer appraisal was measured with a questionnaire and feedback style was determined by analysing the written feedback using a person-oriented approach (cluster analysis). Results showed that the chance that pupils in the experimental group had an authoritative style was three times higher than in the control group. Theoretical and practical implications of the study are discussed.     

The effect of peer feedback for blogging on college students' reflective learning processes

Reflection is an important prerequisite to making meaning of new information, and to advance from surface to deep learning. Strategies such as journal writing and peer feedback have been found to promote reflection as well as deep thinking and learning. This study used an empirical design to investigate the interaction effects of peer feedback and blogging on college students' reflective thinking skills and their learning approaches. Forty-four first- and second-year undergraduate students participated in the study. Students kept blogs each week throughout a whole semester. Two journals were sampled at the beginning and end of the semester for each student. A repeated measure one-way ANOVA suggested that students' reflective thinking level increased significantly over time; however, peer feedback was found to negatively affect students' reflective thinking skills. The result of the study suggests more carefully designed uses in the future.     

Supporting student learning: the use of computer–based formative assessment modules

This paper describes the development of a variety of computer–based assessment opportunities, both formative and summative, that are available to a large first year biology class at The University of Sydney. These materials include: weekly quizzes; a mock exam; quiz sections in tutorials; and special self–assessment modules (SAMs). The weekly quiz is password protected and secure but the remaining materials are available on–line from a Virtual Learning Environment (http://FYBio.bio.usyd.edu.au/VLE/L1/). Evaluations over a number of years of the use and usefulness of the formative assessment materials indicate that the student population is making significant use of materials and that these materials are helping students in their learning     

Conceptions of and approaches to learning through online peer assessment

The present study investigated junior college students' conceptions of and approaches to learning via online peer assessment (PA) using a phenomenographic approach. Participants were 163 college students. Students were asked to accomplish a given learning task via an online PA system. Of the participants, 62 were interviewed after the activity. The interviews revealed hierarchically related and qualitatively different categories of conceptions and approaches to learning via online PA. The main and achieved levels of conceptions of and approaches to learning were determined. The results showed that, within each level, conceptions emphasizing a fragmented and cohesive learning tended to be associated with approaches focusing on surface and deep learning, respectively. In addition, students with cohesive learning conceptions and deep learning approaches were likely to make greater progress in the early stages of online PA activity. The present study finally found that approaches to learning via online PA were less related to the learning outcomes than conceptions of learning.     

The role of self-efficacy, task value, and achievement goals in predicting learning strategies, task disengagement, peer relationship, and achievement outcome

Adopting a combination of expectancy-value and achievement goal theories, this study examined the role of self-efficacy, task value, and achievement goals in students’ learning strategies, task disengagement, peer relationship, and English achievement outcome. A sample of 1475 Year-9 students participated in the study. A structural equation model showed that while task value predicted only mastery goals, self-efficacy predicted each of the three types of achievement goal. Mastery and performance-approach goals were both positive predictors of deep learning and peer relationship. Mastery goals were also negatively associated with task disengagement and positively associated with surface learning. In contrast, performance-avoidance goals were a positive predictor of surface learning and task disengagement but a negative predictor of peer relationship. On the whole, these findings suggest that, like mastery goals, performance-approach goals can generate adaptive outcomes.     

Effects of elicited reflections combined with tutor or peer feedback on self-regulated learning and learning outcomes

This study investigated the effects of students' reflections, combined with suggestive feedback, on the development of self-regulated learning and learning outcomes. Suggestive feedback alerts students that further reflection is advisable, without being very directive. Forty-nine students participated in a regular web-based distance education course on work psychology. In two experimental conditions students were prompted to reflect on their learning process, and received feedback on their reflections, either from peer students or a tutor. These conditions were compared to a control condition in which students studied the regular course without reflection and feedback. In line with the hypothesis, results showed that reflection combined with feedback positively affected students' self-regulated learning. In addition, students in the condition with tutor feedback outperformed students in both other conditions on learning outcomes.     

Informal formative assessment: The role of instructional dialogues in assessing students’ learning

This paper focuses on an unceremonious type of formative assessment - informal formative assessment - in which much of what teachers and students do in the classroom can be described as potential assessments that can provide evidence about the students’ level of understanding. More specifically, the paper focuses on assessment conversations, or dialogic interactions or exchanges, which continuously happen in the classroom and that are at the center of informal formative assessment. It is argued that assessment conversations make students’ thinking explicit in an unobtrusive manner, and when students’ thinking is explicit, it can be examined, questioned, and shaped as an active object of constructive learning. The paper conceptualizes informal formative assessment at the center of effective instructional activities with the use of instructional dialogues as assessment conversations, a typical informal formative assessment practice. The paper then presents a discussion about the evidence on the effect of assessment conversations on student learning.     

The alignment of written peer feedback with draft problems and its impact on revision in peer assessment

Despite the wide use of peer assessment, questions about the helpfulness of peer feedback are frequently raised. In particular, it is unknown whether, how and to what extent peer feedback can help solve problems in initial texts in complex writing tasks. We investigated this research gap by focusing on the case of writing literature reviews in an academic writing course. The dataset includes two drafts from 21 students, sampled to represent a wide range of document qualities, and 84 anonymous peer reviews, involving 1,289 idea units. Our study revealed that: (1) at both substance and high prose levels, drafts of all quality levels demonstrated more common problems on advanced writing issues (e.g. counter-argument); (2) peer feedback was driven by difficulty of the problem rather than overall draft quality, peer comments were not well aligned with the relative frequency of problems, more comments were given to less difficult problems; (3) peer feedback had a moderate impact on revision, and importantly, receiving multiple comments on the same issue led to more repairs and improvement of draft quality, but consistent with the comments received, authors tended to fix basic problems more often. Implications for practice and research are drawn from these findings.     

The overall effects of end-of-course assessment on student performance: A comparison between multiple choice testing, peer assessment, case-based assessment and portfolio assessment

This study investigates the effect of method of assessment on student performance. Five research conditions go together with one of four assessment modes, namely: portfolio, case-based, peer assessment, and multiple choice evaluation. Data collection is done by means of a pre-test/ post-test-design with the help of two standardised tests (N=816). Results show that assessment method does make a difference: assessments do not produce overall effects on student performance. Moreover, student-activating instruction efforts do not automatically result in more extensive learning gains. Finally, test results show, when compared to other assessments, a statistically significant positive effect of the multiple choice test on students' test scores. However, students' preparation level and the closed book format of the tests might serve explanatory purposes.     

Development and formative evaluation of multimedia case studies for Instructional Design and Technology students

This study describes the development of three case studies that included a combination of multimedia production and instructional design skills within a particular setting. These case studies incorporated real-life incidents from 47 professional instructional designers. These instructional designers described a total of 146 activities involving instructional design and/or multimedia production activities. Employing an emergent theme analysis approach, nine distinct themes developed from these activities and were included in the case studies. These themes included: Collaboration; Diffusion of Innovations; Formative Evaluation; Instructional Design; Interactive Instruction; Learner Analysis; Online Instruction; Technical Infrastructure; and Video Solutions. Fifty-seven Instructional Design and Technology graduate students evaluated the effectiveness of these case studies and considered the case studies as being valuable in furthering their understanding of related multimedia production and instructional design activities. The intent of these case studies was to enable Instructional Design and Technology students to comprehend the relationship between multimedia production and instructional design activities.     

Teacher beliefs about feedback within an assessment for learning environment: Endorsement of improved learning over student well-being

Teachers’ understandings of feedback probably influence the type and quality of feedback that they provide. The beliefs of 518 New Zealand practicing teachers about feedback were evaluated with the Teachers’ Conceptions of Feedback (TCoF) inventory and related to practices these teachers considered to be feedback. Nine feedback factors and four practices factors were found and models were statistically invariant between primary and secondary teachers. New Zealand teachers’ understandings of feedback were strongly focused on improving learning instead of enhancing student well-being. Similar factors are expected in other contexts, though agreement rates should reflect local policy priorities and cultural values.     

Bubble dialogue: A new tool for instruction and assessment

Described in this article is the development of a tool compatible with the constructivist approach to learning and instruction. Constructivism holds that instruction is less a process whereby knowledge is communicated to learners and more a matter of nurturing the processes whereby learners come to understand, construct interpretations in real-life settings, appreciate multiple perspectives, develop and defend their own positions while recognizing other views, and become aware of and able to manipulate the knowledge construction process itself. Bubble Dialogue is a HyperCard-based technique which combines elements of role play, comic strip creation, and reflexive dialogue analysis enacted in everyday settings. Based upon evidence accumulated over several years of development, the authors feel that this tool responds to the need for classroom-based techniques for promoting knowledge construction.