The Use of Exemplars and Formative Feedback when Using Student Derived Marking Criteria in Peer and Self-assessment

This paper reports a study which implemented a method of student self and peer assessment involving student constructed marking criteria in the presence of exemplars. Pairs of first-year undergraduate biology students were asked to complete a poster assignment. The study was designed to allow the evaluation of (1) student self and peer marking and tutor marking for individual marking criteria following the use of exemplars; (2) the role of exemplars in providing a focus for formative feedback about subject standards. The present study shows that: (a) the use of exemplars can help students demonstrate greater understanding of both marking criteria and subject standards; (b) the use of exemplars can help students learning so that higher quality outcomes are produced; (c) the use of exemplars forms a focus for meaningful formative feedback; (d) students may make more objective judgements as a result of peer assessment compared to self-assessment.     

Making learning effective – quantity and quality of pre-service teachers’ feedback

ABSTRACT

Feedback is an essential formative assessment practice that has the potential to influence student learning and achievement positively. Providing effective feedback, however, is a challenging task for teachers. Especially beginning teachers struggle with the provision of information that supports students in developing and improving their competences. Learning to provide feedback thus is an important aspect of teacher education. The present exploratory study investigates pre-service chemistry teachers’ practices regarding the judgement of students’ level of achievement and the provision of feedback in the context of the control-of-variables strategy. A sample of N?=?40 bachelor and master students at Kiel University judged students’ written artefacts and provided feedback with the aim to support students in their progression towards the learning goals. The quality of the feedback was coded with respect to its correct judgement of the students’ current levels of achievement and the support it provided regarding next steps in learning. The results show that only a few of the pre-service teachers could correctly judge students’ current levels of achievement and provide feedback that is expected to be effective. Overall, the findings indicate a need for more research in this field and a need for the implementation of formative assessment practices as a topic in pre-service teacher education.     

Peer assessment in thesis oral presentation

Peer assessment has been studied in various situations and actively pursued as a means by which students are given more control over their learning and assessment achievement. This study investigated the reliability of staff and student assessments in two oral presentations with limited feedback for a school-based thesis course in engineering where the different disciplines ran their own assessment. Staff scores were generally found to be more reliable than student scores, but it was not consistent. The engineering disciplines displayed widely varying differences between the staff and student scores. A large variation in reliability was found for the different disciplines making it difficult to reconcile to a standard school-based overall grading scheme for the course when the reliability of marking was low. The results also showed that the average scores for oral presentations are generally much higher than for written examinations, based on the grade point average. Future research will need to investigate how best to develop a consistent assessment framework for oral presentations and moderation of scores for consistency, particularly for large classes where assessment are done by different groups of staff and students and collated into a single set of marks for the class.     

Constructive guidance and feedback for learning: the usefulness of exemplars,marking sheets and different types of feedback in a first year law subject

Criteria and standards‐based assessment models are increasingly being adopted by universities as effective practice. However the promise of these models of assessment may not be realised unless teachers can find ways of making criteria and standards understandable to students. Exemplars or examples of previous students’ work of high and low quality can make criteria and standards concrete. Recent research has focussed on the use of exemplars to help students understand criteria and standards, and less emphasis has been given to exemplars simply as guides for students. This mixed methods study explores students’ perceptions of the usefulness of exemplars and different types of feedback for guiding them in completing assessments. A combination of engaging in marking and discussing exemplars, and receiving individualised and standards‐based feedback provides the most helpful guidance for students’ effective learning.     

Effects of teacher professional characteristics on student achievement: an investigation in blended learning environment with standards-based curriculum

Use of different teaching materials and curriculum for the same subjects is always a confounding factor in studies investigating teacher characteristics and student achievement. The purpose of this study is to understand the effects of different teacher qualities on student achievement in high schools with a standards-based curriculum delivered over a blended learning environment. Utilizing quantitative research approach, this study investigates the effects of teacher characteristics and teaching practices in a course offered through Cisco Networking Academy, which has a standards-oriented curriculum delivered online. Participants in the study were 226 teachers and 3299 students from Networking Academies located in the USA. The findings suggest that certain teacher characteristics such as teaching experience, degree, and primary teaching field do not have effects on student achievement.     

Developmental assessment: lifting literacy through professional learning teams

Outcomes and findings from an evidence‐based approach to targeting primary school students’ developmental reading comprehension levels for effective learning are described. Nineteen schools participated in a literacy assessment project designed to monitor and improve the reading comprehension achievement levels of their students. The project integrated a developmental approach to learning and teaching, information derived from standardised reading comprehension assessments, and professional development for teachers. Reading comprehension achievement across the schools increased at a higher rate than typically expected. Teacher discourse about teaching and learning changed from discrete skill and resource focused, to developmentally focused. The centrality of a professional learning team approach to change for the student, teacher and school is discussed.     

Representations of teaching and standards-based reform: are we closing the debate about teacher education?

Since the mid-eighties, national organizations have been working together in an effort to reform schools and, more specifically, to reform teaching. Paralleling the movement toward developing curriculum standards for students, professional standards for teachers have also been developed for the purpose of teacher education program accreditation. The objective of this national coalition is to strengthen the teaching profession and raise its standards—eventually enhancing the quality of student learning—by redesigning teacher licensing and accountability requirements for teacher education programs, and engaging teachers in on-going professional development. In this study we address three specific questions: (1) what representations of teaching and teachers are portrayed in the professional teaching standards, their related policies and assessment? (2) how are standards-based reform policies affecting teacher education programs? (3) what representations or conceptions of teaching and teachers are currently reflected in teacher education programs in the context of this reform? To address these questions two states were selected as test cases. Reform documents, policies and practices, as well as interviews with key participants in the reform (e.g., teacher educators, state-level administrators) are described and analyzed, and constitute the evidential basis for this study. The patterns emerging from the data indicate that teacher educators’ degrees of resistance or cooperation with externally imposed frameworks is influenced by their conception of teaching, education and its purpose. Further, as teacher educators uncritically participate in the standards-based movement it becomes impossible for them to entertain alternative perspectives on teaching and education outside of the framework provided to them by the standards.     

Effective Professional Development in Science and Mathematics Education: Teachers' and Facilitators' Views

This study compares the views of teachers and professional development facilitators about effective professional development (PD). We analyzed interviews with 72 teacher participants and 23 PD facilitators involved in nine science and mathematics PD projects. The teachers' themes for characterizing effective PD included classroom application, teacher as learner, and teacher networking. Similarly, the PD facilitators discussed effective PD as having classroom application and experiences for teachers as learners. In addition, PD facilitators shared the need to develop collegial relationships with teachers and improve teacher knowledge. These views correspond to some of the standards and recommendations described in policy and research documents on effective PD. Criteria of effective PD in these documents that the participants did not mention included: (1) challenging teachers' content and pedagogical content knowledge with transformative learning experiences, (2) encouraging teacher leadership for sustained support, and (3) focusing on student learning by instructing teachers on how to use student data to inform their teaching practice. Our findings have implications for designing PD that reflects the criteria of standards-based reform.     

Using peer and teacher-student exemplar dialogues to unpack assessment standards: challenges and possibilities

Dialogic use of exemplars is effective in developing student understanding of assessment standards. However, limited studies have investigated how exemplar dialogues are conducted in the post-secondary context. To fill the gap, this teacher-research explores the characteristics of peer and teacher–student exemplar talk in three post-secondary classrooms and the challenges in the dialogic process. The participants involved 69 first-year post-secondary students, a critical friend and a teacher-researcher. Data analysis on selected dialogue excerpts, an open-ended survey, focus group interviews with the students, critical friend’s commentary and a teacher-researcher reflective journal indicated two major issues: (i) students’ lack of capacities to resolve socio-cognitive conflict in the peer dialogue; (ii) the dilemma of addressing students’ immediate assessment needs and developing their long-term evaluative judgements in the teacher–student dialogue. Recommendations are made to tackle the challenges and to orchestrate productive exemplar dialogues.     

A comparative judgement approach to teacher assessment

We report one teacher’s response to a top-down shift from external examinations to internal teacher assessment for summative purposes in the Republic of Ireland. The teacher adopted a comparative judgement approach to the assessment of secondary students’ understanding of a chemistry experiment. The aims of the research were to investigate whether comparative judgement can produce assessment outcomes that are valid and reliable without producing undue workload for the teachers involved. Comparative judgement outcomes correlated as expected both with test marks and with existing student achievement data, supporting the validity of the approach. Further analysis suggested that teacher judgement privileged scientific understanding, whereas marking privileged factual recall. The estimated reliability of the outcome was acceptably high, but comparative judgement was notably more time-consuming than marking. We consider how validity and efficiency might be improved and the contributions that comparative judgement might offer to summative assessment, moderation of teacher assessment and peer assessment.     

Effectively feeding forward from one written assessment task to the next

Most studies into lecturers’ written feedback focus on the types of feedback found to be effective when students have the opportunity to act on that feedback, revise their written assignment and improve the mark they receive. But often students do not have this opportunity. Typically, they receive a mark and feedback on an assignment that they will never be able to rework and resubmit. This can leave students unsure about what to do with the feedback they receive. This paper reports on the use of high impact written feedback from lecturers that significantly improved student outcomes and grades from one assessment task to the next. It examines a range of factors which together make feedback in this context effective including: assessment design, use of grading standards and tutor training. These findings from a very large unit have significant implications for teaching staff who want to use feedback to feed forward and make a real difference to their students’ learning.     

Using Automated Scores of Student Essays to Support Teacher Guidance in Classroom Inquiry

Computer scoring of student written essays about an inquiry topic can be used to diagnose student progress both to alert teachers to struggling students and to generate automated guidance. We identify promising ways for teachers to add value to automated guidance to improve student learning. Three teachers from two schools and their 386 students participated. We draw on evidence from student progress, observations of how teachers interact with students, and reactions of teachers. The findings suggest that alerts for teachers prompted rich teacher–student conversations about energy in photosynthesis. In one school, the combination of the automated guidance plus teacher guidance was more effective for student science learning than two rounds of personalized, automated guidance. In the other school, both approaches resulted in equal learning gains. These findings suggest optimal combinations of automated guidance and teacher guidance to support students to revise explanations during inquiry and build integrated understanding of science.     

First-year students working with exemplars: promoting self-efficacy,self-monitoring and self-regulation

In this paper, we explore first-year undergraduate students’ experiences and perceptions regarding the use of exemplars, with reference to the development of student self-regulation. The study was carried out at a tertiary education institution in New Zealand where the second author was a lecturer. This author used exemplars with two classes of first-year students enrolled in a core introductory paper taught as part of the students' undergraduate business studies degree. Data were gathered through three complementary approaches: students’ responses to a short anonymous survey at the end of each of the four sessions where exemplars were used, written notes in the teacher’s Teaching journal and semi-structured interviews with student volunteers. When strategically employed as part of the teaching design, it was found that exemplars have the capacity to motivate, enhance students’ self-efficacy, promote understanding of task requirements, expand knowledge, understanding and skills in relation to the structure of academic tasks, support and advance subject knowledge, and develop evaluative and productive knowledge and expertise. We concluded the use of exemplars with these students demonstrated their potential as tools for the promotion of student self-efficacy, self-monitoring and to a lesser extent, self-regulation.     

A pedagogical alliance for academic achievement: Socio-Emotional effects on assessment outcomes

Assessment of student learning outcomes is often discussed in relation to curriculum, standards and even administration practices. However, assessment of learning outcomes is rarely discussed in light of students’ socio-emotional contexts, which might help or hinder learning outcomes. For example, do students’ perceptions of the teacher as trustworthy influence their empathic views on classmates, learning engagement, academic achievement and wellbeing? The present paper reports the results of a theoretically-driven, short-term longitudinal study designed to explore the effects of socio-emotional variables, specifically students’ trust in teachers and empathy towards self and peers on engagement for learning, academic achievement, and wellbeing. Grounded in attachment theory and the LEAFF model, Panel Structural Equation Modeling (P-SEM) results revealed three significant pathways, which provide evidence that student trust in teachers may serve to launch a domino effect of outcomes, predicting students’ empathic views of classmates, learning engagement, academic achievement, and wellbeing.     

Transitioning to learning outcomes at the coalface: An academic’s quantitative evaluation at the course level

Implementation of learning outcomes in Universities has not been seamless – their alignment with assessments requires faculty to specify learning outcomes and show transparent evaluation of this alignment. Difficulties arise transitioning from grading within a range of pre-existing assessment formats to grading common learning outcomes across those different formats. Experiencing this, we performed a quantitative evaluation of students completing an academic literacy course. We aimed to see if student achievement was determined by assessment format or by learning outcomes, and then to identify whether achievement of a learning outcome was equivalent across assessment formats. A twostep cluster analysis identified three clusters of students: high, medium, and low achievers. Strongest predictors of cluster-membership were learning outcomes assessed in the written essay, however scores for each learning outcome when assessed across assessment formats correlated poorly. Faculty should ensure consistent standards in learning outcomes achievement when assessed across different formats, or clearly separate learning outcomes.