Explicitness of criteria in peer assessment processes for first-year engineering students

This article will outline the motives for the implementation of peer assessment as put into action at the first year of the Civil Engineering course of the University of Minho. The implementation of new assessment methods was a consequence of the successful implementation of peer assessment at other engineering courses. During the semester, three assessment moments took place, in which students assessed the work of their peers. Assessment criteria were defined in a negotiation process between students and teacher. The students subsequently graded the work of their colleagues and had to provide a clear justification for each given grade. The aim of this method is to involve students in the subject, enhance their motivation and deepen their learning. Advantages and disadvantages of peer assessment are discussed. The results that have been obtained so far draw attention to the nature of assessment criteria and the corresponding explicitness.     

Assessment for learning research in East Asian countries

Educational agencies in East Asia have heeded the advice of research findings and therefore acknowledge the value of assessment for learning (AfL) practices through various policy initiatives. At definitional level, the evolving conceptions and theories of AfL have consistently streamed in from overwhelmingly European and Anglophone-based research. We present a review of mainly peer-reviewed journal articles on selected AfL research in East Asian countries. The findings show that the current implicit and atheoretical approach towards defining and implementing AfL suggests opportunities for further deliberation and theorisation about what constitutes AfL in East Asian countries. It is conceivable that teachers who understand the principles and frequently prepare students for summative assessment in the East Asian classroom are concurrently practising a particular process and practice of AfL. We conclude that the practices of AfL can therefore not just be variable; they will also be very situated and contested.     

Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features

This paper reports a systematic literature review examining empirical studies on the effects of peer assessment for learning. Peer assessment is fundamentally a social process whose core activity is feedback given to and received from others, aimed at enhancing the performance of each individual group member and/or the group as a whole. This makes peer assessment an interpersonal and interactional process. Using this social perspective in order to study learning effects, we focus on the impact of the structural arrangement of peer assessment on learning, and the influence of interpersonal variables.The literature search, focusing on empirical studies measuring learning outcomes in a peer assessment setting, resulted in 15 studies conducted since 1990 dealing with effects (performance or perceived learning gains) of peer assessment. Our analysis reveals that, although peer assessment is a social process, interpersonal variables have hardly been studied; more specifically, they were measured in only 4 out of 15 studies. Moreover, they are not used to explain learning gains resulting from peer assessment. Finally, comparing the studies with respect to structural features reveals that, although the differences between the studies are significant, there seems to be no relation with the occurrence of learning benefits. The results of this review seem to indicate that research on peer assessment from a social perspective is still in its infancy and deserves more attention.     

A systematic review of prerequisites for implementing assessment for learning in classroom practice

Although many researchers acknowledge that Assessment for Learning can significantly enhance student learning, the factors facilitating or hindering its implementation in daily classroom practice are unclear. A systematic literature review was conducted to reveal prerequisites needed for Assessment for Learning implementation. Results identified prerequisites regarding the teacher, student, assessment and context. For example, teachers must be able to interpret assessment information on the spot, student engagement in the assessment process is vital, assessment should include substantial, constructive and focussed feedback, and the school should have a school-wide culture that facilitates collaboration and encourages teacher autonomy. The results of this review contribute to a better understanding of the multiple facets that need to be considered when implementing Assessment for Learning, from both a theoretical and a practical standpoint.     

The quality of procedures to assess and credit prior learning: Implications for design

The ever-changing requirements of working life require individuals to develop their competencies throughout their life cycle. This lifelong learning paradigm requires a renewed vision concerning assessment in which, besides formal learning, informal and non-formal learning experiences are also recognized. To support this lifelong learning paradigm, procedures have been developed worldwide to assess and credit prior learning experiences (APL). While research on APL stresses the importance of a high-quality standard, so far the literature has applied only a psychometric quality framework. However, from the perspective of APL, where, besides prior knowledge and skills, competencies need to be measured, it is more appropriate to use a combination of the psychometric and edumetric quality criteria. This article will analyze and describe the relationship between quality criteria and the characteristics of APL. The results have revealed that quality criteria based on both are fundamental for APL, but that some criteria are more recognized than are others. Based on this analysis, design guidelines for APL have been formulated.     

Small group discussion as a key component in online assessment training for enhanced student learning in web-based peer assessment

The purpose of this study is to examine the effect of online assessment training, with synchronous group discussion as a key component, on subsequent web-based peer assessment results. Participants included 81 college students, mostly women, taking a business writing class. After initial submission of a draft counter-offer letter, they completed online assessment training by engaging in a consensus-seeking, synchronous group discussion of the rubric-based ratings they gave to sample counter-offer letters. They then engaged in web-based assessment of randomly assigned peer counter-offer letters and upon receiving peer feedback, submitted revision of their own counter-offer letter. The discussion groups were randomly assigned to either anonymous (using a pseudo name) or identified conditions (using real name). Findings indicate that the quality of student counter-offer letters improved after online assessment training, and improved even more after web-based peer assessment. There were no significant differences between anonymous discussion groups and identified groups.     

Assessment for learning in the Hong Kong assessment reform: A case of policy borrowing

This paper describes Hong Kong’s borrowing, primarily from the UK, of the assessment for learning policy, in the context of prolonged use of formal summative public examinations. The narrative review and analysis are guided by a social positivist critique of the assessment for learning policy in the Hong Kong context. This paper concludes that Hong Kong’s attempts to implement assessment for learning are ambitious but somewhat futile because of persistent use of public examinations for important decision-making. Change in Hong Kong may only happen through a gradual implementation of assessment for learning practices that takes into account cultural, societal, and historic norms. Critical issues are identified and recommendations are proposed for further implementation of assessment for learning in Hong Kong.     

Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks

Because learning and instruction are increasingly competence-based, the call for assessment methods to adequately determine competence is growing. Using just one single assessment method is not sufficient to determine competence acquisition. This article argues for Competence Assessment Programmes (CAPs), consisting of a combination of different assessment methods, including both traditional and new forms of assessment. To develop and evaluate CAPs, criteria to determine their quality are needed. Just as CAPs are combinations of traditional and new forms of assessment, criteria used to evaluate CAP quality should be derived from both psychometrics and edumetrics. A framework of 10 quality criteria for CAPs is presented, which is then compared to Messick's framework of construct validity. Results show that the 10-criterion framework partly overlaps with Messick's, but adds some important new criteria, which get a more prominent place in quality control issues in competence-based education.     

The impact of formative assessment on student achievement: A study of the effects of changes to classroom practice after a comprehensive professional development programme

A random sample of 22 Year 4 teachers in mathematics from a middle-sized Swedish municipality participated in a teacher professional development programme in formative assessment. The content of the programme was formative assessment conceptualised as a unity of different, integrated strategies. The study examines the effects on student achievement of the changes in the teachers’ formative classroom practice that followed the professional development input. Results show that, after controlling for pretest scores, the classes in the intervention group significantly outperformed the classes in the control group in a posttest administered one school year after the end of the programme (p = 0.036, d = 0.66). The study contributes to the understanding of under-studied areas of the impact of professional development in formative assessment, and the impact of teacher practice based on formative assessment conceptualised as a unity of different formative assessment strategies.     

Integrating assessment for learning in the teacher education programme at the University of Oslo

Abstract

This article provides an analysis of the integration of assessment for learning principles in the newly revised five-year Master of Education programme at the University of Oslo, Norway, across didactic subjects, pedagogy and school practice. The analysis draws on lecture notes, student videos and student exam papers among 143 student teachers, aiming to identify (a) the operationalisation of the assessment curriculum at the university campus, and in school practice, (b) how the student teachers use assessment principles as tools in their instructional designs and (c) how they self-assess their teaching practice. Our main finding is that student teachers seem to be more concerned with assessing their students than using self-assessment to improve their instruction. Based on the findings, we argue the importance of relating the teaching and learning activities with the assessment situations used in the teacher education programme.     

Formative self-and peer assessment for improved student learning: the crucial factors of design,teacher participation and feedback

Self and peer-assessment are becoming central aspects of student-centred assessment processes in higher education. Despite increasing evidence that both forms of assessment are helpful for developing key capabilities in students, such as taking more responsibility for their own learning, developing a better understanding of the subject matter, assessment criteria and their own values and judgements, and developing critical reflection skills, both forms of assessment are still not the norm at universities. This paper provides the findings of a two-year study of formative self and peer-assessment at an Australian university. The study supports other research showing that students tend to regard formative self and peer-assessment as beneficial for gaining more insights about the assessment process and for improving their own work. We argue that self and peer-assessment requires careful design and implementation for it to be an effective tool for formative assessment processes; and that the development of students’ capacities for giving feedback, and the continuous and timely involvement of the teacher, are central aspects for successful self and peer-assessment. The move to self and peer-assessment is not simple for teachers and students but is worthwhile and necessary for twenty-first century higher education.     

Is assessment for learning feasible in large classes? Challenges and coping strategies from three case studies

ABSTRACT

While assessment for learning (AfL) has been widely used in college English classrooms in China where teachers usually teach large classes, how AfL is implemented in the large-class higher education (HE) context is relatively underexplored. To address this gap, this study explores some of the challenges presented by large classes and the teachers’ coping strategies through case studies of three university English teachers’ AfL practices. Challenges are identified surrounding how the teachers dealt with (a) inadequate attention to individual students, (b) reduced opportunities for individualised feedback, (c) overwhelming marking responsibilities, and (d) involving students in assessment, with strategies of putting students in groups, matching names with faces, conducting teacher-student conferences, utilising technology to facilitate feedback processes, enabling peer feedback, and transforming students from assessees to assessors. Findings are framed around useful forms of AfL in large classes and the feasibility of AfL in terms of compromises between teacher agency and their temporal and contextual conditions. The paper concludes with implications for teacher assessment training, teacher education and policy-making.     

Feedback in Hong Kong secondary writing classrooms: Assessment for learning or assessment of learning?

While much of L2 teacher feedback research has focused on the effectiveness of feedback and its impact on student revision and writing, little has been done to examine teachers’ feedback in the larger classroom context of teaching and learning to ascertain the functions teacher feedback serves from an assessment-for-learning perspective. Using multiple sources of data from 26 secondary teachers’ written feedback to 174 student texts, interviews with six of the teachers and 18 students, the present study investigates the nature of teacher feedback and the functions it serves in the teaching-learning-assessment process in the writing classroom. The findings show that teacher feedback focuses largely on assessing writing summatively, primarily serving the purpose of assessment of learning, rather than assessment for learning – i.e., using feedback as a pedagogical tool for improving the teaching and learning of writing. The study calls for greater attention to the implementation of assessment for learning in the writing classroom, and specifically the use of feedback for formative purposes.     

Supporting teacher responsiveness in assessment for learning through disciplined noticing

Abstract

Assessment for learning (AfL) practices are commonly recommended as effective classroom strategies for providing teachers with information about student understanding. For teachers, the substantive potential of these AfL practices to inform student learning actions depends on what teachers notice and select as a focus and how they interpret and act on the information they have. This paper suggests research on teacher professional noticing has something to offer in understanding how teacher AfL attention and actions are framed in the moment. It explains noticing as a responsive act that invites action that is an inclusive, dynamic and purposeful response to evidence of student ideas. Noticing is what enables a teacher to act on the fly because it informs and underpins possible actions. Three frames of noticing are introduced to guide noticing and action – curriculum connoisseurship, cultural and community connectedness, and collaborative ways of working. Through consideration of classroom events using these frames, teachers become better prepared to choose to respond in the moment creatively. The three frames provide teachers and researchers with a conceptual language to articulate and sharpen their AfL practices as part of a flexible and rigorous responsiveness to their students’ learning.     

The quality assurance of a national English writing assessment: Policy implications for quality improvement

The purpose of this study was to examine the quality assurance issues of a national English writing assessment in Chinese higher education. Specifically, using generalizability theory and rater interviews, this study examined how the current scoring policy of the TEM-4 (Test for English Majors – Band 4, a high-stakes national standardized EFL assessment in China) writing could impact its score variability and reliability. Eighteen argumentative essays written by nine English major undergraduate students were selected as the writing samples. Ten TEM-4 raters were first invited to use the authentic TEM-4 writing scoring rubric to score these essays holistically and analytically (with time intervals in between). They were then interviewed for their views on how the current scoring policy of the TEM-4 writing assessment could affect its overall quality. The quantitative generalizability theory results of this study suggested that the current scoring policy would not yield acceptable reliability coefficients. The qualitative results supported the generalizability theory findings. Policy implications for quality improvement of the TEM-4 writing assessment in China are discussed.     

Peer review in the classroom: student perceptions,peer feedback quality and the role of assessment

Abstract

Peer review in the classroom can enhance numerous employability skills such as critical appraisal, writing skills, reflection practices and collaborative experiences. This study takes place over two years and discusses the implementation of a repeating blind peer review cycle across a single semester for final year chemistry students enrolled on a compulsory employability module. The feedback cycle promotes personal reflection through the use of mini-reflective questionnaires. The process was assessed by academic tutors at the resubmission stage and/or the peer feedback stage where the quality of peer feedback was directly assessed. The research investigates the quality of peer feedback, the importance of assessment and student perceptions of what is most useful. Methods include directed content analysis of feedback produced, student opinions and a focus group. Students were capable of offering useable feedback across a range of assessment criteria but tend to focus on the important criteria best aligned to the particular assignment. A range of motivational factors and tactics were noticed but students tended to find reviewing the most beneficial.     

Quality of WIL assessment design in higher education: a systematic literature review

We investigated the quality of work-integrated learning (WIL) assessment design in higher education programmes, through review of peer-reviewed studies published internationally and in English, 1990–2015. Such a review is timely in light of vested interests from a range of WIL stakeholders; high-level endorsement of WIL across university programmes; a regulatory environment requiring development and assurance of higher-order learning outcomes; and a WIL assessment literature that identifies a number of challenges and opportunities. We searched six electronic databases, yielding 20 intervention studies that met inclusion criteria. Findings reveal high-quality assessment design, albeit a need for greater involvement of industry/professional partners in assessment practices and stronger alignment between reflective assessment tasks and students' WIL experiences. The evidence base under review largely comprised qualitative and mixed methods studies, with the indication that the quality of the study design had improved over time, although variably across disciplinary fields. The key recommendation from this review is that resources are needed to support research-active WIL academics, partners and students to: (a) design and participate in assessment practices, which promote integration of student learning, across university and work settings, and achievement of higher-order learning outcomes and (b) pursue a collaborative research agenda involving robust evaluation research, inclusive of quantitative studies.     

Promoting assessment for learning through curriculum-based performance standards: teacher responses in the Northern Territory of Australia

Governments in Australia claim that standards-based reforms to schooling will result in greater use of assessment for and as learning. This study analyses the assessment practices that evolved within the planned curricula for senior secondary schooling in the Northern Territory of Australia during standards-based reforms. Case-study methodology was used, with each of the six teachers participating forming one case. The data collected and analysed included the texts created by the six participating teachers from a range of subject areas as they planned to enact the new curriculum. Semi-structured interviews were also conducted with each of the six teachers. The findings indicate that the inclusion of subject-based performance standards in the curriculum for senior schooling did not result in teachers planning a curriculum that included assessment practices for actively engaging students in learning.     

Assessment for learning: using programmatic assessment requirements as an opportunity to develop information literacy and data skills in undergraduate students

Departments and programmes in higher education are required to participate in an increasing number of programme, course and student assessments. These assessment requirements are also opportunities to develop student skills related to scientific literacy and research, if students are included in the process of developing, administering and interpreting these assessments. This paper describes a course designed to build student research skills through incorporating undergraduate students into the process of programme review for a psychology department, a comprehensive assessment required of this department every five years. This course proved to be an effective way to engage students, as students developed and administered assessment surveys, analysed and interpreted results, and prepared both a professional report for the department as well as a research presentation. The paper discusses recommended course activities, and shows how this opportunity can benefit students and faculty in a myriad of ways.     

A typology for a social justice approach to assessment: learning from universal design and culturally sustaining pedagogy

This paper aims to provide a tentative roadmap for ensuring that higher education policy makers and practitioners are apprised of what might be done to advance a concept of socially just assessment praxis. It extends current thinking around the notion of social justice approaches to assessment by further developing the conceptual framework proposed in McArthur's recent work (2016). It does so by extending understandings of how a socially just perspective might be realised. Drawing upon recent conceptual developments within both Universal Design for Learning (UDL) and Culturally Sustaining Pedagogy (CSP), the paper proposes a typology for praxis and organisational change. Crucially, this typology focuses upon enhancing learning outcomes for all learners, but it is particularly concerned with enhancing educational experiences and learning outcomes for students that have been systematically marginalised by the normative procedural practices that have traditionally informed the nature of supposedly objective assessment.