A multi-peer assessment platform for programming language learning: considering group non-consensus and personal radicalness

Multi-peer assessment has often been used by teachers to reduce personal bias and make the assessment more reliable. This study reviews the design and development of multi-peer assessment systems that detect and solve two common issues in such systems: non-consensus among group members and personal radicalness in some assessments. A multi-peer assessment model is proposed to address these issues. The model captures roles, activities, and data structures in a typical multi-peer assessment setting that can be generalized to other scenarios. We implemented the model in a multi-peer code review system and conducted several empirical experiments in programming language classes. The studies showed that the model can significantly improve student learning outcomes than in a single-peer assessment. Also, we used statistical measures to detect non-consensus and radicalness issues that often occur in the model. The results reveal many insights and provide valuable guidance for teachers to implement a multi-peer assessment system.     

Supporting the less-adaptive student: the role of learning analytics,formative assessment and blended learning

Abstract

How can we best facilitate students most in need of learning support, entering a challenging quantitative methods module at the start of their bachelor programme? In this empirical study into blended learning and the role of assessment for and as learning, we investigate learning processes of students with different learning profiles. Specifically, we contrast learning episodes of two cluster analysis-based profiles, one profile more directed to deep learning and self-regulation, the other profile more directed toward stepwise learning and external regulation. In a programme based on problem-based learning, where students are supposedly being primarily self-directed, this first profile is regarded as being of an adaptive type, with the second profile less adaptive. Making use of a broad spectrum of learning and learner data, collected in the framework of a dispositional learning analytics application, we compare these profiles on learning dispositions, such as learning emotions, motivation and engagement, learning performance and trace variables collected from the digital learning environments. Outcomes suggest that the blended design of the module with the digital environments offering many opportunities for assessment of learning, for learning and as learning together with actionable learning feedback, is used more intensively by students of the less adaptive profile.     

A framework for classroom assessment,learning, and self-regulation

ABSTRACT

We present a conceptual framework that leverages synergies between classroom assessment (CA) practices and self-regulated learning (SRL) theory to support academic growth and instruction. We articulate the processes shared by CA and SRL, drawing on a model of SRL with three phases: forethought, performance, and self-reflection. We blend this SRL model with CA to create the CA:SRL framework in four stages: (1) pre-assessment, (2) the cycle of learning, doing, and assessing, (3) formal assessment, and (4) summarizing assessment evidence. We elucidate how SRL processes are involved at each stage and can be drawn on to support learning development and teacher understanding and co-regulation of learning. This framework is important in that it depicts how assessment and learning processes interact dynamically for both teachers and students in classrooms, and demonstrates that such interactions encompass the full breadth of purposes in CA, from planning through summation of evidence.     

Comparing student learning experiences of in-text commentary and rubric-articulated feedback: strategies for formative assessment

This study compares students’ experiences of two types of criteria-based assessment: in-text commentary and rubric-articulated feedback, in an assessment design combining the two feedback channels. The main aim is to use students’ responses to shed light on how feedback strategies for formative assessment can be optimised. Following action research methodology, the study discusses key categories of student responses from three sources: reflective texts, a questionnaire, and interviews. Results show that different functions were attributed to the two feedback channels: in-text commentary to lower-order concerns related to language proficiency, and rubric-articulated feedback to higher-order concerns related to an overview of writing achievement. We argue that the different functions have the potential of creating a sufficiently balanced assessment design with the possibility of serving both short-term and continuous learning goals. On the other hand, some students found it difficult to navigate between the two feedback channels. The article therefore ends with a ‘lessons learned’ section where we list possible ways in which the current assessment design can be improved for optimal use of the synergy effects emanating from a combination of in-text commentary and rubric-articulated feedback for formative purposes.     

Formative self-and peer assessment for improved student learning: the crucial factors of design,teacher participation and feedback

Self and peer-assessment are becoming central aspects of student-centred assessment processes in higher education. Despite increasing evidence that both forms of assessment are helpful for developing key capabilities in students, such as taking more responsibility for their own learning, developing a better understanding of the subject matter, assessment criteria and their own values and judgements, and developing critical reflection skills, both forms of assessment are still not the norm at universities. This paper provides the findings of a two-year study of formative self and peer-assessment at an Australian university. The study supports other research showing that students tend to regard formative self and peer-assessment as beneficial for gaining more insights about the assessment process and for improving their own work. We argue that self and peer-assessment requires careful design and implementation for it to be an effective tool for formative assessment processes; and that the development of students’ capacities for giving feedback, and the continuous and timely involvement of the teacher, are central aspects for successful self and peer-assessment. The move to self and peer-assessment is not simple for teachers and students but is worthwhile and necessary for twenty-first century higher education.     

Enhanced student learning in accounting utilising web-based technology,peer-review feedback and reflective practices: a learning community approach to assessment

Higher education is becoming a major driver of economic competitiveness in an increasingly knowledge-driven global economy. Maintaining the competitive edge has seen an increase in public accountability of higher education institutions through the mechanism of ranking universities based on the quality of their teaching and learning outcomes. As a result, assessment processes are under scrutiny, creating tensions between standardisation and measurability and the development of creative and reflective learners. These tensions are further highlighted in the context of large undergraduate subjects, learner diversity and time-poor academics and students. Research suggests that high level and complex learning is best developed when assessment, combined with effective feedback practices, involves students as partners in these processes. This article reports on a four-phase, cross-institution and cross-discipline project designed to embed peer-review processes as part of the assessment in two large, under-graduate accounting classes. Using a social constructivist view of learning, which emphasises the role of both teacher and learner in the development of complex cognitive understandings, we undertook an iterative process of peer review. Successive phases built upon students’ feedback and achievements and input from language/learning and curriculum experts to improve the teaching and learning outcomes.     

The relationship between student learning process,study success and the nature of assessment: A qualitative study

Assessment has an important role in affecting students’ learning, but qualitative studies regarding this relationship are rather scarce. In addition, course grades do not necessarily reflect the quality of learning outcomes. The aim is to qualitatively examine why high and low achieving students in the same course study differently according to the assessment method and how they explain their behaviour. Altogether 24 students were interviewed in this study. Analyses were conducted with inductive content analysis. The results indicate that course grades do not necessarily reflect qualitative learning outcomes due to the nature of the assessment. In addition, the present study suggests that self-regulation and motivation are important factors in studying and might prevent the negative effect of inappropriate assessment.     

Embedding librarians in online tertiary classrooms: A new model for learner support

Embedded librarians working in online classrooms present a new avenue of learner support direct to students in the space where they learn. The development of embedded librarian practice has evolved since 2004 and is becoming commonplace in online tertiary classrooms, but what is not yet so common is theory underlying this new practice. A multiple case study of embedded librarians in online tertiary classrooms in New Zealand was conducted to identify what makes this practice effective in providing support. Six factors were identified as contributing to effectiveness: relationality, immediacy, relevancy, interactivity, parity and faculty–librarian collaboration. The Embedded Librarian Support model was derived from, and is presented as the culmination of, the findings of this research.     

Dimensions of student success: a framework for defining and evaluating support for learning in higher education

ABSTRACT

The capabilities students need for success during and beyond higher education extend far beyond specific discipline skills to include the development of productive mindsets, the management of life circumstances and the way they relate to others and identify with their profession. The provision of support for these capabilities at university can be both diverse in scope and diffuse in delivery. Consequently, the development of streamlined and integrated evaluation strategies to measure the extent to which these capabilities are being successfully delivered can be challenging. This paper describes how one Australian university used a collaborative process to design an evaluation framework for student learning services. The framework, a first for this university, represents the breadth of student support, including a typology of support for learning: connectedness, mindsets, self-management, professional identity and academic capabilities. These terms, coined as dimensions, form the scaffold of university-wide delivery of support for learning initiatives.     

Transforming professional learning and practice in assessment for learning

Assessing student learning is a complex process requiring teachers to have deep knowledge of the curriculum, assessment, and pedagogy. Changing political climates mean that teachers are expected to respond to new approaches or systems and adjust their classroom practice accordingly. Teachers often engage in professional learning (PL) to assist their knowledge and classroom practice, but what impact does PL have on their assessment practice as well as student learning and achievement?

This paper examines optimal PL principles, the New Zealand assessment policy context, and application of assessment for learning principles in a nation-wide PL programme. Empirical data presented in relation to the five levels of Guskey's (2002) Guskey, T. R. (2002). Does it make a difference? Evaluating professional development. Educational Leadership, 59(6), 45–51.[Web of Science ®] , [Google Scholar] model evaluating PL indicate successful outcomes at the classroom level. However, application of Carless’ (2005) Carless, D. (2005). Prospects for the implementation of assessment for learning. Assessment in Education: Principles, Policies & Practices, 12(1), 39–45.[Taylor & Francis Online] , [Google Scholar] theoretical framework on embedding PL changes in schools reveals an area of neglect: attending to system (macro) level needs. Collaborative partnerships between schools and the wider community are posited as essential determinants of assessment literacies and transformational PL in times of political change.     

Small group discussion as a key component in online assessment training for enhanced student learning in web-based peer assessment

The purpose of this study is to examine the effect of online assessment training, with synchronous group discussion as a key component, on subsequent web-based peer assessment results. Participants included 81 college students, mostly women, taking a business writing class. After initial submission of a draft counter-offer letter, they completed online assessment training by engaging in a consensus-seeking, synchronous group discussion of the rubric-based ratings they gave to sample counter-offer letters. They then engaged in web-based assessment of randomly assigned peer counter-offer letters and upon receiving peer feedback, submitted revision of their own counter-offer letter. The discussion groups were randomly assigned to either anonymous (using a pseudo name) or identified conditions (using real name). Findings indicate that the quality of student counter-offer letters improved after online assessment training, and improved even more after web-based peer assessment. There were no significant differences between anonymous discussion groups and identified groups.     

The assessment of learning skills in nursing student selection: A scoping review

Abstract

Higher education student selection has significant societal, institutional and individual impacts. Thousands of applicants apply only for nursing, one of the major higher education disciplines. As the nursing profession is characterised by cognitive requirements, higher education institutions assess the learning skills of nursing applicants. However, there has been no comprehensive analysis of learning skills assessment for nursing student selection. The purpose of this scoping review was to describe the assessment of learning skills in undergraduate nursing student selection. Five databases were systematically searched, and 24 studies published between 2006 and 2016 were included. Learning skills were most commonly assessed using standardised tests in the areas of language and communication, reasoning, mathematics and natural sciences. Overall scores of onsite selection methods were found to best predict future academic performance. The results indicate that higher education institutions may benefit from comprehensive assessment of learning skills in their selection processes. This assessment should focus on a wider range of cognitive aptitudes, including reasoning skills. This review focussed on nursing education, but the results may benefit other higher education disciplines due to the generic nature of learning skills and similar cognitive requirements of higher education studies. The results support the development of more comprehensive and valid methods for assessing learning skills.     

A design-based trial of Lesson Study for assessment purposes: evaluating a new classroom based dynamic assessment approach

This paper is about a design-based research project which evaluated the novel use of Lesson Study for assessment (LSfA) purposes. It starts by explaining the principles and design of LSfA procedures based on a Lesson Study model and dynamic assessment principles. It outlines the training and support provided to six Lesson Study teams in three primary and three secondary schools, each involving two class teachers and the school’s SEN coordinator. The evaluation findings focused on the LSfA processes and procedures, for example, the outcomes of LSfA for teacher confidence and knowledge, assessment capabilities and continued benefits. The LSfA process was seen to involve processes such as honest and constructive observations, analysing lessons to see what to change and using knowledge about learning difficulties. Some of the context factors that supported and inhibited the process are also outlined. Analysis of the pre-post LSfA pupil assessments showed a decrease in identifying areas of pupil difficulties and an increased identification of enabling factors. This is interpreted as evidence for the LSfA having dynamic assessment potential. The paper concludes with an account of how the LSfA procedures were adapted in response to this evaluation and suggestions for future development and research.     

Assessment as learning: blurring the boundaries of assessment and learning for theory,policy and practice

This paper explores assessment and learning in a way that blurs their boundaries. The notion of assessment as learning (AaL) is offered as an aspect of formative assessment (assessment for learning). It considers how pupils self-regulate their own learning, and in so doing make complex decisions about how they use feedback and engage with the learning priorities of the classroom. Discussion is framed from a sociocultural stance, yet challenges some of the perspectives that have widely become accepted. It offers three new views to help explore the concept of AaL: understanding feedback; understanding the learning gap; and exploring vocabularies of assessment. Pragmatically, the ideas examined suggest that teachers may need to consider less about focused and directive feedback, but more about how learners interpret and understand feedback from their self-regulatory and self-productive identities and how vocabularies for assessment can be more collaboratively shared in learning contexts.     

Portfolios as a tool for AfL and student motivation: are they related?

ABSTRACT

The power of portfolios in enhancing student motivation is frequently emphasised. Portfolios are conceptualised as a tool to implement Assessment for Learning (AfL) in classroom practice. However, the relation between portfolios and AfL on the one hand, and student motivation on the other hand, is complex and subject to many assumptions. This study investigates whether portfolio use actually supports the integration of AfL in classroom practice and the relation with students’ motivation. Questionnaires were administered to a total of 419 grade 4 to 6 students from seven Dutch elementary schools. The results of structural equation modelling did not confirm a direct relation between portfolio use and students’ motivational orientation. The findings indicate that the relation between portfolios and motivational orientation is fully mediated by students’ perceptions of AfL. The findings indicate that the tool portfolio is not related to students’ motivation, however, AfL practices do affect motivation.     

A preliminary investigation and analysis of student learning style preferences in further and higher education

In the current further education (FE) and higher education (HE) environment, students are under increasing pressure to perform well and achieve good results. As educators we must strive to enable students to make the most of their higher education by providing an education of the highest quality and enhance the learning experience. This study was undertaken as a preliminary investigation of how students learn and how we, as educators, can improve their learning. In this study, a range of students on different (FE and HE) courses completed the VARK (visual, audio, read/write, kinaesthetic) questionnaire on learning style preferences at the beginning of the semester. Responses were analysed and the author adapted various teaching techniques into course materials for the semester to accommodate the results obtained in the initial VARK survey. Students' responses were encouraging, with many being keen to participate with a view to improving their course performance. The author found that using a range of teaching techniques encouraged student participation in the course and, for some, enhanced their performance in assessment. This investigation provides an insight into the awareness of learning style preferences and how such awareness can be positively exploited in teaching and learning processes.     

A systematic review of prerequisites for implementing assessment for learning in classroom practice

Although many researchers acknowledge that Assessment for Learning can significantly enhance student learning, the factors facilitating or hindering its implementation in daily classroom practice are unclear. A systematic literature review was conducted to reveal prerequisites needed for Assessment for Learning implementation. Results identified prerequisites regarding the teacher, student, assessment and context. For example, teachers must be able to interpret assessment information on the spot, student engagement in the assessment process is vital, assessment should include substantial, constructive and focussed feedback, and the school should have a school-wide culture that facilitates collaboration and encourages teacher autonomy. The results of this review contribute to a better understanding of the multiple facets that need to be considered when implementing Assessment for Learning, from both a theoretical and a practical standpoint.     

Assessment innovation and student experience: a new assessment challenge and call for a multi-perspective approach to assessment research

The impact of innovative assessment on student experience in higher education is a neglected research topic. This represents an important gap in the literature-given debate around the marketisation of higher education, international focus on student satisfaction measurement tools and political calls to put students at the heart of higher education in the UK. This paper reports on qualitative findings from a research project examining the impact of assessment preferences and familiarity on student attainment and experience. It argues that innovation is defined by the student, shaped by diverse assessment experiences and preferences, and therefore its impact is difficult to predict. It proposes that future innovations must explore assessment choice mechanisms which allow students to shape their own assessments. Cultural change and staff development will be required to achieve this. To be accepted, assessment for student experience must be viewed as a complementary layer within a complex multi-perspective model of assessment, which also embraces assessment of learning, assessment for learning and assessment for lifelong learning. Further research is required to build a meta-theory of assessment to enhance the synergies between these alternative approaches and minimise the tensions between them.     

Peer assessment for learning from a social perspective: The influence of interpersonal variables and structural features

This paper reports a systematic literature review examining empirical studies on the effects of peer assessment for learning. Peer assessment is fundamentally a social process whose core activity is feedback given to and received from others, aimed at enhancing the performance of each individual group member and/or the group as a whole. This makes peer assessment an interpersonal and interactional process. Using this social perspective in order to study learning effects, we focus on the impact of the structural arrangement of peer assessment on learning, and the influence of interpersonal variables.The literature search, focusing on empirical studies measuring learning outcomes in a peer assessment setting, resulted in 15 studies conducted since 1990 dealing with effects (performance or perceived learning gains) of peer assessment. Our analysis reveals that, although peer assessment is a social process, interpersonal variables have hardly been studied; more specifically, they were measured in only 4 out of 15 studies. Moreover, they are not used to explain learning gains resulting from peer assessment. Finally, comparing the studies with respect to structural features reveals that, although the differences between the studies are significant, there seems to be no relation with the occurrence of learning benefits. The results of this review seem to indicate that research on peer assessment from a social perspective is still in its infancy and deserves more attention.     

Quality of learning outcomes in an online video-based learning community: potential and challenges for student teachers

This study investigates the learning outcomes of 25 student teachers in an online video-based learning community (VBLC). Data were drawn from the student teachers' written comments and feedback recorded in the VBLC and the post-course interviews. Based on Biggs and Collis's Structure of Observed Learning Outcomes (SOLO) taxonomy, the majority of comments and feedback were classified as uni-structural, but more sophisticated responses could also be found. The interviews revealed that the student teachers benefited from the opportunities of peer interaction and self-reflection. The study thus makes suggestions for further improvement of the operation of the VBLC.