Making space for democracy through assessment and feedback in higher education: thoughts from an action research project in education studies

This paper reports on an action research project into the development of a ‘democratic feedback model’ with students on an education studies programme at a post-1992 university in the UK. Building on work that has explored the dialogic dimensions of assessment and feedback, the research explored the potential for more democratic practice in this area. Although much learning and teaching on the programme in question took a collaborative and dialogic approach, assessment and feedback were modelled entirely differently, around the concept of an ‘expert’ marker and ‘novice’ marked. The findings of the research indicate the elements necessary for ‘democratic feedback’, and illustrate the emotional impact of moving from more transmission-based models, grounded in notions of expertise, towards democratic practice. They also highlight the ways in which such work can alert students to the imperfect, messy and human nature of the assessment process. Although the model has limited applicability in its extant form, its constitutive elements might be usefully incorporated within existing practice to promote democratic learning.     

Assessment360: A Promising Assessment Technique for Preservice Teacher Education

The call for preservice teachers to exercise reflective practices has echoed through U.S. education policy and research for a number of years and is evident in U.S. preservice teaching standards. As a result, educator preparation programs are challenged to design learning experiences and assessments that foster reflection skills. This study describes a promising assessment technique, named Assessment360, which can be implemented during coursework to prepare future teachers to be reflective practitioners. Assessment360 is a formative assessment technique in which students reflect on the content of a quiz individually and collaboratively in order to develop a deeper and more integrated understanding of the material. Thirty-one preservice education students enrolled in an undergraduate educational psychology course participated in this research. Results of the questionnaire suggested participation in Assessment360 potentially (a) fostered reflection, (b) encouraged peer interaction and learning (i.e., collaboration), and (c) promoted timely and frequent feedback. Implications and significance are discussed.     

绩效工资改革背景下高职教师评价体系再构建研究

360°考核法是一种科学的考核方法和管理工具。针对绩效工资改革背景下高职教师考核中存在的问题,结合高职院校教师工作的特点,运用360°考核法基本原理,按照确定评价内容、选择评价主体、评价过程管理、评价结果反馈与辅导、评价结果运用与强化等五个关键步骤重新构建了一套高职教师考核评价体系。     

Reconfiguring and realigning the assessment feedback processes for an undergraduate criminology degree

A reconfigured and realigned system of assessment feedback was implemented with undergraduates taking criminology modules at Swansea University. The reformulated system integrated explicit engagement with assessment criteria in feedback given on an electronic template form with the use of a statement bank and the offer of follow‐up, feedback consolidation meetings with the tutor. Student module evaluation feedback questionnaires demonstrated a positive qualitative impact upon students' ratings of their awareness and understanding of module learning outcomes, assessment criteria and required levels of learning; moving the focus of assessment feedback from teaching to (higher‐order) learning. Substantive qualitative impact has also been evidenced by departmental adoption of the reconfigured assessment process as standard. Quantitative impact was evidenced in the form of improvements in assessment performance within‐modules and within students from 2004–2005 to 2005–2006. The educational implications of the study findings are discussed.     

Are they using my feedback? The extent of students’ feedback use has a large impact on subsequent academic performance

Feedback is known to have a large influence on student learning gains, and the emergence of online tools has greatly enhanced the opportunity for delivering timely, expressive, digital feedback and for investigating its learning impacts. However, to date there have been no large quantitative investigations of the feedback provided by large teams of markers, feedback use by large cohorts of students, nor its impact on students’ academic performance across successive assessment tasks. We have developed an innovative online system to collect large-scale data on digital feedback provision and use. Our markers (n = 38) used both audio and typed feedback modalities extensively, providing 388 ± 4 and 1126 ± 37 words per report for first- and second-year students, respectively. Furthermore, 92% of first year and 85% of second-year students accessed their feedback, with 58% accessing their feedback for over an hour. Lastly, the amount of time students spent interacting with feedback is significantly related to the rate of improvement in subsequent assessment tasks. This study challenges assertions that many students do not collect, or use, their feedback. More importantly, we offer novel insights into the relationships between feedback provision, feedback use and successful academic outcomes.     

Peer group learning in roche pharma development

Pharma Development has used action learning to help participants in their 360° feedback programme develop their leadership competencies. The article describes how the programme was designed, supported and run across four sites over a period of 2 years. The programme was systematically evaluated and found to be successful in meeting its objectives. The evaluation summary, strengths of the programme, main issues and recommendations to management are given, as well as conclusions about the programme. Finally, the impact of this experience on the practice of action learning is given.     

Response of students to statement bank feedback: the impact of assessment literacy on performances in summative tasks

Efficiency gains arising from the use of electronic marking tools that allow tutors to select comments from a statement bank are well documented, but how students use this type of feedback remains under explored. Natural science students (N = 161) were emailed feedback reports on a spreadsheet assessment that included an invitation to reply placed at different positions. Outcomes suggest that students either read feedback completely, or not at all. Although mean marks for repliers (M = 75.5%, N = 39) and non-repliers (M = 57.2%, N = 68) were significantly different (p < .01), these two groups possessed equivalent attendance records and similar submission rates and performances in a contemporaneous formatively assessed laboratory report. Notably, average marks for a follow-up summative laboratory report, using the same assessment criteria as the formative task, were 10% higher for students who replied to the original invite. It is concluded that the repliers represent a group of assessment literate students, and that statement bank feedback can foster learning: a simple ‘fire’ analogy for feedback is advanced that advocates high-quality information on progress (fuel) and a curricular atmosphere conducive to learning (oxygen). However, only if students are assessment literate (ignition) will feedback illuminate.     

教学活动中反馈与测评关系的实然分析

现代教学论认为,用系统的观点考察教学活动的各个要素及其联系,对改革教学活动有着特殊的意义。反馈与测评是教学活动中不可缺少的环节,反馈调节着教学活动的进度,测评检查着教学活动的方向。在教学活动中,没有测评的反馈是空洞无效的反馈,没有反馈的测评是行尸走肉的测评。     

Supporting students to develop collaborative learning skills in technology-based environments

The paper focuses on the question of how to advance collaboration through the Web and support lifelong learning. First, the theoretical framework and architecture of a new web‐based tool, the ‘IQ Team’, is introduced. IQ Team is an interactive online assessment and support system to learn social skills needed in cooperative work, and belongs in an interactive online assessing and tutoring system, ‘IQ Form’, developed for the Finnish Virtual University. IQ Team has three main elements: (1) interactive self‐evaluation test banks, (2) online tutoring sets and (3) learning diaries. In the creation of IQ Team, the validation process was conducted with two samples (n = 259 and n = 275). The online students’ social skills in different groups were explored, and the feedback data from different user groups were analysed. The online students scored high values for social skills, and no differences were discerned between university, Open University and technical students. The qualitative data (n = 35) were collected in order to get users’ feedback of the tool. The qualitative data consisted of interviews, open‐ended questions and online discussions. The users of IQ Team reflected that the tool benefited them to become aware of their group work skills and developed their collaborative learning skills. IQ Team provides a powerful tool for online instruction and communication in higher education and in the Open University to promote joint‐regulated learning.     

高校学生学习评价中存在的问题与改革研究

针对目前我国高校学生学习评价体系中存在的一些问题,如评价形式僵化、题型选择不合理、学习评价指标体系与课程培养目标体系脱节、评价反馈机制缺失等。在分析了这些问题产生的原因的基础上,提出了科学化学习评价改革要从转变传统学习评价观念入手,在考试管理"去中心化"基础上,实现与培养目标相契合的、教师自主组织的学习评价多样化形式。     

Literacy,achievement and success in a Māori tourism certificate programme: reading the world in order to read the word

ABSTRACT

This article critiques international assessment of adult literacy using research findings from students completing a Māori tourism certificate who achieved significant gains in assessment. It is argued that the focus of literacy assessments potentially forces educators to narrow their teaching and learning approaches, manoeuvring them into teaching toward singular or convergent literacy. This leads to utilising teaching and learning strategies drawn from the cultural and social capital of the dominant culture, which is problematic for students without abundance of such capital.

Blending Kaupapa Māori research theory with appreciative inquiry, research revealed that students made significant gains in assessment scores because their educators acknowledged and utilised ways in which they made sense and meaning of their world. Educators drew upon the social and cultural capital of students and engaged them as partners in culturally based teaching and learning processes. Relating to students’ multiple literacies, and the contexts with which they bring meaning to their world in this way, is referred to as drawing from divergent literacies.

This study proposes a need to develop and research alternative ways for improving convergent literacy outcomes. It is suggested that multiple interventions using multi-modal and 360° approaches need to occur simultaneously for best outcomes.     

网络课程答疑系统的研究与设计

以往的答疑系统基本上是内建于课程平台之内,不利于系统向专业化方向发展,信息搜索功能也存在一定的缺陷,并且缺乏对答疑效率的关注.为此,将开放体系结构与协作学习模式作为答疑系统建构的基础模型,将电子邮件和MSN消息实时发送运行于信息反馈过程,将全文检索、中文分词技术运用于信息查询过程,设计了一种基于协作学习模式、信息动态反馈机制和智能分词技术的网络答疑系统.     

Psychological determinants of university students’ academic performance: An empirical study

This study utilises an integrated conceptual model of academic performance which captures a series of psychological factors: cognitive style; self-theories such as self-esteem and self-efficacy; achievement goals such as mastery, performance, performance avoidance and work avoidance; study-processing strategies such as deep and surface learning; and effort. We investigate a group of first-year university undergraduates taking a course in business statistics at a British university. The results show a significant causal path of the form: self-esteem→ self-efficacy→ mastery→ effort→ performance. We conclude that the strengthening of any of the elements in this path would have beneficial effects on students’ academic performance and discuss various approaches to pedagogy, primarily assessment and feedback, to achieve this goal.     

Facilitating Learning from Formative Feedback in Level 3 Assessment

This paper presents the development, through action research, of formative elements in assessment in a level 3 compulsory module of the BSc Health Studies and BSc Nursing programmes at the University of Sunderland. The paper reviews three cycles of planning, implementing and evaluating change in assessment strategy and is written in the first person to emphasise the connections between the writer and the material. From a consideration of the format and characteristics of the assessment within the module, the action research is reported through the implementation of actions taken to facilitate more effective use of formative feedback. The evaluation of these actions through my own reflections, student performance, dialogue with team colleagues and student feedback through the production of short narrative accounts of their learning experience is outlined. The paper demonstrates that through explicitly using the learning potential within assessment, learning can be facilitated through challenging students to move from 'doing' assignments, to reflexive thinking about their writing.     

Students’ Interpretation of Formative Assessment Feedback: Three Claims for Why We Know So Little About Something So Important

If K‐12 students are to be fully integrated as active participants in their own learning, understanding how they interpret formative assessment feedback is needed. The objective of this article is to advance three claims about why teachers and assessment scholars/specialists may have little understanding of students’ interpretation of formative assessment feedback. The three claims are as follows. First, there is little systematic research of K‐12 students’ interpretations of feedback. Systematic research requires gathering substantive evidence of students’ cognitive and emotional processes using psychological methods and tools. Second, there is an overemphasis on the external assessment process at the expense of uncovering learners’ internal reasoning and emotional processes. This overemphasis may be due to vestiges of behavioral approaches and lack of training in social cognitive methods. Third, there are psychological tools such as the clinical interview, pioneered by Piaget and used by psychologists to “enter the child's mind,” which may be helpful in uncovering students’ interpretation of feedback and associated behavioral responses. If the purpose of formative assessment is to change student learning, and feedback is delivered as a conduit to help with this long‐term change, understanding students’ interpretation of feedback plays a central role in the validity of the process.     

Implementing summative assessment with a formative flavour: a case study in a large class

Teaching a large class can present real challenges in design, management and standardisation of assessment practices. One of the main dilemmas for university teachers is how to implement effective formative assessment practices with accompanying high-quality feedback consistently over time with large classroom groups. This article reports on how elements of formative practices can be implemented as part of summative assessment in very large undergraduate cohorts (n = 1500 in one semester), studying in different modes (on- and off-campus), with multiple markers, and under common cost and time constraints. Design features implemented include the use of exemplars, rubrics and audio feedback. The article draws on the reflections of the leading teacher, and argues that, for summative assessment to benefit learners, it should contain formative assessment elements. The teaching practices utilised in the case study provide some means to resolve the tensions between formative assessment and summative assessment that may be more generally applicable.     

Comparing the effectiveness of video and stereoscopic 360° virtual reality-supported instruction in high school biology courses

The promise of using immersive technologies in learning has increasingly been attracting researchers' and practitioners' attention. However, relevant empirical works are usually conducted in fully controlled Virtual Reality (VR) laboratories, as opposed to conventional settings. This quasi-experimental study compares the effectiveness of video learning resources to that of stereoscopic 360° VR, as supplements to the traditional instructional approach. The potential of such methods was examined in high school settings, in the context of the ‘Life and Evolution’ module, with participants (n = 70) divided equally into control and experimental groups. As a point of reference (control condition), we considered the adoption of Video Learning Resources, as students are more acquainted with this instructional method. In the intervention approach (experimental condition), students adopted the use of low-end mobile-VR (VeeR Mini VR Goggles). The key findings indicate differences in the learning motivation, confidence and satisfaction, but no statistically significant difference was identified regarding the factual or conceptual knowledge gains. The study offers insights on the potential of the investigated technologies in the subject of secondary school Biology and further provides implications for theory and practice.

Practitioner notes

What is already known about this topic

• Researchers' interest over the potential of Virtual Reality on different STEM disciplines is increasing consistently.
• An increasing number of efforts can be identified discussing the integration of multimedia learning resources in the secondary school context.
• Empirical studies on the subject of Biology are focusing on students' academic performance and achievement but not on learning motivation and satisfaction.

What this paper adds

• This quasi-experimental study comparatively examines academic performance, with the focus being on learning motivation and satisfaction, across different modalities (stereoscopic 360° Virtual Reality applications-VR, Video Learning Recourses-VLR).
• The findings demonstrate that both instructional methods are sufficient in enhancing students' knowledge acquisition and academic performance.
• The adoption of stereoscopic 360° VR influences students' learning motivation and impacts long-term memory retention.

Implications for practice and policy

• Educators are advised to consider the systematic adoption of “immersive” multimedia tools to enhance the subject of Biology as they can greatly encourage scientific inquiry.
• Instructional designers are advised to adopt open educational resources aligned to the curriculum of the local context.
• Educational researchers are advised to integrate stereoscopic 360°-VR solutions in the conventional classroom settings.

     

Assessment as learning: blurring the boundaries of assessment and learning for theory,policy and practice

This paper explores assessment and learning in a way that blurs their boundaries. The notion of assessment as learning (AaL) is offered as an aspect of formative assessment (assessment for learning). It considers how pupils self-regulate their own learning, and in so doing make complex decisions about how they use feedback and engage with the learning priorities of the classroom. Discussion is framed from a sociocultural stance, yet challenges some of the perspectives that have widely become accepted. It offers three new views to help explore the concept of AaL: understanding feedback; understanding the learning gap; and exploring vocabularies of assessment. Pragmatically, the ideas examined suggest that teachers may need to consider less about focused and directive feedback, but more about how learners interpret and understand feedback from their self-regulatory and self-productive identities and how vocabularies for assessment can be more collaboratively shared in learning contexts.     

A pedagogical alliance for trust,wellbeing and the identification of errors for learning and formative assessment

Formative assessments and feedback are vital to enhancing learning outcomes but require that learners feel at ease identifying their errors, and receiving feedback from a trusted source – teachers. An experimental test of a new theoretical framework was conducted to cultivate a pedagogical alliance to enhance students’ (a) trust in the teacher, (b) well-being in the learning environment and (c) identification of confusion and errors for the purpose of learning, assessment and feedback. A sample of 101 undergraduate students was randomly assigned to either an intervention (n = 51) or control (n = 50) condition in Elementary Statistics. Results indicated that a pedagogical alliance could be created to enhance student trust in the instructor, leading students to report greater well-being and a higher number of potential areas of confusion in their understanding of new content material relative to a control group. These results have implications for formative feedback, assessments, and by extension learning outcomes.     

The relationship between students’ perceptions of portfolio assessment practice and their approaches to learning

This study focuses on students’ learning approaches in the context of a competency‐based program on Applied Sciences, with portfolio assessment as its core mode of assessment. The study examines students’ perceptions of these assessment practices and the relationships to their learning approaches. Additionally, differences in perceptions and learning approaches between first‐year students and second‐year students, who already have one year of experience with the portfolio assessment practice, are investigated. A total of 110 students completed two questionnaires at the end of the academic year: the Revised SPQ (Study process questionnaire) was used to measure their approaches to learning and the AEQ (Assessment experience questionnaire) was used to measure their perceptions of the assessment practice. The results indicate that, when implementing portfolio assessment, feedback is essential. Moreover, it seems not to be the assessment design, but rather how it is implemented, and how students’ learning is stimulated through the assessment, that influences the quality of learning.