如何提高网络环境下形成性评价的反馈质量——北外网院的实践

形成性评价在远程教学中发挥了重要的作用。科学的形成性评价不仅可以帮助学生理清课程学习重点,还可有效地督促学生学习。反馈是形成性评价的重要环节,及时有效的反馈对于远程学习者尤为重要。本文介绍并评估在北京外国语大学网络教育学院(以下简称"北外网院")开展的一项旨在提高形成性评价反馈质量的教学实验。结果表明,实验教师认识到反馈的重要性并能够提供有效反馈。研究提出的基于网络的教师反馈监控机制对于保证和提高远程教学质量的探索将有重要的启示作用。     

Formative assessment in mathematics for engineering students

In this paper, we present a range of formative assessment types for engineering mathematics, including in-class exercises, homework, mock examination questions, table quizzes, presentations, critical analyses of statistical papers, peer-to-peer teaching, online assessments and electronic voting systems. We provide practical tips for the implementation of such assessments, with a particular focus on time or resource constraints and large class sizes, as well as effective methods of feedback. In addition, we consider the benefits of such formative assessments for students and staff.     

Designing and implementing peer formative feedback within online learning environments

With increasing need to achieve appropriate balance between learning support and self-regulation within the context of online learning, formative feedback has been identified as a viable means to achieve meaningful engagement. Specifically, this study sought to establish how peer–peer formative feedback was facilitated in an online course and to what extent this engaged students in meaningful learning experiences. This case study entailed an in-depth investigation into the design and implementation of an online course in a New Zealand university. The studied course was part of a postgraduate programme in continuing (in-service) teacher education. The study adopted a case study methodology with a bias on qualitative techniques. Online observations, analysis of the archived course discourse and interviews were utilised as sources of data. The data from multiple sources were subsequently triangulated to corroborate the evidence. The findings indicate that peer formative feedback promoted active learners’ participation and meaningful engagement. The findings further showed that opportunities for dialogic peer formative feedback promoted learning support and self-regulation.     

现代远程教育中形成性评价的现状及对策研究

课程评价是网络教育中学生评价的重要内容,也是网络教育教学质量的保证。目前我国现代远程教育的课程评价仍然侧重于终结性评价,对形成性评价的重视程度不足。在对16所试点高校网络教育学院的调研中,从各网院的形成性评价和终结性评价的比例、内容、形式的分析中证实,这些试点高校开始重视形成性评价。实践表明,形成性评价对学生学习起着重要作用,是保证学习质量的一个关键环节。针对调研中发现的问题,加强形成性评价的策略是：对形成性评价正确定位、从教学设计和技术角度加强学习参与度评价、建立健全形成性评价的反馈机制,不断推进和创新形成性评价工作。     

Using summative and formative assessments to evaluate EFL teachers’ teaching performance

Using classroom observations (formative) and student course experience survey results (summative) to evaluate English lecturers’ teaching performances is not new in practice, but surprisingly only a few studies have investigated this issue in a higher education context. This study was conducted in an English department of a large university in Vietnam. The data include: (1) semi-structured interviews with all the full-time lecturers, (2) two department heads and (3) course experience surveys from English as a foreign language (EFL) students (N?=?2886). Three lessons can be learned: (1) formative assessments do not seem to have an effect on promoting better teaching practices when their feedback is not helpful in improving high-stakes summative assessment results, (2) without sharing a common definition of good teaching practices among assessors, summative assessments appear to make the feedback from formative assessments less meaningful and applicable, and (3) as a result, the combination of formative and summative assessments tends to make EFL lecturers’ self-assessment practices less effective.     

Classroom action research on formative assessment in a context-based chemistry course

Context-based science courses stimulate students to reconstruct the information presented by connecting to their prior knowledge and experiences. However, students need support. Formative assessments inform both teacher and students about students’ knowledge deficiencies and misconceptions and how students can be supported. Research on formative assessments suggests a positive impact on students’ science achievement, although its success depends on how the formative assessment is implemented in class. The aim of this study was to provide insights into the effects of formative assessments on achievement during a context-based chemistry course on lactic acid. In a classroom action research setting, a pre-test/post-test control group design with switching replications was applied. Student achievement was measured in two pre-tests, two post-tests and a retention test. Participants were Grade 9 students from one secondary school in the Netherlands. Repeated-measures analysis showed a significant effect of formative assessments on students’ achievement. During the implementation of the formative assessments, intriguing discussions emerged between students, between students and teacher, and between teachers. Adding formative assessments to context-based approaches reinforces their strength to meet with the current challenges of chemistry education. Formative assessments affect students’ achievement positively and stimulate feedback between students and teacher(s).     

Formative assessment systems: evaluating the fit between school districts’ needs and assessment systems’ characteristics

Recent legislative and local school accountability efforts have placed a premium on the collection, analysis, and use of student assessment data for educational improvement. As a result, schools have sought assessments that will provide additional information about student performance. In response, a burgeoning boon industry formed—formative educational assessment systems. In this study we describe how districts search for and acquire formative assessment systems to meet their needs. We focus on three school districts that adopted three different formative assessment systems. Our findings suggest the fit between a system’s characteristics and a school district’s intended use is the most important consideration in instituting a successful formative assessment system that will have a positive impact on teacher education and student learning.     

Personal Backgrounds of Effective Teaching

Planning time for giving students effective feedback is an important and challenging aspect of the teaching and learning process. In our article we describe and analyze how we engage students as partners in providing formative feedback in time for students to modify their own thinking or behavior to improve learning. We have found ways to provide formative feedback more frequently and to involve students in providing effective formative feedback to each other. The four techniques we describe are the following: a) three-color group quiz with feedback on product, process, and progress; b) midterm student conferencing; c) shared revision of student generated questions and statements; and d) timely feedback using collaborative assignment blogs. These techniques give feedback in time for revisions to occur, provide scaffolding for learners, inform instruction, and most importantly, involve students as partners in assessment. These pedagogical strategies show that the resulting benefits of improved instruction, enhanced student learning, and better student products are worth the time and effort and contribute to a productive classroom climate where the focus is on learning more than on grading. Formative feedback involving students as partners is a key strategy to enhance the teaching and learning process.     

The effects of POWERSOURCE© assessments on middle-school students’ math performance

This paper describes results from field testing of middle-school math formative assessments alongside professional development and instructional resources. We employed a randomised, controlled design to address the question: Does using our formative assessment strategies improve student performance on assessments of key mathematical ideas relative to a comparison group? This study also provided data on the instructional sensitivity of the assessments, which is part of the validation needed for formative assessments. Teachers were recruited from two districts and seven middle schools. Nineteen treatment and 17 comparison group teachers and their students were included in study analyses. Scores on extended response and short-answer questions indicated that students in the treatment group performed better than students in the comparison group who received the formative assessments alone. These findings demonstrate both the feasibility and value of including performance task-types in a brief assessment context.     

网上形成性考核改革促使开放教育实践水平的提升——以中央广播电视大学“综合英语”系列课程为例

中央广播电视大学是世界上最大的远程教育机构,开设有849门课程,大部分课程都有在线学习的成分,因此,利用网上形成性考核的方式进行开放教育学习支持服务具有重要作用。中央广播电视大学开放教育英语专业专科层次综合英语系列课程先后经过‘综合英语’网络课程建设项目(2006-2008年)与‘综合英语’课程教学团队建设项目网上形成性考核子项目(2009年至今)两次大规模的开放教育资源革新。本研究通过运用开放教育实践矩阵以及问卷调查分析方法判定两个项目实施前后综合英语课程开放教育资源和开放教育实践的开放程度。通过分析该课程的社会参与程度、协作使用和开放的程度以及开放教育实践自主自由度和社会协作程度,本研究得出以下结论:中央广播电视大学综合英语课程开放教育实践经历了一个从初级到中级再到高级的发展过程;从使用者处获得的信息反馈对于丰富和提升网上形成性考核学习资源的质量具有持续性价值;需要持续关注、研究和改进开放教育实践,以满足学习者不断新增的学习需要。希望本研究对其他教学机构和个人在促进开放教育实践的开放度方面有一定借鉴意义。     

Rethinking exams and letter grades: how much can teachers delegate to students?

In this article we report a 3-yr study of a large-enrollment Cell Biology course focused on developing student skill in scientific reasoning and data interpretation. Specifically, the study tested the hypothesis that converting the role of exams from summative grading devices to formative tools would increase student success in acquiring those skills. Traditional midterm examinations were replaced by weekly assessments administered under test-like conditions and followed immediately by extensive self, peer, and instructor feedback. Course grades were criterion based and derived using data from the final exam. To alleviate anxiety associated with a single grading instrument, students were given the option of informing the grading process with evidence from weekly assessments. A comparative analysis was conducted to determine the impact of these design changes on both performance and measures of student affect. Results at the end of each year were used to inform modifications to the course in subsequent years. Significant improvements in student performance and attitudes were observed as refinements were implemented. The findings from this study emphasized the importance of prolonging student opportunity and motivation to improve by delaying grade decisions, providing frequent and immediate performance feedback, and designing that feedback to be maximally formative and minimally punitive.     

Implementing curriculum-embedded formative assessment in primary school science classrooms

The implementation of formative assessment strategies is challenging for teachers. We evaluated teachers’ implementation fidelity of a curriculum-embedded formative assessment programme for primary school science education, investigating both material-supported, direct application and subsequent transfer. Furthermore, the relationship between implementation fidelity and teacher variables was explored. N = 17 German primary school teachers participated in professional development on formative assessment, N = 11 teachers formed a control group. Teachers’ implementation fidelity was evaluated via classroom observations student ratings and an analysis of students’ workbooks, focusing on the frequency and quality of intended formative assessment elements (assessments, feedback and instructional adaptations). Regarding direct application, treatment group teachers’ implementation fidelity was high, with slight variations in quality. Regarding transfer, implementation fidelity was lower but teachers still implemented more formative assessment elements than the control group. Teachers’ pedagogical content knowledge and their evaluation of the formative assessment intervention were associated with implementation success.     

Capacity building to bridge data use and instructional improvement through evidence on student thinking

While schools and systems across the globe promote data-driven decision making, teachers often struggle to use data, especially from external assessments, to inform daily instruction. In this paper, we examine teacher capacity building for a less typical form of data use - evidence on student thinking. We draw on data from a longitudinal, in-depth qualitative study involving middle school math teachers who were engaged in an instructional improvement project. Findings show that data use occurred when evidence from student thinking was introduced as part of the instructional planning process. This shift was facilitated by an instructional coach whose capacity building efforts with teachers focused on coherence, specifically planning high quality instruction and using data effectively, while also meeting district pacing and unit planning goals. When teachers put new strategies into practice, feedback from formative assessment data allowed teachers to identify and address misconceptions in student thinking. Learning how to use data as part of instruction helped teachers build capacity to reflect on their own practice. Implications for theory, practice, and research are discussed.     

Supporting change and scholarship through review of online resources in professional development settings

How can we accelerate innovation and ensure effective dissemination of knowledge about online learning resources? This paper advocates strategies that systematically link online professional development with the research, development and diffusion cycle. The systemic approach we describe can accelerate knowledge advancement and help manage change by improving communication among teachers, trainers, developers and researchers. The examples that are provided are set within two funded projects in the United States that led to the development of two distinct but related strategies—the Online Site Evaluation Form for educators (a web‐based review form used in workshops) and a six‐week online course on technology‐supported assessments. Both strategies make it easier to give feedback to developers and offer incentives to do so in ways that help teachers to learn about online resources individually and with colleagues. The examples are discussed with analysis of their strengths and weaknesses in supporting different modes of interaction. We highlight implications for instructional development, professional development, research and knowledge management in online communities.     

Utilisation of formative peer-assessment in distance online education: a case study of a multi-model sport management unit

Higher education system has been globally evolving over time with the development of courses that offer distance online models of delivery to meet the changing needs of students in an era of technology-driven transformation. However, one of the biggest challenges of distance online education has been higher attrition rates mostly due to difficulties in engaging the students in the learning process adequately. In this regard, peer-assessment has been recommended in the literature as an interactive method to optimise student engagement and learning in collaborative environments. This study demonstrates how formative peer-assessment was utilised in an online multi-model sport management unit to enhance engaged learning outcomes.     

RECONSTRUCTING CHILDREN'S EXPERIENCE WHEN TEACHING AND ASSESSING THEM: LESSONS FROM DEWEY

This paper describes an online early childhood assessment course that was developed through a multi-university collaboration with support from a state improvement grant. Collaborators from three universities developed the course to address a new early childhood unified license (birth to age 8, regular and special education) in the state of Kansas. After reviewing the new state content standards, we identified targeted understandings, performance assessments, and online activities for 15 modules using a backward design process. Emphasis was placed on active learning through synchronous and asynchronous interactions facilitated by the use of a course management system. Positive evidence of learning was indicated by anonymous student feedback, pretest/posttest gain scores, and performance assessments evaluated with rubrics. We viewed the implementation of the course as a success and anticipate that it may lead to more sharing of online coursework in Kansas teacher education programs in the future.     

Moving Toward a Comprehensive Assessment System: A Framework for Considering Interim Assessments

Local assessment systems are being marketed as formative, benchmark, predictive, and a host of other terms. Many so-called formative assessments are not at all similar to the types of assessments and strategies studied by   Black and Wiliam (1998)   but instead are interim assessments. In this article, we clarify the definition and uses of interim assessments and argue that they can be an important piece of a comprehensive assessment system that includes formative, interim, and summative assessments. Interim assessments are given on a larger scale than formative assessments, have less flexibility, and are aggregated to the school or district level to help inform policy. Interim assessments are driven by their purpose, which fall into the categories of instructional, evaluative, or predictive. Our intent is to provide a specific definition for these "interim assessments" and to develop a framework that district and state leaders can use to evaluate these systems for purchase or development. The discussion lays out some concerns with the current state of these assessments as well as hopes for future directions and suggestions for further research.     

Student perceptions of quality feedback in teacher education

Many reports have identified a perceived lack of quality in regard to assessment feedback in higher education contexts. One research study in 2007 on undergraduate university students found that less than half of the students (46%) collected their formative feedback, suggesting that from their perspective feedback clearly was not fulfilling the role it should. This is a study of 465 graduate students and 101 undergraduate students studying teacher education at a major Australian university. The study investigated what students perceived to be effective, quality feedback based upon their extensive higher education experiences. Students identified preferences in regard to form, detail and timing of assessment feedback. The data were collected by means of pen and paper survey and identified which strategies the students perceived to be the most effective, particularly within the context of large cohort teaching and written assessment formats. Findings agreed with research elsewhere regarding problems with assessment feedback quality and quantity, but students also provided clear indications of how realistic improvements could be made in terms of assessment feedback processes and strategies.     

Cognitive Style FD/FI as a Learner Selection Criterion in Formative Evaluations A Qualitative Analysis

Single–subject formative evaluation appears to be a cost–effective strategy for improving instructional products. However, the criterion to use for selecting an appropriate test subject who could generate optimal feedback data for improving the instructional product remains a central concern among performance technologists. This article reports the results of a qualitative study of the effectiveness of the cognitive style construct field–dependent/independent as a student selection criterion in formative evaluation. In the study, we collected formative evaluation data from two field–dependent (FD) and two field–independent (FI) test subjects while they were individually interacting with a CAI package. We focused on four different sources of data: think–aloud protocols, researcher/subject interactions, informal observations, and debriefing interviews. Our analysis of the formative evaluation data indicates that the FI individuals were better test subjects than their FD counterparts. FI subjects showed a great deal of confidence in entering the formative evaluation process. Their feedback was abundant and precise and included specific suggestions for improving the material. They not only identified their own difficulties but also speculated about difficulties other students may encounter. In contrast, the FD subjects were anxious and demonstrated less confidence in approaching the evaluation activities. Frequent probing was necessary to trigger their reactions and generate their feedback. Their feedback data were vague, and more inferences were required for translating them into revision decisions. Both FD and FI subjects could identify major discrepancies in the presentation of the material (events of learning) as well as gross misconceptions in the processing of information. Although the FD and FI feedback data differ both qualitatively and quantitatively, no conflicting observation was made.     

Evaluate to improve: useful approaches to student evaluation

Many teachers in higher education use feedback from students to evaluate their teaching, but only some use these evaluations to improve their teaching. One important factor that makes the difference is the teacher’s approach to their evaluations. In this article, we identify some useful approaches for improving teaching. We conducted focus groups with award-winning university teachers who use student evaluations to improve their teaching, and we identified how they approach their evaluation data. We found that these teachers take a reflective approach, aiming for constant improvement, and see their evaluation data as formative feedback, useful for improving learning outcomes for their students. We summarise this as the improvement approach, and we offer it for other teachers to emulate. We argue that if teachers take this reflective, formative, student-centred approach, they can also use student evaluations to improve their teaching, and this approach should be fostered by institutions to encourage more teachers to use student evaluations to improve their teaching.