Assessment literacy and student learning: the case for explicitly developing students ‘assessment literacy’

In this paper, we report on a study to quantify the impact on student learning and on student assessment literacy of a brief assessment literacy intervention. We first define ‘assessment literacy’ then report on the development and validation of an assessment literacy measurement instrument. Using a pseudo-experimental design, we quantified the impact of an assessment literacy-building intervention on students’ assessment literacy levels and on their subsequent performance on an assessment task. The intervention involved students in the experimental condition analysing, discussing and applying an assessment rubric to actual examples of student work that exemplified extremes of standards of performance on the task (e.g. poor, excellent). Results showed that such a procedure could be expected to impact positively on assessment literacy levels and on student performance (on a similar or related task). Regression analyses indicated that the greatest predictor of enhanced student marks (on the assessment task that was the subject of the experiment), was the development of their ability to judge standards of performance on student work created in response to a similar task. The intervention took just 50 minutes indicating a good educational return on the pedagogical investment.     

真实性评价在美国社会科课程中的应用及启示

真实性评价是美国评价社会科课程的重要评价方法和评价理念,其出于美国教育改革的需要,并在实践中得到丰富和发展。真实性评价强调评价任务的真实性,重视考察学生综合运用知识技能解决复杂社会问题的表现,主张采用多种方法收集有关学生课程学习的信息。真实性评价能为我国课程评价改革提供有益的启示。我国课程评价改革应制定真实性评价任务、明确具体的标准、采用多元的评价方法及构建多主体参与的评价体系。     

Preservice Teachers' Development and Implementation of Science Performance Assessment Tasks

Preservice teachers in a science methods course were provided instruction on performance assessment, then guided through a design and implementation process of performance assessment tasks. We assessed the effect of designing and implementing a performance assessment task on preservice teachers' understanding of standards-based assessment. The findings show that these preservice teachers improved in their understanding of assessment as a formative process as well as their science content understanding of the topic addressed in their designed task. We found that preservice teachers need to experiment with performance assessment tasks in an authentic context in order to understand the full potential and value of the task.     

Assessment training effects on student assessment skills and task performance in a technology-facilitated peer assessment

This study examines the impact of an assessment training module on student assessment skills and task performance in a technology-facilitated peer assessment. Seventy-eight undergraduate students participated in the study. The participants completed an assessment training exercise, prior to engaging in peer-assessment activities. During the training, students reviewed learning concepts, discussed marking criteria, graded example projects and compared their evaluations with the instructor’s evaluation. Data were collected in the form of initial and final versions of students’ projects, students’ scoring of example projects before and after the assessment training, and written feedback that students provided on peer projects. Results of data analysis indicate that the assessment training led to a significant decrease in the discrepancy between student ratings and instructor rating of example projects. In addition, the degree of student vs. instructor discrepancy was highly predictive of the quality of feedback that students provided to their peers and the effectiveness of revisions that they made to their own projects upon receiving peer feedback. Smaller discrepancies in ratings were associated with provision of higher quality peer feedback during peer assessment, as well as better revision of initial projects after peer assessment.     

Assessing students’ performance by measured patterns of perceived strengths: does preference make a difference?

This paper is an empirical field study of whether college students’ preferences for assessment type correspond to their performance in assessment that tests that particular strength. For example, if students say they prefer assessment that tests their creativity, do they actually perform better on assessment tasks requiring the use of creativity? Seventy-eight students in three different courses were surveyed to determine their preferences in four types of assessment: memorisation, analysis, creativity and practical application. These preferences were then compared to student grades on corresponding forms of assessment to see if the preferences corresponded to actual performance. The study found that, while students had a clear preference for memorisation, they were not likely to deliver their best performances on memorisation tasks. There was no relationship at all between student preferences in assessment type and their performance in the respective assessments. These results indicate that, while in theory assessing students based on their preferences is reasonable for improved learning, we were not able to find evidence that it actually leads to higher performance.     

Promoting assessment efficacy through an integrated system for online clinical assessment of practical skills

This paper presents evaluation outcomes from an externally funded research project involving the online clinical assessment of practical skills (eCAPS) using web-based video technologies within a university medical programme. eCAPS was implemented to trial this web-based approach for promoting the efficacy of practical skills assessment in knee joint examinations for a pre-clinical cohort of second year medical students. eCAPS involves a progressive organisation of online video experiences and task expectations for formative and summative assessments of selected competencies. Data are presented from semi-structured interviews with the medical students (N?=?40). eCAPS successfully supported students’ skill development in knee joint examinations on fellow students and allowed remote assessment of candidates’ performances by clinicians experienced in musculoskeletal examinations. An ‘indicative standard’ paradigm, involving formative assessment of one randomly selected performance from the submission of all students’ responses within a small group, offered an efficient and efficacious avenue for providing consolidated feedback to students and promoted desirable learning behaviours. Overall, there was evidence of reciprocal learning benefits to the ‘blending’ of an online learning and assessment approach with an existing face-to-face environment. The medical students were able to successfully engage online with high quality and consistent practical skill-based materials in a flexible, independent and individual manner.     

First-year students’ appraisal of assessment tasks: implications for efficacy,engagement and performance

This study investigated students’ appraisals of assessment tasks and the impact of this on both their task-related efficacy and engagement and subsequent task performance. Two hundred and fifty-seven first-year students rated their experience of an assessment task (essay, oral presentation, laboratory report or exam) that they had previously completed. First-year students evaluated these assessment tasks in terms of two general factors: the motivational value of the task and its manageability. Students’ evaluations were consistent across a range of characteristics and level of academic achievement. Students’ evaluations of motivational value generally predict their engagement and their evaluations of task manageability generally predict their sense of task efficacy. Engagement was a significant predictor of task performance (viz. actual mark) for exam and laboratory report tasks but not for essay-based tasks. Findings are discussed in terms of the implications for assessment design and management.     

基于“互联网+”的教育评估改革路径与方法探讨——以江苏省为例

教育评估是促进教育教学发展的重要手段,从内容上来看,有效的教育评估能够为教学工作提供有力的指导,推动教育改革与发展。随着智能技术的发展与普及,传统评估方式的弊端日益显露,教育评估改革势在必行。本文通过实践调研与文献分析,以江苏省为例,通过分析当前江苏教育评估操作的现状与问题,从评估环节着眼,提出了“互联网+”时代教育评估智能发展的趋势;从评估要素入手,给出了教育评估改革的路径,并参照教育信息化2.0计划,以信息生命周期理念为基础,从信息更替的各个环节逐层剖析,得到智能化信息时代教育评估改革的对策。“互联网+技术”的快速发展为教育评估提供了新的思路与方法,新时代背景下的教育评估正朝着信息化、数字化和智能化的方向发展。合理利用技术发展带来的便捷,正确对待技术变革带来的冲击,才能高效发挥教育评估的作用,为深化新时代背景下的教育改革奠定坚实基础。     

Feedforward strategies in the first-year experience of online and distributed learning environments

Feedforward is an educational strategy focusing on providing students with prior exposure to, and prior practise with assessment in order to clarify expectations and standards. Current research into feedforward has yet to fully consider the specificity of online and distributed learning environments. We have developed a feedforward-based approach to teaching a large first year prerequisite course in screen history involving the modified and critical use of exemplars and assessment guidance in ways designed to stimulate self and peer assessment throughout the course. Our focus is the improvement of task compliance, quality and criteria in order to stimulate meaningful engagement with assessment and enhanced student performance in the unit.     

Developing assessment literacy through approximations of practice: Exploring secondary mathematics pre-service teachers developing criteria for a rich quadratics task

This study draws on a theoretical model for teacher assessment literacy to investigate the process of how secondary mathematics pre-service teachers (PSTs) might develop theirs through experiencing an approximation of practice. Sixty PSTs participated in designing and then refining assessment criteria for a rich open-ended quadratics task using real student example responses. Data analysis examined variations in the PSTs’ attention to quadratic features and mathematical language as assessment criteria for the task throughout the activity. This paper discusses evidence suggesting that the collaborative analysis of example student responses can make certain, but not all, assessment criteria salient to PSTs.     

Student performance on argumentation task in the Swedish National Assessment in science

The aim of this study is to investigate the influence of content knowledge on students’ socio-scientific argumentation in the Swedish National Assessment in biology, chemistry and physics for 12-year-olds. In Sweden, the assessment of socio-scientific argumentation has been a major part of the National Assessment during three consecutive years and this study utilizes data on student performance to investigate (a) the relationship between tasks primarily addressing argumentation and tasks addressing primarily content knowledge as well as (b) students’ performance on argumentation tasks, which differ in relation to content, subject, aspect of argumentation and assessment criteria. Findings suggest a strong and positive relationship between content knowledge and students’ performance on argumentation tasks. The analysis also provides some hypotheses about the task difficulty of argumentation tasks that may be pursued in future investigations.     

Cultivating student skills in self-regulated learning through evaluation of task complexity

In order to self-regulate, students need to honestly reflect on their learning and to take appropriate corrective action. A simple procedure to cultivate student skills in self-regulated learning, known as the Task Evaluation and Reflection Instrument for Student Self-Assessment (TERISSA) is discussed in this paper. TERISSA guides students through two evaluations of the complexity of a task: the first is undertaken just before solving the task and the second straight after completing the task. This study involved 63 undergraduate students and observed a statistically significant difference (p = 0.007) in performance between the students who did (6.1/10) and did not (4.1/10) use TERISSA during tutorials leading up to an assessment task.     

表现性评价在电大开放教育课程考核中的应用研究

阐述了表现性评价的理论内涵,分析了表现评价在开放教育课程考核中的适用性,以开放教育《中国法律思想史》为例,具体分析了表现性评价在开放教育课程考核模式改革中的实践应用,并对其中存在的一些问题进行了思考和探讨。     

Students’ perspectives of early assessment tasks in their first-year at university

Assessment plays an important role in first-year students’ transition to university, influencing how they learn. Poorly designed assessment can be demotivating and may even cause students to withdraw from university. An early low-stakes assessment task is recommended in first year, to provide information to students and staff on student progress, provide students with a positive experience and help them prepare for future assessments. This paper reports on first-year students’ experiences of two different early assessment tasks for two disciplines, one using an online quiz and the other a ‘research challenge’ that served as a precursor to writing an essay. The paper also presents lecturers’ reflections on what they learned, what they would change and the implications for others planning early assessment tasks for first years. Finally, ideas for future research are provided.     

Test-task Authenticity: The Multiple Perspectives

Leung and Lewkowicz remind us that the debate over the past two decades that is most relevant to ELT (English languge teaching) pedagogy and curriculum concerns test-task authenticity. This paper first reviews how the authenticity debate in the literature of second language acquisition, pedagogy and testing has evolved. Drawing on a body of research work on learner task-based language performance in recent years, the paper then discusses the four dimensions of test task that should help us locate and determine the relative degree of authenticity of a test task. It is further argued that these authenticity dimensions interact with each other, and may be variably manifested in a way that apparently involves a complex and dynamic process under different assessment conditions.     

国外教师数据素养测评研究及启示

随着大数据和智能技术在教育中的广泛应用,教师数据素养测评研究已进入实践层面。国内教师数据素养领域的研究现状表明,借鉴国外研究成果对促进我国该领域的发展非常必要。在国外文献调研的基础上,本研究首先将教师数据素养的测评工具归纳为问卷调查、知识测试和情境访谈等三种类型,继而详细分析有代表性的测评工具的内容及形式特点,并结合案例描述各种测评工具的应用场景和测评效果。结合证据中心设计模式的相关要素,本研究从开发过程、测评效度和应用程度等角度总结和比较各种测评工具,最后分析了我国该领域在评价体系建构、测评任务设计和测评实施等方面得到的启示,以期为大数据时代教师专业发展提供借鉴。     

Interventions in teaching first-year law: feeding forward to improve learning outcomes

Aligned assessment is a cornerstone of higher education curriculum design. Yet, it does not address the problem of how students learn how they should proceed when faced with a new assessment task. That teaching task is often left to the role of ‘feedback’. This article examines changes to a first year law unit, introduced following negative student feedback about an assessment task. The feedback made apparent a lack of alignment between staff and student expectations as to what was being assessed and how it was being assessed. The article considers formative assessment through a feed-forward model, and relates this to the design and process of the intervention. The teaching method of the feed-forward model is detailed. The analysis of the results shows that a number of key misperceptions were held by students as to the points of difficulty in the assignment. A significant number of students also found it extremely difficult to judge the quality of past assignments against a marking guide. This direct feeding forward allowed some of the students’ tacit and false assumptions to emerge and be addressed, before they began their first major assessment, resulting in improved pass rates for the assessment.     

中小学生学业评价标准的实验研究

中小学生学科学业评价标准主由表现标准评价方法建议评价样例和学生作品及其评分四个部分构成。各学科评价标准在学校进行了试用,以检验其科学性和可行性,并为修改评价标准收集实验证据。本文报告这一阶段的研究设计、过程和主结果,并讨论本研究所引发的几点思考。