Estimating the quality of performance assessments: The case of an ‘interactive examination’ for teacher competencies

Professional schools prepare students to become competent professionals. Consequently, there is a need for assessments that can determine the acquisition of the relevant professional competencies. Although using performance assessment to replace traditional paper-and-pencil tests might provide one way to move forward, the use of performance assessments for summative purposes has been shown to be problematic (e.g. marker consistency and construct representation). With the aid of a comprehensive framework of quality criteria for competence assessments, this article considers if one particular existing competence assessment methodology is suitable for summative as well as formative use. It is argued that the comprehensive quality estimation of the examination procedure aids in identifying strengths and weaknesses in the assessment methodology, and that this information can be used to facilitate the inclusion of performance assessment in higher education, both for summative and formative use.     

Back to school in later life: Older Chinese adults’ perspectives on learning participation barriers

ABSTRACT

Background and aim: Even though the beneficial effects of elderly learning are widely acknowledged, many older Chinese people are still not involved. This paper aims to examine the barriers that affect the level of educational participation of older adults in China. Methodology: Using a focus group methodology, 43 older participants (aged 55 years and over) were assigned to five focus groups based on gender and (in)activity rate in Xi’an, China. The focus groups were conducted to identify the individual learning experiences including motivation, learning preferences, and, especially, participation barriers between older adults who have already participated in the Universities of the Third Age (U3As) and others who have not. Strategies to overcome or minimize these barriers were also raised in focus group interviews. Results: The results have demonstrated that barriers associated with situational factors were reported the most; informational barriers and dispositional barriers were perceived as the least obstructive. The active group was characterized by a more optimistic mood in dealing with these learning barriers. The study also suggests that the Selective Optimisation with Compensation Model of successful ageing can be useful in explaining older adults’ strategy for coping with the barriers. Conclusion: The findings of the study provide policy makers and U3A practitioners an insight into the barriers to educational participation. These findings provide input to develop targeted intervention strategies and tailor-made measures to reduce these constraints and increase participation.     

Autonomy supported,learner-controlled or system-controlled learning in hypermedia environments and the influence of academic self-regulation style

This study focuses on learning in three different hypermedia environments that either support autonomous learning, learner-controlled learning or system-controlled learning and explores the mediating role of academic self-regulation style (ASRS; i.e. a macro level of motivation) on learning. This research was performed to gain more insight in the conditions under which learning in hypermedia environments is effective. Sixty-nine grade five students from a primary school answered short essay questions using video material from a hypermedia environment. The effects on task motivation and test performance were measured. It was found that learners in the autonomy supported hypermedia environment reported lower levels of controlled task motivation, compared to the learners in the system-controlled and learner-controlled hypermedia environments. But there were no effects of hypermedia environment on autonomous task motivation or the reported need fulfilment for autonomy. Furthermore, learners in the learner-controlled hypermedia environment scored lower on a delayed, In-Depth Knowledge Test compared to learners in the other two environments. Moreover, learners in the autonomy supported hypermedia environment watched more (unique) videos compared to learners in the system-controlled and the learner-controlled hypermedia environments. As for the role of learners' ASRS, we found no interaction with the type of hypermedia environment on task motivation or performance. Learners' ASRS did, however, affect the learners' motivation for the task. And, when presented with advice (as in the autonomy supported hypermedia environment) learners with an autonomous self-regulation style followed more advice, compared to learners with a more controlled self-regulation style.     

Risk on Elder Abuse and Mistreatment - Instrument: Development,psychometric properties and qualitative user-evaluation

Prevalence data of elder abuse from social and health services only present a tip of the iceberg. A large amount of situations of abuse is left undetected. Professionals often lack knowledge and skills on the topic of abuse. Consequently, this paper focuses on supporting professionals to prevent and assess elder abuse by developing and testing the Risk on Elder Abuse and Mistreatment Instrument (REAMI), using a mixed method design. Quantitative data from a widespread screening among 1920 older clients of home care are analysed with exploratory and confirmatory analysis. In addition, 24 professionals who have used the REAMI have been qualitatively interviewed about their practical experience and critical reflection on the instrument and its use. These interviews were transcribed and content analysed. Findings provide evidence of good internal reliability and internal validity of the REAMI and its three dimensions: 1) risk factors of the older person, 2) risk factors of the environment, 3) signals of abuse and mistreatment. Users report an increase of knowledge and awareness among staff, satisfaction with the possibilities for prevention and the user-friendliness and brevity of the REAMI. However, some users are missing a follow-up protocol: what to do when risks are assessed? In the discussion, the argument is developed that the REAMI can be used to examine risk at elder abuse, although an overarching protocol to support prevention and intervention which embeds the detection instrument is needed.     

Content validity of game-based assessment: case study of a serious game for ICT managers in training

Serious games foster the acquisition of complex problem-solving skills. Assessment of such skills should be in line with instruction, and within a serious game environment its content validity should equal face-to-face assessment. Research on assessment in serious gaming has remained rather scarce. This article shows how assessment can be implemented in serious gaming in a way that assures content validity. The core of the authors’ validation method entails mapping learning activities (as contained in the game scenario) on performance indicators and outputs (as derived from formal attainment levels). They present how they have elaborated and applied the method for an assessment game for ICT managers in secondary vocational education. They describe the procedure and extent to which this assessment is content-valid compared to face-to-face assessment.     

Evaluating assessment quality in competence-based education: A qualitative comparison of two frameworks

Because learning and instruction are increasingly competence-based, the call for assessment methods to adequately determine competence is growing. Using just one single assessment method is not sufficient to determine competence acquisition. This article argues for Competence Assessment Programmes (CAPs), consisting of a combination of different assessment methods, including both traditional and new forms of assessment. To develop and evaluate CAPs, criteria to determine their quality are needed. Just as CAPs are combinations of traditional and new forms of assessment, criteria used to evaluate CAP quality should be derived from both psychometrics and edumetrics. A framework of 10 quality criteria for CAPs is presented, which is then compared to Messick's framework of construct validity. Results show that the 10-criterion framework partly overlaps with Messick's, but adds some important new criteria, which get a more prominent place in quality control issues in competence-based education.     

四成分教学设计模型——面向复杂学习环境的多媒体原理

四成分教学设计模型认为要实现复杂学习需要借助学习任务、支持性信息、程序性信息和子任务练习等因素.本文探讨运用这一模型设计多媒体学习环境及相关的14条多媒体原理.学习者可以在一个模拟的环境下完成学习任务,利用相关的多媒体原理有利于促进归纳学习过程;学习者还可以在超媒体系统中掌握支持性信息,相关的多媒体原理有利于促进信息的提取和精细加工过程;学习者也可以通过电子绩效支持系统来获取程序性信息,相关的多媒体原理有利于知识处理;最后,学习者能通过基于计算机练习程序来进行子任务的练习与训练,相关的多媒体原理有利于促进知识技能的巩固.本文还讨论了四成分模型框架的启示和不足.     

Selecting learning tasks: Effects of adaptation and shared control on learning efficiency and task involvement

Complex skill acquisition by performing authentic learning tasks is constrained by limited working memory capacity [Baddeley, A. D. (1992). Working memory. Science, 255, 556–559]. To prevent cognitive overload, task difficulty and support of each newly selected learning task can be adapted to the learner’s competence level and perceived task load, either by some external agent, the learner herself, or both. Health sciences students (N = 55) participated in a study using a 2 × 2 factorial design with the factors adaptation (present or absent) and control over task-selection (program control or shared control). As hypothesized, adaptation led to more efficient learning; that is, higher learning outcomes combined with less effort invested in performing the learning tasks. Shared control over task-selection led to higher task involvement, that is, higher learning outcomes combined with more effort directly invested in learning. Adaptation also produced greater task involvement.