Evaluating the future value of educational interventions in a health care setting

Program evaluation is intended to systematically determine the merit and worth of a program. We broadly define merit and worth to include economic perspectives obtained through the analysis of the finances of a program—both the costs and the benefits. A study reported here addressed the cost and benefit of investing in an educational intervention to change and improve processes in a health care setting. The potential economic impact of a training intervention intended to improve the uptake of a new purchasing process was evaluated. The study illustrates economic evaluation that anticipates the future value of educational interventions, a prospective view.     

Brains on the Table: Diagnostic and formative assessment through observation

The majority of assessment in secondary schools in England and Wales is summative both in nature and purpose but as a tool for learning, assessment must be a dynamic dialogue between teacher and pupil which influences future teaching and learning activities. Diagnostic assessment of pupils is constrained by the nature of the tasks in which they are asked to engage. This paper describes the use of 'mysteries' as an enjoyable assessment tool which provides evidence of pupils' cognitive processes through the observation and analysis of pupils' manipulation of data slips to solve the mystery. Stages in this process are identified and their resemblance to the structure of the observed learning outcome (SOLO) taxonomy is established. A diagnostic interpretation of the difficulties pupils may be observed to encounter is suggested and appropriate formative action is outlined. Finally, the collaborative nature of learning and the validity of mysteries as assessment tools are discussed.     

It's Not Just Angoff: Misperceptions of Hard and Easy Items in Bookmark-Type Ratings

A common belief is that the Bookmark method is a cognitively simpler standard-setting method than the modified Angoff method. However, a limited amount of research has investigated panelist's ability to perform well the Bookmark method, and whether some of the challenges panelists face with the Angoff method may also be present in the Bookmark method. This article presents results from three experiments where panelists were asked to give Bookmark-type ratings to separate items into groups based on item difficulty data. Results of the experiments showed, consistent with results often observed with the Angoff method, that panelists typically and paradoxically perceived hard items to be too easy and easy items to be too hard. These perceptions were reflected in panelists often placing their Bookmarks too early for hard items and often placing their Bookmarks too late for easy items. The article concludes with a discussion of what these results imply for educators and policymakers using the Bookmark standard-setting method.     

‘It's the best job in the world,but one of the hardest,loneliest, most misunderstood roles in a school.’ Understanding the complexity of the SENCO role post-SEND reform

In schools, the Special Educational Needs Coordinator (SENCO) could be considered a key policy implementer of special educational needs and inclusive policy. Issues related to time, status and the effective facilitation of the SENCO role have been reported on extensively, yet literature has predominantly focused on the role prior to the introduction of the Special Educational Needs and Disability (SEND) reforms in 2014. This paper reports on research which explored the SENCO role post-reform. The research aimed to understand how widely the role varied across differing educational settings and phases, whilst also exploring the breadth and depth of the role, post-SEND reform. The research design was mixed methods and had two phases: online focus groups (n = 15), followed by a national online survey (n = 1903). The findings suggest that the facilitation of the SENCO role remains problematic post-reform. Constraints include the time to undertake responsibilities, the increasing breadth of the role and how the role is understood by others. This combined with increased external bureaucracy, budgetary constraints and a lack of consistency nationally has led to a situation where only approximately one-third of SENCOs intend to remain in the role in five years’ time.     

A Pilot Study of a Physical Therapy Cadaveric Anatomy Course: Assessing the Impact of Examination Format on the Learning Environment

Anatomy education continues to evolve in health professional programs as curricula shift to competency-based models and contact hours decrease. These changes in curricula may significantly alter the learning environment for students. Importantly, changes in learning environment have been shown to impact student learning strategies and well-being. It follows, then, that an investigation of students' perceptions of the learning environment is key to understand the impact of modern anatomy curriculum alterations. The current pilot study evaluated the impact of modifying examination format on the learning environment of physical therapy students participating in a human cadaveric anatomy course. Two study cohorts of first year (entry-level) physical therapy students were invited to complete a preliminary learning environment questionnaire with 13 visual analog scale items and four short answer items. One study cohort was tested with a viva (oral) practical examination, and the other, with a bell-ringer practical examination. Analysis of quantitative items revealed two significant findings: physical therapy students in the bell-ringer cohort found it was more difficult to prepare for their examination, and that they had inadequate time to respond to questions compared to the viva cohort. Analysis of qualitative items revealed distinct themes that concerned physical therapy student learning environment specific to cadaveric anatomy. These results demonstrate that examination format can influence the learning environment of physical therapy students studying cadaveric anatomy. As a result, care needs to be taken to ensure that modernized curricula align the examination format to the method of instruction and the future application of students' knowledge in clinical practice.     

The Interaction of Issue and Image Frames on Political Candidate Assessment

This study examined the combined effect of character and policy coverage about a political candidate in news media on voters’ evaluations and thoughts about the candidate. A 2 (issue coverage: present/absent) × 2 (image coverage: present/absent) between-subject factorial experiment was conducted whereby participants (N = 134) read a version of an editorial article that varied in the coverage of a candidate’s character and policy positions. Results indicated that issue and image coverage, each in isolation, enhanced perceptions of the candidate’s image strength and issue strength. However, issue and image coverage when presented in combination compared to in isolation did not enhance perceptions of the candidate and in some cases hindered perceptions of candidate strength. When both coverage types were present, issue coverage dominated image coverage when participants formed image evaluations, whereas image coverage dominated issue coverage when they formed issue evaluations. Similar findings resulted for issue- and image-relevant thoughts. Implications about the effects of multicoverage political messages are discussed.     

Breaking with Tradition: A Scoping Meta-Analysis Analyzing the Effects of Student-Centered Learning and Computer-Aided Instruction on Student Performance in Anatomy

While prior meta-analyses in anatomy education have explored the effects of laboratory pedagogies and histology media on learner performance, the effects of student-centered learning (SCL) and computer-aided instruction (CAI) have not been broadly evaluated. This research sought to answer the question, “How effective are student-centered pedagogies and CAI at increasing student knowledge gains in anatomy compared to traditional didactic approaches?” Relevant studies published within the past 51 years were searched using five databases. Predetermined eligibility criteria were applied to the screening of titles and abstracts to discern their appropriateness for study inclusion. A summary effect size was estimated to determine the effects of SCL and CAI on anatomy performance outcomes. A moderator analysis of study features was also performed. Of the 3,035 records screened, 327 underwent full-text review. Seven studies, which comprised 1,564 participants, were included in the SCL analysis. An additional 19 studies analyzed the effects of CAI in the context of 2,570 participants. Upon comparing SCL to traditional instruction, a small positive effect on learner performance was detected (standardized mean difference (SMD = 0.24; [CI = 0.07, 0.42]; P = 0.006). Likewise, students with CAI exposure moderately outscored those with limited or no access to CAI (SMD = 0.59; [CI = 0.20, 0.98]; P = 0.003). Further analysis of CAI studies identified effects (P ≤ 0.001) for learner population, publication period, interventional approach, and intervention frequency. Overall, learners exposed to SCL and supplemental CAI outperformed their more classically-trained peers as evidenced by increases in short-term knowledge gains. Anat Sci Educ. © 2018 American Association of Anatomists.