Do images influence assessment in anatomy? Exploring the effect of images on item difficulty and item discrimination

Anatomists often use images in assessments and examinations. This study aims to investigate the influence of different types of images on item difficulty and item discrimination in written assessments. A total of 210 of 460 students volunteered for an extra assessment in a gross anatomy course. This assessment contained 39 test items grouped in seven themes. The answer format alternated per theme and was either a labeled image or an answer list, resulting in two versions containing both images and answer lists. Subjects were randomly assigned to one version. Answer formats were compared through item scores. Both examinations had similar overall difficulty and reliability. Two cross‐sectional images resulted in greater item difficulty and item discrimination, compared to an answer list. A schematic image of fetal circulation led to decreased item difficulty and item discrimination. Three images showed variable effects. These results show that effects on assessment scores are dependent on the type of image used. Results from the two cross‐sectional images suggest an extra ability is being tested. Data from a scheme of fetal circulation suggest a cueing effect. Variable effects from other images indicate that a context‐dependent interaction takes place with the content of questions. The conclusion is that item difficulty and item discrimination can be affected when images are used instead of answer lists; thus, the use of images as a response format has potential implications for the validity of test items. Anat Sci Educ © 2012 American Association of Anatomists.     

Equivalence of students' scores on timed and untimed anatomy practical examinations

Untimed examinations are popular with students because there is a perception that first impressions may be incorrect, and that difficult questions require more time for reflection. In this report, we tested the hypothesis that timed anatomy practical examinations are inherently more difficult than untimed examinations. Students in the Doctor of Physical Therapy program at Thomas Jefferson University were assessed on their understanding of anatomic relationships using multiple‐choice questions. For the class of 2012 (n = 46), students were allowed to circulate freely among 40 testing stations during the 40‐minute testing session. For the class of 2013 (n = 46), students were required to move sequentially through the 40 testing stations (one minute per item). Students in both years were given three practical examinations covering the back/upper limb, lower limb, and trunk. An identical set of questions was used for both groups of students (untimed and timed examinations). Our results indicate that there is no significant difference between student performance on untimed and timed examinations (final percent scores of 87.3 and 88.9, respectively). This result also held true for students in the top and bottom 20th percentiles of the class. Moreover, time limits did not lead to errors on even the most difficult, higher‐order questions (i.e., items with P‐values < 0.70). Thus, limiting time at testing stations during an anatomy practical examination does not adversely affect student performance. Anat Sci Educ 6: 281–285. © 2013 American Association of Anatomists.     

An audience response system may influence student performance on anatomy examination questions

This study integrated an in-house audience response system (ARS) in the human anatomy course over two years to determine whether students performed better on high-stakes examinations following exposure to similar interactive questions in a large lecture format. Questions in an interactive ARS format were presented in lectures via PowerPoint presentations. Students who chose to participate in the anonymous ARS sessions submitted answers via their personal wireless devices (e.g., laptops, smartphones, PDAs, etc). Students were surveyed for feedback. Student participation in ARS activities was greatest (65-80%) in the first lecture. The number of students who actively participated in ARS activities decreased over the next four sessions, and then slightly increased in the last two sessions. This trend was the same for both years. Use of the ARS did not dramatically enhance overall student performance on examination questions that dealt with content similar to content presented in the ARS sessions. However, students who scored in the lower quartile of the examination performed better on the examination questions after the ARS was implemented. Accordingly, our findings suggest that the effect of ARS to improve student performance on examinations was not uniform. The overall benefit of an ARS to enhance the lecture experience was confirmed by student surveys.     

Analysis of testing with multiple choice versus open‐ended questions: Outcome‐based observations in an anatomy course

The pedagogical approach for both didactic and laboratory teaching of anatomy has changed in the last 25 years and continues to evolve; however, assessment of student anatomical knowledge has not changed despite the awareness of Bloom's taxonomy. For economic reasons most schools rely on multiple choice questions (MCQ) that test knowledge mastered while competences such as critical thinking and skill development are not typically assessed. In contrast, open‐ended question (OEQ) examinations demand knowledge construction and a higher order of thinking, but more time is required from the faculty to score the constructed responses. This study compares performances on MCQ and OEQ examinations administered to a small group of incoming first year medical students in a preparatory (enrichment) anatomy course that covered the thorax and abdomen. In the thorax module, the OEQ examination score was lower than the MCQ examination score; however, in the abdomen module, the OEQ examination score improved compared to the thorax OEQ score. Many students attributed their improved performance to a change from simple memorization (superficial learning) for cued responses to conceptual understanding (deeper learning) for constructed responses. The results support the view that assessment with OEQs, which requires in depth knowledge, would result in student better performance in the examination. Anat Sci Educ 11: 254–261. © 2017 American Association of Anatomists.     

Anatomy "steeplechase" online: necessity sometimes is the catalyst for innovation

In most medical schools, summative practical examination in Anatomy usually takes the format of a “steeplechase” (“spotters” or “bell ringers”) conducted in the gross anatomy laboratory using cadaveric material and prosected specimens. Recently, we have started to administer similar examinations online using the quiz facility in WebCT? and Moodle?. This article chronicles how we conceived and developed this method within the peculiar nature of our medical school setting. Over a five year period, practical summative examinations were organized as “steeplechase” online. The online examinations were administered using WebCT? and later Moodle? learning management software. Assessment “objects” were created from the materials available for anatomy teaching. These were digital images of cadaveric materials, radiological, and prosected specimens. In addition, short video clips of 30 seconds duration demonstrating muscle action were produced. These objects were optimized for online viewing and then uploaded onto the learning management software. A bank of questions (multiple choice or short answer type) was then created and linked to the assessment objects. These were used in place of the steeplechase in the computer laboratory. This method serves a crucial purpose in places like ours where continuous availability of human cadavers is impossible. Although time consuming initially, once questions are setup online, future retrieval, and administration becomes convenient especially where there are large batches of students. In addition, the online environment offers distinct advantages with regards to image quality, psychometric analysis of the examination and reduction of staff preparation time compared to traditional “steeplechase.” Anat Sci Educ 4: 115–118, 2011. © 2010 American Association of Anatomists.     

Student use of PeerWise: A multi-institutional,multidisciplinary evaluation

This study explores the relationship between engagement with an online, free-to-use question-generation application (PeerWise) and student achievement. Using PeerWise, students can create and answer multiple-choice questions and can provide feedback to the question authors on question quality. This provides further scope for students to engage in discussion about the question with their peers. Data on PeerWise use and examination performance were collected from over 3000 students across six large undergraduate courses (in physics, chemistry and biology) over three academic years in three research-intensive UK universities. A reliable and valid measure of overall PeerWise activity was created and a multilevel model developed describing the relationship between PeerWise activity and student performance in end of course examinations. Using this approach, a significant positive association was found between students’ engagement with PeerWise and their academic attainment in end of course exams, even controlling for prior ability. The implications of these findings for educators are discussed.     

Multiple-Choice versus Open-Ended Questions in Advanced Clinical Neuroanatomy: Using a National Neuroanatomy Assessment to Investigate Variability in Performance Using Different Question Types

Methods of assessment in anatomy vary across medical schools in the United Kingdom (UK) and beyond; common methods include written, spotter, and oral assessment. However, there is limited research evaluating these methods in regards to student performance and perception. The National Undergraduate Neuroanatomy Competition (NUNC) is held annually for medical students throughout the UK. Prior to 2017, the competition asked open-ended questions (OEQ) in the anatomy spotter examination, and in subsequent years also asked single best answer (SBA) questions. The aim of this study is to assess medical students’ performance on, and perception of, SBA and OEQ methods of assessment in a spotter style anatomy examination. Student examination performance was compared between OEQ (2013–2016) and SBA (2017–2020) for overall score and each neuroanatomical subtopic. Additionally, a questionnaire explored students’ perceptions of SBAs. A total of 631 students attended the NUNC in the studied period. The average mark was significantly higher in SBAs compared to OEQs (60.6% vs. 43.1%, P < 0.0001)—this was true for all neuroanatomical subtopics except the cerebellum. Students felt that they performed better on SBA than OEQs, and diencephalon was felt to be the most difficult neuroanatomical subtopic (n = 38, 34.8%). Students perceived SBA questions to be easier than OEQs and performed significantly better on them in a neuroanatomical spotter examination. Further work is needed to ascertain whether this result is replicable throughout anatomy education.     

Anatomy practical examinations: How does student performance on computerized evaluation compare with the traditional format?

Practical examinations in anatomy are usually conducted on specimens in the anatomy laboratory (referred to here as the “traditional” method). Recently, we have started to administer similar examinations online using the quiz facility in Moodle?. In this study, we compare student scores between two assessment environments viz. online and traditional environments. We hypothesized that regardless of the examination medium (traditional or online) overall student performance would not be significantly different. For the online medium, radiological images, prosected specimens, and short video clips demonstrating muscle action were first acquired from resources used for teaching during anatomy practical classes. These were optimized for online viewing and then uploaded onto Moodle learning management software. With regards to the traditional format, actual specimens were usually laid out in a circular stream. Identification tags were then attached to specific spots on the specimens and questions asked regarding those identified spots. A cohort of students taking practical examinations in six courses was studied. The courses were divided into three pairs with each pair credit‐weight matched. Each pair consisted of a course where the practical examination was conducted online and the other in the traditional format. There was no significant difference in the mean scores within each course pair. In addition, a significant positive correlation between score in traditional and online formats was found. We conclude that mean grades in anatomy practical examination conducted either online or in the traditional format were comparable. These findings should reassure teachers intending to use either format for their practical examinations. Anat Sci Educ. © 2011 American Association of Anatomists.     

Effects of Alternating Dissection with Peer Teaching and Faculty Prosected Cadaver Demonstrations in a Physical Therapy and Occupational Therapy Gross Anatomy Course

The most effective method to teach gross anatomy is largely unknown. This study examined two teaching methods utilized in a physical therapy and occupational therapy gross anatomy course, (1) alternating dissection with peer teaching every other laboratory session and (2) faculty demonstrations during laboratory sessions. Student (n = 57) subgroup (A or B) academic performance was determined using written, laboratory practical, and palpation practical examinations. Subgroup A performed significantly better on laboratory practical examination questions pertaining to dissected, in comparison to peer-taught structures (67.1% vs. 60.2%, P = 0.008). Subgroup B performed significantly better on laboratory practical examination questions pertaining to peer-taught, in comparison to dissected structures (64.1% vs. 57.9%, P = 0.001). When Subgroup A was compared to Subgroup B, there were no statistically significant differences on laboratory practical examination question types, whether the subgroup learned the structure through dissection or peer teaching. Based on within and between subgroup comparisons, faculty demonstrations had no effect on written, laboratory practical, or palpation practical examination scores. Although limited, data suggest that the student roles when alternating dissection with peer teaching every other laboratory session appear to be equally effective for learning gross anatomy. The benefits of this method include decreased student/faculty ratio in laboratory sessions and increased time for independent study. Faculty demonstrations during laboratory sessions do not seem to improve student academic performance.     

Oral anatomy laboratory examinations in a physical therapy program

The process of creating and administering traditional tagged anatomy laboratory examinations is time consuming for instructors and limits laboratory access for students. Depending on class size and the number of class, sections, creating, administering, and breaking down a tagged laboratory examination may involve one to two eight‐hour days. During the time that a tagged examination is being created, student productivity may be reduced as the anatomy laboratory is inaccessible to students. Further, the type of questions that can be asked in a tagged laboratory examination may limit student assessment to lower level cognitive abilities and may limit the instructors' ability to assess the students' understanding of anatomical and clinical concepts. Anatomy is a foundational science in the Physical Therapy curriculum and a thorough understanding of anatomy is necessary to progress through the subsequent clinical courses. Physical therapy curricula have evolved to reflect the changing role of physical therapists to primary caregivers by introducing a greater scope of clinical courses earlier in the curriculum. Physical therapy students must have a thorough understanding of clinical anatomy early in the education process. However, traditional anatomy examination methods may not be reflective of the clinical thought processes required of physical therapy students. Traditional laboratory examination methods also reduce student productivity by limiting access during examination set‐up and breakdown. To provide a greater complexity of questions and reduced overall laboratory time required for examinations, the Physical Therapy Program at Mercer University has introduced oral laboratory examinations for the gross anatomy course series. Anat Sci Educ 6: 271–276. © 2012 American Association of Anatomists.     

Summative and Formative Style Anatomy Practical Examinations: Do They Have Impact on Students’ Performance and Drive for Learning?

Anatomical knowledge is commonly assessed by practical examinations that are often administered in summative format. The format of anatomy practical examination was changed at the Lee Kong Chian School of Medicine in Singapore from summative (graded; must pass) to formative (ungraded; no pass/fail) in academic year (AY) 2017–2018. Both assessment formats were undertaken online, but the formative mode used a team-based learning activity comprising individual and team assessments. This gave an unique opportunity to investigate: (1) the impact of two different online assessment formats on student performance in practical examination; (2) the impact of new formative practical examination on students’ performance in summative examinations; and (3) students’ opinions of these two practical examination formats. The class of 2021 perceptions was obtained as they experienced both formats. A retrospective cohort study was also conducted to analyze the Year 2 students’ performance in anatomy practical and year-end summative examinations of cohorts AY 2015–2016, AY 2016–2017 (summative format), and AY 2017–2018 (formative format). There were no significant differences in students’ performance between two practical examination formats. The cohort who experienced the formative format, performed significantly better in summative examinations (mean ± SD: 82.32 ± 10.22%) compared with the cohort who experienced the summative format (73.77 ± 11.09%) (P < 0.001). Students highlighted positive features of the formative practical examination, including team reinforcement of learning, instant feedback, and enhanced learning. These findings indicate that students continue to study for anatomy practical examination without the need for external drivers. The team-based learning style practical examination enhances students’ performance in summative examinations.     

The effect of curricular reform on gross anatomy laboratory examination performance: An institutional analysis

Many medical schools have undergone curricular reform recently. With these reforms, time spent teaching anatomy has been reduced, and there has been a general shift to a pass/fail grading system. At Indiana University School of Medicine (IUSM), a new curriculum was implemented in fall 2016. The year-long human gross anatomy course taught in 2015 was condensed into an integrated, semester-long course starting in 2016. Additionally, the grading scale shifted to pass/fail. This study examined first-year medical student performance on anatomy practical laboratory examinations—specifically, among lower-order (pure identification) questions and higher-order (function, innervation) questions. Participants included medical students from a pre-curricular reform cohort (year 2015, 34 students) and two post-curricular reform cohorts (years 2016, 30 students and 2017, 33 students). A Kruskal–Wallis ANOVA test was used to determine differences of these questions among the three cohorts. Additionally, 40 of the same lower-order questions that were asked on gross anatomy laboratory examinations from medical student cohort year 2015 and year 2016 were further analyzed using an independent samples t-test. Results demonstrated that the pre-curricular reform cohort scored significantly higher on both lower-order (median = 81, p < 0.001) and higher-order questions (median = 82.5, p < 0.05) than both post-curricular reform cohorts. Additionally, when reviewing the selected 40 similar questions, it was found that the pre-curricular reform cohort averaged significantly higher (82.1 ± 16.1) than the post-curricular reform cohort from 2016 (69.3 ± 21.8, p = 0.004). This study provides evidence about the impact of curricular reform on medical student anatomical knowledge.     

Computerized grading of anatomy laboratory practical examinations

At the Medical College of Wisconsin, a procedure was developed to allow computerized grading and grade reporting of laboratory practical examinations in the Clinical Human Anatomy course. At the start of the course, first year medical students were given four Lists of Structures. On these lists, numbered items were arranged alphabetically; the items were anatomical structures that could be tagged on a given lab practical examination. Each lab exam featured an anatomy laboratory component and a computer laboratory component. For the anatomy lab component, students moved from one question station to another at timed intervals and identified tagged anatomical structures. As students identified a tagged structure, they referred to a copy of the list (provided with their answer sheet) and wrote the number corresponding to the structure on their answer sheet. Immediately after the anatomy lab component, students were escorted to a computer instruction laboratory where they typed their answer numbers into a secured testing component of a learning management system that recorded their answers for automatic grading. After a brief review of examination scores and item analysis by faculty, exam scores were reported to students electronically. Adding this brief computer component to each lab exam greatly reduced faculty grading time, reduced grading errors and provided faster performance feedback for students without changing overall student performance. Anat Sci Ed 1:220–223, 2008. © 2008 American Association of Anatomists.     

A “Do‐It‐Yourself” interactive bone structure module: Development and evaluation of an online teaching resource

A stand‐alone online teaching module was developed to cover an area of musculoskeletal anatomy (structure of bone) found to be difficult by students. The material presented in the module was not formally presented in any other way, thus providing additional time for other curriculum components, but it was assessed in the final examination. The module was developed using “in‐house” software designed for academics with minimal computer experience. The efficacy and effectiveness of the module was gauged via student surveys, testing student knowledge before and after module introduction, and analysis of final examination results. At least 74% of the class used the module and student responses were positive regarding module usability (navigation, interaction) and utility (learning support). Learning effectiveness was demonstrated by large significant improvements in the post‐presentation test scores for “users” compared with “non‐users” and by the percentage of correct responses to relevant multiple choice questions in the final examination. Performance on relevant short answer questions in the final examination was, on average, comparable to that for other components. Though limited by study structure, it was concluded that the module produced learning outcomes equivalent to those generated by more traditional teaching methods. This “Do‐It‐Yourself” e‐learning approach may be particularly useful for meeting specific course needs not catered for by commercial applications or where there are cost limitations for generation of online learning material. The specific approaches used in the study can assist in development of effective online resources in anatomy. Anat Sci Educ 6: 107–113. © 2012 American Association of Anatomists.     

The effect of images on item statistics in multiple choice anatomy examinations

Although multiple choice examinations are often used to test anatomical knowledge, these often forgo the use of images in favor of text‐based questions and answers. Because anatomy is reliant on visual resources, examinations using images should be used when appropriate. This study was a retrospective analysis of examination items that were text based compared to the same questions when a reference image was included with the question stem. Item difficulty and discrimination were analyzed for 15 multiple choice items given across two different examinations in two sections of an undergraduate anatomy course. Results showed that there were some differences item difficulty but these were not consistent to either text items or items with reference images. Differences in difficulty were mainly attributable to one group of students performing better overall on the examinations. There were no significant differences for item discrimination for any of the analyzed items. This implies that reference images do not significantly alter the item statistics, however this does not indicate if these images were helpful to the students when answering the questions. Care should be taken by question writers to analyze item statistics when making changes to multiple choice questions, including ones that are included for the perceived benefit of the students. Anat Sci Educ 10: 68–78. © 2016 American Association of Anatomists.     

Investigating the influence of question type and cognitive process on academic performance in VCE Physical Education: a secondary data analysis

External written examinations are commonly used for determining student academic achievement. The influence of question type and cognitive process on examination performance in senior-secondary physical education is unclear. A secondary data analysis of Victorian Certificate of Education (VCE) Physical Education examination data (2011; n?=?9,323, 2012; n?=?8,781) was conducted. Question type (multiple choice and short answer) and overall examination performance were compared and the predictive value of question type, cognitive process (based on Bloom’s revised taxonomy), and overall examination scores determined. In 2011 and 2012, students performed significantly better on multiple-choice questions; however, short-answer performance better predicted overall exam performance. A significant difference between marks achieved by cognitive level and grade (Ungraded [UG] – A+) was found. Low-achieving students (UG – D) were performing well below the examination mean across all questions. Developing higher order thinking skills for all students may lead to improved overall examination performance in VCE Physical Education.     

Student laboratory presentations as a learning tool in anatomy education

Previous studies have shown that anatomy students who complete oral laboratory presentations believe they understand the material better and retain it longer than they otherwise would if they only took examinations on the material; however, we have found no studies that empirically test such outcomes. The purpose of this study was to assess the effectiveness of oral presentations through comparisons with other methods of assessment, most notably, examination performance. Specifically, we tested whether students (n = 256) performed better on examination questions on topics covered by their oral presentations than on other topics. Each student completed two graded, 12‐minute laboratory presentations on two different assigned topics during the course and took three examinations, each of which covered a third of the course material. Examination questions were characterized by type (memorization, pathway, analytical, spatial). A two‐way repeated measures analysis of variance revealed that students performed better on topics covered by their presentations than on topics not covered by their presentations (P < 0.005), regardless of presentation grade (P > 0.05) and question type (P > 0.05). These results demonstrate empirically that oral presentations are an effective learning tool. Anat Sci Educ 2: 260–264, 2009. © 2009 American Association of Anatomists.     

What is an objective structured practical examination in anatomy?

Assessing teaching‐learning outcomes in anatomical knowledge is a complex task that requires the evaluation of multiple domains: theoretical, practical, and clinical knowledge. In general, theoretical knowledge is tested by a written examination system constituted by multiple choice questions (MCQs) and/or short answer questions (SAQ). The assessment of practical knowledge (three‐dimensional anatomical concepts) involves oral, spot, or objective structured practical examinations (OSPE). Finally, the application of anatomical knowledge to patients is tested mainly through objective structured clinical examinations (OSCE). The major focus of this study is the OSPE. Although many schools challenge students using this tool in practical examinations in the early phase of the curriculum, the true meaning of OSPE is frequently forgotten and it becomes, in reality, a spot examination. This article, for the first time, describes how the concept of the OSPE has evolved and is currently being used to assess the practical domain of anatomical knowledge in a problem‐based curriculum at Alfaisal University College of Medicine. In addition, it describes the main differences from the spot examination, which is normally used in traditional medical curricula. The authors believe that the OSPE remains the most efficient tool to assess the practical aspects of anatomical knowledge in a system where basic knowledge is integrated with the clinical or functional part of anatomy. However, this contention only holds true if the OSPE process revolves around structured objectives. Anat Sci Educ 6: 125–133. © 2012 American Association of Anatomists.     

A Pilot Study of a Physical Therapy Cadaveric Anatomy Course: Assessing the Impact of Examination Format on the Learning Environment

Anatomy education continues to evolve in health professional programs as curricula shift to competency-based models and contact hours decrease. These changes in curricula may significantly alter the learning environment for students. Importantly, changes in learning environment have been shown to impact student learning strategies and well-being. It follows, then, that an investigation of students' perceptions of the learning environment is key to understand the impact of modern anatomy curriculum alterations. The current pilot study evaluated the impact of modifying examination format on the learning environment of physical therapy students participating in a human cadaveric anatomy course. Two study cohorts of first year (entry-level) physical therapy students were invited to complete a preliminary learning environment questionnaire with 13 visual analog scale items and four short answer items. One study cohort was tested with a viva (oral) practical examination, and the other, with a bell-ringer practical examination. Analysis of quantitative items revealed two significant findings: physical therapy students in the bell-ringer cohort found it was more difficult to prepare for their examination, and that they had inadequate time to respond to questions compared to the viva cohort. Analysis of qualitative items revealed distinct themes that concerned physical therapy student learning environment specific to cadaveric anatomy. These results demonstrate that examination format can influence the learning environment of physical therapy students studying cadaveric anatomy. As a result, care needs to be taken to ensure that modernized curricula align the examination format to the method of instruction and the future application of students' knowledge in clinical practice.     

Higher-Order Assessment in Gross Anatomy: A Comparison of Performance on Higher- Versus Lower-Order Anatomy Questions between Undergraduate and First-Year Medical Students

Gross anatomy is considered by many the backbone of medical education. While learning anatomy has a reputation of requiring mainly rote memorization, modern day anatomy education often involves instruction and assessment at cognitive levels that foster higher-order thinking. In many instances, these higher-order anatomical concepts are taught to graduate students in healthcare-related fields, such as medicine. At this level, students are expected to apply and analyze anatomical information since that is what will ultimately be expected of them as professionals. In contrast, undergraduate anatomy education is typically more introductory in nature and often takes place in the setting of a large-enrollment course that serves as a prerequisite for many health sciences degree programs. In this study, variables related to the assessment of higher-order concepts in clinical anatomy were compared between first-year medical students and undergraduate students enrolled in an upper-level human gross anatomy course. Results demonstrate that undergraduate students perform lower than medical students overall, but the degree of difference in how they perform on higher- versus lower-order questions is comparable. The most notable exception is on practical examinations, where undergraduate students tend to struggle more with applying and analyzing information. Exploration of additional variables provides insight into how the cognitive level being assessed affects the time it takes to answer a question and how different practical examination question types and formats influence student performance. Findings presented in this study have implications for designing anatomy courses and underscore the importance of blueprinting assessments.