Automatic testing and assessment of neuroanatomy using a digital brain atlas: Method and development of computer‐ and mobile‐based applications

Preparation of tests and student's assessment by the instructor are time consuming. We address these two tasks in neuroanatomy education by employing a digital media application with a three‐dimensional (3D), interactive, fully segmented, and labeled brain atlas. The anatomical and vascular models in the atlas are linked to Terminologia Anatomica. Because the cerebral models are fully segmented and labeled, our approach enables automatic and random atlas‐derived generation of questions to test location and naming of cerebral structures. This is done in four steps: test individualization by the instructor, test taking by the students at their convenience, automatic student assessment by the application, and communication of the individual assessment to the instructor. A computer‐based application with an interactive 3D atlas and a preliminary mobile‐based application were developed to realize this approach. The application works in two test modes: instructor and student. In the instructor mode, the instructor customizes the test by setting the scope of testing and student performance criteria, which takes a few seconds. In the student mode, the student is tested and automatically assessed. Self‐testing is also feasible at any time and pace. Our approach is automatic both with respect to test generation and student assessment. It is also objective, rapid, and customizable. We believe that this approach is novel from computer‐based, mobile‐based, and atlas‐assisted standpoints. Anat Sci Educ 2:244–252, 2009. © 2009 American Association of Anatomists.     

Performance equivalency between computer‐based and traditional pen‐and‐paper assessment: A case study in clinical anatomy

The time, material, and staff‐consuming nature of anatomy's traditional pen‐and‐paper assessment system, the increase in the number of students enrolling in medical schools and the ever‐escalating workload of academic staff have made the use of computer‐based assessment (CBA) an attractive proposition. To understand the impact of such shift in the assessment method, an experimental study evaluating its effect on students’ performance was designed. Additionally, students’ opinions toward CBA were gathered. Second‐year medical students attending a Clinical Anatomy course were randomized by clusters in two groups. The pen‐and‐paper group attended two sessions, each consisting of a traditional sectional anatomy steeplechase followed by a theoretical examination, while the computer group was involved in two similar sessions conducted in a computerized environment. At the end of each of the computer sessions, students in this group filled an anonymous questionnaire. In the first session, pen‐and‐paper group students scored significantly better than computer‐group students in both the steeplechase (mean ± standard deviation: 66.00 ± 14.15% vs. 43.50 ± 19.10%; P < 0.001) and the theoretical examination (52.50 ± 12.70% vs. 39.00 ± 21.10%; P < 0.001). In the second session, no statistically significant differences were found for both the steeplechase (59.50 ± 17.30% vs. 54.50 ± 17.00%; P = 0.085) and the theoretical examination (57.50 ± 13.70% vs. 54.00 ± 14.30%; P = 0.161). Besides, an intersession improvement in students’ perceptions toward CBA was registered. These results suggest that, after a familiarization period, CBA might be a performance equivalent and student accepted alternative to clinical anatomy pen‐and‐paper theoretical and practical examinations. Anat Sci Educ 11: 124–136. © 2017 American Association of Anatomists.     

Automated performance assessment and feedback for free‐play simulation‐based training

Practice and experience, whether simulated or on the job, are not enough to ensure effective learning. Learners must be able to make sense of those experiences to identify poor decisions and actions, missing knowledge, and weak skills that deserve attention. Using instructors to provide one‐on‐one instruction is effective but also expensive. This article describes ways of using intelligent software to assess student performance and provide feedback automatically in free‐play simulations. Case studies describe applications of these methods.     

Lecturers’ views of assessment design,marking and feedback in higher education: a case for professionalisation?

Abstract

Research on professional assessment practice in higher education does not always take account of lecturers’ perceptions and experiences, or their disciplinary context. This questionnaire study was designed to explore lecturers’ views about three inter-related elements of professional assessment practice: assessment design, marking and feedback. It consisted of two questionnaires: the Assessment Design Inventory (ADI), previously published, and the Assessment, Marking and Feedback Inventory (AMFI) which was newly developed for this study. The two main purposes of the research were to: i) establish the validity of these two questionnaires as research tools, ii) explore the existence of professional practice in assessment and analyse any perceived hindrances as well as possible effects of participants’ subject discipline. 356 lecturers from two UK universities, with a range of teaching experience and representing soft and hard disciplines, took part. Results indicated that the ADI and the AMFI were robust instruments. Findings showed evidence of assessment professionalism, but also some areas where it was lacking, together with a perceived need for formal training. Participants recognised that there were certain hindrances in putting their pedagogical beliefs into practice. The implications of this study are discussed in terms of advancing the sector’s understanding of assessment professionalism.     

Evaluation of a computer‐based reading intervention in infant and junior schools

Recent studies (Fawcett et al, 1999; Nicolson et al, 1999) have demonstrated that 6 and 8 year old children found to be at risk of reading failure can be provided with very cost‐effective support using traditional teaching methods. Nonetheless, a significant minority of children failed to make sufficient progress. The present studies undertook a replication of these studies, using the RITA computer‐based literacy support system. The RITA system assists, rather than replaces, the teacher in providing support tailored to each child’s profile of reading attainments. The results indicated that the RITA support was both educationally effective and economically effective, with comparable cost effectiveness to the traditional approach when used with infant school children (6 year olds), and somewhat greater cost effectiveness when used with junior school children (8 year olds). All groups of children reacted very positively to the RITA lessons. It is suggested that computer‐assisted reading support can be effective in supporting children at risk of reading failure, even at junior school level.     

Evaluating a computer‐based reading programme with children with Intellectual Disabilities: feasibility and pilot research

Increasing evidence indicates that individuals with Intellectual Disabilities (ID) might benefit from phonics‐based reading instruction. However, research and instruction in this field has predominantly focused on sight word reading. Models for complex interventions recommend that feasibility research be conducted prior to conducting randomised studies to assess efficacy of interventions (Thabane et al., 2010). The aim of the current paper is therefore to investigate feasibility questions relating to conducting a full‐scale randomised controlled trial (RCT) evaluation of an online, phonics‐based reading programme (Headsprout^? Early Reading; HER) with children with ID. Employing a randomised pre‐test post‐test group design, this study explores and trials important aspects of a RCT evaluation to inform a full‐scale RCT. We also found that HER had a significant effect on reading skills when compared with ‘education as usual’, with large effect sizes on the main outcome measure. This indicates that further, more robust evaluations using HER with children with ID are a worthwhile pursuit.     

Assisting transition to university: using assessment as a formative learning tool

This study explores the effectiveness of an intervention involving formative assessment in a first‐year core business subject. Students were invited to receive feedback on a draft of their first written assessment during the early weeks of the semester. Consideration is given to the economic and ethical issues raised by the intervention. A multi‐method approach of qualitative and quantitative data collection and analysis is used. The research finds that the intervention facilitates significantly higher marks in assessments and grades, while assisting student learning overall. Findings are reinforced by comparison with a subject where the intervention was not offered.     

Investigating the role of computer‐supported annotation in problem‐solving‐based teaching: An empirical study of a Scratch programming pedagogy

For more than 2 years, Scratch programming has been taught in Taiwanese elementary schools. However, past studies have shown that it is difficult to find appropriate learning methods or tools to boost students’ Scratch programming performance. This inability to readily identify tutoring tools has become one of the primary challenges addressed in Scratch programming studies. To cope with this problem, we propose an innovative approach, which combines an Annotation‐based Scratch Programming (ASP) tool with the problem‐solving‐based teaching approach in Scratch programming pedagogy. The ASP tool was developed to enable students to create, review and share Scratch programming and homework annotations. In a quasi‐experimental study, we have evaluated Scratch programming pedagogy at a North Taiwanese elementary school to investigate the effects of instructional‐tools‐supported programming instructional modes on Scratch programming performance. The experimental results show that students who received ASP tool support in conjunction with a problem‐solving‐based teaching approach performed significantly better than the other groups. Based on our findings, the innovative approach was believed to play an important role in improving the learning patterns of younger pupils. Therefore, we suggest that teachers consider incorporating the innovative method into their teaching environments in order to boost students’ learning achievements in the area of Scratch programming and the subsequent learning process.     

Supporting learning of variable control in a computer‐based biology environment: Effects of prompting college students to reflect on their own thinking

While instruction on control of variables has been shown to be effective, especially when it encourages students to focus explicitly on rules or procedures, little evidence of application to novel problems has been obtained. We hypothesized that prompting students to understand their own learning processes while doing experiments involving control of variables would allow them to activate their repertoire of knowledge and strategies and learn in a way that would enhance transfer of learning. Students were assigned to one of four versions of a computer‐based biology simulation learning environment, each employing a different type of prompt: reason justification, rule based, emotion focused, or none (control). Learning in this computer environment, college biology students designed and conducted experiments involving control of variables. Students' ability to solve both contextually similar (near transfer) and contextually dissimilar (far transfer) problems was assessed. The treatment groups performed equally well on contextually similar problems. However, on a contextually dissimilar problem, the reason justification group had significantly higher scores than the other groups. Qualitative data showed that the reason justification prompts directed students' attention to understanding when, why, and how to employ experiment design principles and strategies, and this in turn helped students to transfer their understanding to a novel problem. © 1999 John Wiley & Sons, Inc. J Res Sci Teach 36: 837–858, 1999     

Design and usability assessment of a dialogue‐based cognitive tutoring system to model expert problem solving in research design

Technology provides the means to create useful learning and practice environments for learners. Well‐designed cognitive tutor systems, for example, can provide appropriate learning environments that feature cognitive supports (ie, scaffolding) for students to increase their procedural knowledge. The purpose of this study was to conduct a series of usability tests of a dialogue‐based design framework for the presentation of domain knowledge and assess how it can be used to actively engage learners in learning about research methods. Three formal usability assessments and an instructor adoption assessment were conducted during the development of the tutoring system. Each usability assessment employed diverse data collection methods to ensure broad and in‐depth coverage of findings. The findings revealed that the dialogue metaphor enabled natural and participatory interactions between the system and users. The feedback prompts or hints and support resources provided opportunities for learning during the process of problem solving. Future research to extend the support of usability assessments is also discussed.     

From copying to learning: using exemplars to engage students with assessment criteria and feedback

Feedback is central to pedagogic theory, and if feedback is to be effective, students need to engage with it and apply it at some point in the future. However, student dissatisfaction with feedback – as evidenced in the National Student Survey – suggests that there are problems which limit student engagement with feedback, such as their perception that much of their feedback is irrelevant to future assignments. This article reports on a study which sought to enhance engagement by giving students exemplar assignments annotated with feedback before submission of their final assignments. This was done by providing an online facility where students could view exemplars and post comments or questions to tutors and peers on a discussion board. The exemplar facility was highly valued by students, although there were no quantitative effects such as an increase in students’ assignment marks when compared with the previous cohort. The article reflects on possible reasons for this result and discusses ways to improve the exemplar facility, for example by facilitating dialogue between tutors and students. The article concludes with lessons learned about how to construct exemplars, and considers how exemplars might also be used within marking teams to improve consistency of marking.     

Self‐regulation empowerment program: A school‐based program to enhance self‐regulated and self‐motivated cycles of student learning

This article describes a training program, Self‐Regulation Empowerment Program (SREP), that school professionals can use to empower adolescent students to engage in more positive, self‐motivating cycles of learning. It is a two‐part approach whereby self‐regulated learning coaches (SRC) (a) use microanalytic assessment procedures to assess students' self‐regulation beliefs and study strategies and (b) train students to use these strategies in a cyclical, self‐regulation feedback loop. Ultimately, students learn how to set goals, select and monitor strategy effectiveness, make strategic attributions, and adjust their goals and strategies. The program was developed from social‐cognitive theory and research and integrates many of the essential features of the problem‐solving model. Interventions used in the SREP include graphing, cognitive modeling, cognitive coaching, and structured practice sessions. A case study is presented to illustrate procedures for implementing the program. Implications for school psychologists and teachers also are presented and discussed. © 2004 Wiley Periodicals, Inc. Psychol Schs 41: 537–550, 2004.     

Communication in performance‐based training and instruction: From design to practice

Communication is inextricably important to instructional design and performance‐based training. Promoting effective communication as an integral part of the performance support system improves professional instructional design functions and offers greater avenues for meaningful discourse among end users of the instruction. In this article, we highlight communication in performance training and instruction for meaningful learning and effective exchange of knowledge. Internal and external communications are discussed as a means to promoting successful relationships, commitment, and ownership.     

Performance‐based certification—How to design a valid,defensible, cost‐effective program

Initiating a certification program is the overall intent of this book. Judith Hale starts with the program's business case, navigating you through requirements, standards, and assessments, and concluding with implementation and tips for “going global.” The book provides you with a roadmap for starting and implementing a certification program. Whether you are a C‐level executive or a performance improvement practitioner, this book has the explanation and rationale for implementing a certification program, using examples demonstrating learning points clearly and effectively. Performance‐Based Certification‐How to Design a Valid, Defensible, Cost‐Effective Program (ISBN: 978‐1‐118‐02724‐0) is published by Pfeiffer.