Using self- and peer-assessments for summative purposes: analysing the relative validity of the AASL (Authentic Assessment for Sustainable Learning) model

The purpose of this paper is to provide a proof of concept of a collaborative peer-, self- and lecturer assessment processes. The research presented here is part of an ongoing study on self- and peer assessments in higher education. The authentic assessment for sustainable learning (AASL) model is evaluated in terms of the correlations between sets of marks. The article provides an explanation of the assessment process, and analyses sets of marks as a means of justifying the validity of the process. The results suggest that students, even those with no prior experience in peer- or self-evaluation, in their first year of tertiary study, under the right conditions, are able to accurately judge their own work and make reasonably accurate judgements of the work of their peers. While previous studies have expounded the benefits of self- and peer assessments in tertiary study, undertaking a prescribed process, such as AASL, has a further implication in allowing others to replicate the process with reasonable assuredness of the validity of the process across various fields of study.     

Using rasch measurement to score,evaluate, and improve examinations in an anatomy course

Any examination that involves moderate to high stakes implications for examinees should be psychometrically sound and legally defensible. Currently, there are two broad and competing families of test theories that are used to score examination data. The majority of instructors outside the high‐stakes testing arena rely on classical test theory (CTT) methods. However, advances in item response theory software have made the application of these techniques much more accessible to classroom instructors. The purpose of this research is to analyze a common medical school anatomy examination using both the traditional CTT scoring method and a Rasch measurement scoring method to determine which technique provides more robust findings, and which set of psychometric indicators will be more meaningful and useful for anatomists looking to improve the psychometric quality and functioning of their examinations. Results produced by the more robust and meaningful methodology will undergo a rigorous psychometric validation process to evaluate construct validity. Implications of these techniques and additional possibilities for advanced applications are also discussed. Anat Sci Educ 7: 450–460. © 2014 American Association of Anatomists.     

Validity of High‐Stakes Assessment: Are Students Engaged in Complex Thinking?

The validity of high-stakes assessments and accountability systems is discussed in relation to the requirements of No Child Left Behind (NCLB). The extent to which content standards and assessments are cognitively rich, the challenges in setting performance standards, and the impact of high-stakes assessments on instruction and student learning are addressed. The article argues for quality content standards, cognitively rich assessments, and a cohesive, balanced assessment system.     

Using a discussion about scientific controversy to teach central concepts in experimental design

Students may need explicit training in informal statistical reasoning in order to design experiments or use formal statistical tests effectively. By using scientific scandals and media misinterpretation, we can explore the need for good experimental design in an informal way. This article describes the use of a paper that reviews the measles mumps rubella vaccine and autism controversy in the UK to illustrate a number of threshold concepts underlying good study design and interpretation of scientific evidence. These include the necessity of sufficient sample size, representative and random sampling, appropriate controls and inferring causation.     

Damaging confusions in England's KS2 reading tests: a response to Anne Kispal

This article is written in response to the article published in issue 39.3 of this journal, in November 2005, on the nature of the Key Stage 2 National Curriculum reading tests: ‘Examining England's National Curriculum assessments: an analysis of the KS2 reading test questions’ by Anne Kispal of the National Foundation for Educational Research. It argues that, far from providing a valid and rewarding assessment experience for pupils as Kispal suggests, the primary English tests at the end of KS2 are invalid as a measuring instrument and are having a damaging effect on pedagogy. The tests and the information on them provided by the Qualifications and Curriculum Agency are based on a misleading unidimensional conception of reading literacy attainment. Because the test assessment simply adds together marks achieved for very different cognitive skills, it propagates a dysfunctional model of literacy pedagogy that conflates and confuses two separate developmental trajectories – word reading and text comprehension. The article goes on to argue that the unidimensionality of the national tests and their pedagogic apparatus has constricted the primary English curriculum in ways that are damaging for young pupils and for the national need for creativity and enterprise.     

浅谈如何提高行政管理的有效性

行政管理是一项非常重要的现代管理手段。行政管理和行政管理的有效性,重点体现在效率、科学、效益和效果,基本是扎根在创新和改革上面,重点落在实效上面。行政管理的有效性,所包含的层面是非常多的,每一个层面都涉及到很多方面的内容。行政管理的有效性也是我国在整个改革的过程当中逐渐引起国人注意的一个热点,笔者认为,行政管理的有效性应该是整个改革发展总体过程中一个至关重要的环节。本文主要对当前我国行政管理有效性的重要意义进行分析,并提出了一些建设性的建议。     

How to improve the design of experimental studies in computing education: Evidence from the international assessments

Education and Information Technologies - Cluster randomized trials are frequently used in educational research for methodological reasons. This study aims to improve the efficiency of cluster...     

如何解答普通物理选择题

准确掌握物理学的基本理论和基本方法是正确解答选择题的关键,对定理、定律的熟练应用是迅速解题的保证。结合实例介绍了解答单选题的一些技巧:直接法、间接法、筛选法、作图法等。     

From course assessment to redesign: a hybrid-vehicle course as a case illustration

Assessment has become a central aspect of engineering education for evaluating student learning, attaining accreditation, and ensuring accountability. However, the final step of the assessment process, which requires assessment results be used to redesign courses and programmes, is appreciably underdeveloped in the literature. As such, this work suggests a process, based on the engineering problem-solving method, to analyse and act on problems and successes identified in the assessment results. The process is illustrated through an application to Colorado State University's new programme for Hybrid-Electric Vehicle Engineering, for which the redesign process was originally created. Readers will benefit by simplifying and systematising the essential aspect of the assessment process – application to course redesign – for use in both research and practice applications.     

Using bilingual students to link and evaluate different language versions of an exam

Many researchers and the International Test Commission＇s （Hambleton, 2005） caution against treating scores from different language versions of a test as equivalent, without conducting empirical research to verify such equivalence. In this study, we evaluated the equivalence of English and Malay versions of a 9^th-grade math test administered in Malaysia by conducting several statistical analyses. All analyses were conducted on data from a large sample of English-Malay bilingual students who took both versions of the exam. First, we conducted two equating analyses---one based on classical test theory and another based on item response theory （IRT）. Then differential item functioning analyses （DIF） were performed to see if any items functioned differentially across their English and Malay versions. The DIF results flagged 7 items for statistically significant DIF, but only one had a non-negligible effect size. We then conducted another equating analysis dropping the DIF items. The equating results suggested an adjustment of 1 or 2 points, depending on the mathematics achievement levels. The results indicate that bilingual examinees can be useful for evaluating different language versions of a test and adjusting for differences in difficulty across test forms due to translation.     

Assisting transition to university: using assessment as a formative learning tool

This study explores the effectiveness of an intervention involving formative assessment in a first‐year core business subject. Students were invited to receive feedback on a draft of their first written assessment during the early weeks of the semester. Consideration is given to the economic and ethical issues raised by the intervention. A multi‐method approach of qualitative and quantitative data collection and analysis is used. The research finds that the intervention facilitates significantly higher marks in assessments and grades, while assisting student learning overall. Findings are reinforced by comparison with a subject where the intervention was not offered.     

Multiple-Choice versus Open-Ended Questions in Advanced Clinical Neuroanatomy: Using a National Neuroanatomy Assessment to Investigate Variability in Performance Using Different Question Types

Methods of assessment in anatomy vary across medical schools in the United Kingdom (UK) and beyond; common methods include written, spotter, and oral assessment. However, there is limited research evaluating these methods in regards to student performance and perception. The National Undergraduate Neuroanatomy Competition (NUNC) is held annually for medical students throughout the UK. Prior to 2017, the competition asked open-ended questions (OEQ) in the anatomy spotter examination, and in subsequent years also asked single best answer (SBA) questions. The aim of this study is to assess medical students’ performance on, and perception of, SBA and OEQ methods of assessment in a spotter style anatomy examination. Student examination performance was compared between OEQ (2013–2016) and SBA (2017–2020) for overall score and each neuroanatomical subtopic. Additionally, a questionnaire explored students’ perceptions of SBAs. A total of 631 students attended the NUNC in the studied period. The average mark was significantly higher in SBAs compared to OEQs (60.6% vs. 43.1%, P < 0.0001)—this was true for all neuroanatomical subtopics except the cerebellum. Students felt that they performed better on SBA than OEQs, and diencephalon was felt to be the most difficult neuroanatomical subtopic (n = 38, 34.8%). Students perceived SBA questions to be easier than OEQs and performed significantly better on them in a neuroanatomical spotter examination. Further work is needed to ascertain whether this result is replicable throughout anatomy education.     

Assessment terms half a century in the making and unmaking: from conceptual ingenuity to definitional anarchy

As the parameters of the field of educational assessment have extended past testing into learning, assessment concepts have evolved and become ever more nuanced. It is frequently lamented in the English language literature that there is insufficient conformity and clarity in the way they are defined and used. This paper offers a survey of the problem over the past five decades and scrutinises conceptualisations of a number of key assessment terms. Additionally, it argues that some of these may not, or may no longer, be necessary, and recommends the phrase ‘evaluation for learning’ as the most suitable term for embodying the spirit of using testing for improving learning and teaching. It closes by offering suggestions for tackling the problem.     

How to improve argumentation comprehension in university students: experimental test of a training approach

Instructional Science - The ability to comprehend informal arguments is essential for scientific literacy but students often lack structural knowledge about these arguments, especially when the...     

Learning with their peers: Using a virtual learning community to improve an in-service Biology teacher education program in Brazil

This study investigated the use of a virtual learning community (VLC-Bio) combined to an online teachers' professional development program. VLC-Bio enabled the sharing of biological knowledge, teaching methods and didactic resources. Although they presented a limited initial profile of internet use directed to socialization, the results indicated that participation in the VLC-Bio focused internet use for teaching and learning purposes. The VLC-Bio offered opportunities to develop the ability to learn from their peers about how to deal with matters of difficult approach in everyday school life, as well as of sharing resources for Biology education that are frequently lacking.     

Standards-based performance assessment for the evaluation of student teachers: a consequential validity study

The instrument Samples of Teaching Performance (STP) was developed to assess student teachers' capacity to plan, deliver and evaluate a unit of instruction. The current study reports consequential validity data collected from supervisors (n?=?20) and student teachers (n?=?62) from three elementary and five secondary teacher preparation programs in Chile that participated in the field-testing of the STP. Student teachers described how this assessment had honed their sense of professionalism and promoted learning of the skills assessed. Supervisors reported enlarging the topics discussed with student teachers and making some changes to the supervisory process. These findings are complemented by an analysis of the STP scores obtained by 24 student teachers, which showed better development of instructional skills when compared to pedagogical reasoning and reflection. These results raise questions about the structure of student teaching to support the implementation of standards-based assessments that entail tasks at different levels of cognitive complexity.     

从内因着手提升课外阅读有效性策略分析

影响小学生课外阅读有效性的因素有很多,内在因素至关重要,课外阅读要想保质保量,还需要从内部去挖掘影响其有效性的因子。本文从阅读动机、阅读主体认知差异、阅读过程中的体验及阅读目的入手,探讨分析影响小学生课外阅读的内部因子并提出相应策略。