University Admissions and the Prediction of Degree Performance: an Analysis in the Light of Changes to the English Schools’ Examination System

The United Kingdom's Department for Education has recently changed the nature of the AS‐level examinations normally taken by students aspiring to enter higher education degree courses one year into their post‐compulsory education. In the face of protests from universities and other institutions that this would both harm students’ progression towards the A‐level qualifications, on which entry to English universities is normally based, and make it difficult for universities determining which of their applicants were best placed to benefit from their degree courses, the Department conducted research which it claimed showed that degree outcome could be predicted as well from the results of GCSE examinations taken one year before AS‐levels as from AS‐levels themselves. This paper critiques those analyses and their conclusions showing, through a re‐analysis and extended interpretation of the Department's data, that AS‐levels provide a more reliable predictor of degree performance in 2011 than GCSEs and that many students who performed better at AS‐level than at GCSE gained a place, and performed well, at a university with high entrance standards.     

Managing What We Can Measure: Quantifying the Susceptibility of Automated Scoring Systems to Gaming Behavior

As methods for automated scoring of constructed‐response items become more widely adopted in state assessments, and are used in more consequential operational configurations, it is critical that their susceptibility to gaming behavior be investigated and managed. This article provides a review of research relevant to how construct‐irrelevant response behavior may affect automated constructed‐response scoring, and aims to address a gap in that literature: the need to assess the degree of risk before operational launch. A general framework is proposed for evaluating susceptibility to gaming, and an initial empirical demonstration is presented using the open‐source short‐answer scoring engines from the Automated Student Assessment Prize (ASAP) Challenge.