Managing What We Can Measure: Quantifying the Susceptibility of Automated Scoring Systems to Gaming Behavior

As methods for automated scoring of constructed‐response items become more widely adopted in state assessments, and are used in more consequential operational configurations, it is critical that their susceptibility to gaming behavior be investigated and managed. This article provides a review of research relevant to how construct‐irrelevant response behavior may affect automated constructed‐response scoring, and aims to address a gap in that literature: the need to assess the degree of risk before operational launch. A general framework is proposed for evaluating susceptibility to gaming, and an initial empirical demonstration is presented using the open‐source short‐answer scoring engines from the Automated Student Assessment Prize (ASAP) Challenge.     

Meeting players where they are: Digital games and learning ecologies

This paper explores the relationship between digital games and learning along the continuum from formal to informal learning. Games are an increasingly important component of our learning ecologies but their potential can prove difficult to fully harness due to the downsides of gaming in informal contexts and to the limitations games can present when adopted in formal learning contexts. The aim of this paper is to shed light on the strengths and weaknesses of games for learning, and to gather stakeholder recommendations for the main actors in the field. The adopted research approach is qualitative and participatory, involving players, teachers, parents, researchers and developers. Data from 25 interviews and 11 focus groups yielded insights into the educational, psychological and ethical aspects of play. Results suggest that gameplay influences individuals in a very personal manner, and that its effects cover a wide range of skills, extending to identity development. Hence, to harness this potential, all actors in the ecology need strong awareness of both the affordances and the risks of games, and deep respect for individuals’ LEs. The study has implications for a wide range of actors, including educators, players, researchers, parents, game developers and policymakers.