Managing What We Can Measure: Quantifying the Susceptibility of Automated Scoring Systems to Gaming Behavior

As methods for automated scoring of constructed‐response items become more widely adopted in state assessments, and are used in more consequential operational configurations, it is critical that their susceptibility to gaming behavior be investigated and managed. This article provides a review of research relevant to how construct‐irrelevant response behavior may affect automated constructed‐response scoring, and aims to address a gap in that literature: the need to assess the degree of risk before operational launch. A general framework is proposed for evaluating susceptibility to gaming, and an initial empirical demonstration is presented using the open‐source short‐answer scoring engines from the Automated Student Assessment Prize (ASAP) Challenge.     

The Issue of Range Restriction in Bookmark Standard Setting

This article uses data from a large‐scale assessment program to illustrate the potential issue of range restriction with the Bookmark method in the context of trying to set cut scores to closely align with a set of college and career readiness benchmarks. Analyses indicated that range restriction issues existed across different response probability (RP) values and item response theory (IRT) models if one were to apply the Bookmark procedure using intact test forms. Results also suggested that range restriction may still be present if one had access to additional data from an item bank. This demonstration critically highlights challenges that may exist in some practical applications of the Bookmark method due items not being designed to cover the full range of examinee abilities.     

A Critical Look into the Beuk Standard-Setting Method

One commonly used compromise standard-setting method is the Beuk (1984) method. A key assumption of the Beuk method is that the emphasis given to the pass rate and the percent correct ratings should be proportional to the extent that the panelists agree on their ratings. However, whether the slope of Beuk line reflects the emphasis that panelists believe should be assigned to the pass rate and the percentage correct ratings has not be fully tested. In this article, I evaluate this critical assumption of the Beuk method by asking panelists to assign importance weights to their percentage correct and pass rate judgments. I show that in several cases that the emphasis suggested by the Beuk slope is noticeably different from what one would expect and is inconsistent with importance weight ratings. I also suggest two ways that the importance weights can be used to calculate alternate cut scores, and I show that one of the ways of calculating cut scores using the importance weights leads to larger potential differences in cut score estimates. I suggest that practitioners should consider collecting importance weights when the Beuk method is used for determining cut scores.     

Mapping migrant vernacular discourses: Mestiza consciousness,nomad thought,and Latina/o/x migrant movement politics in the United States

ABSTRACT

This essay reads the Latina/o/x migrant vernacular discourses that emerge out of pro-migrant activism. Anzaldúa’s notion of mestizaje – a logic of border consciousness – is put into conversation with Deleuze’s notion of nomad thought – a logic of movement – to inform a rhetorical strategy for reading the vernacular archive of social movement discourse. The “No Papers, No Fear” is one such social movement that demonstrate the logic of mestizaje/nomadism in their communication strategies. This study illuminates three tensions that define the ways Latina/o/x migrants in the US navigate the spaces of citizenship: tensions between movement/stasis, migrant identity/national identity, and fear/safety.     

The Potential Impact of COVID-19 on Student Learning and How Schools Can Respond

There is no denying the impact that the coronravirus disease (COVID-19) outbreak has had on many aspects of our lives. This article looks at the potential impact of COVID-19 on student learning as schools abruptly morphed into virtual learning environments using data from several instructional, practice, and assessment solutions offered by Renaissance. First, three hypothetical learning scenarios are considered using normative data from Star assessments to explore the potential impact on reading and math test performace. Next, data on Focus Skills are used to highlight which grades may have missed the most foundational math and reading content if instruction was stopped or reduced. Last, data from two of Renaissance's practice tools are used to evaluate whether students were practicing key skills following school closures. The article concludes that academic decline will likely occur but may be tempered by the increased use of practice tools; effects may look different for math and reading; and may impact grades and schools differently. As such, schools may need to leverage decision-making frameworks, such as the Multi-tiered Systems of Support/Response-to-Intervention (MTSS/RTI) framework, more than ever to identify needs and target instruction where it matters most when school begins in fall 2020.     

The cumulative and residual fatigue response associated with soccer-specific activity performed on different playing surfaces

ABSTRACT

This study aimed to assess the effect of playing surface (Natural [NT] and Artificial [AT] Turf) on the fatigue response to a soccer-specific exercise protocol (SSEP). Eighteen male soccer players completed the SSEP on NT and AT with pre-, post-, and 48 h post-assessments of eccentric knee flexor (eccKF) and concentric knee extensor peak torque (PT), peak countermovement (CMJ) and squat jump (SJ) height, and Nordic hamstring break angle. No significant main effects for surface or any surface and time interactions were observed for any of the outcome measures, except for eccKF PT recorded at 3.14 rad·s^-1, which was significantly lower 48 h post-trial in the AT condition (AT = 146.3 ± 20.4 Nm; NT = 158.8 ± 24.7 Nm). Main effects for time were observed between pre- and post-trial measures for eccKF PT at all angular velocities, Nordic break angle, CMJ and SJ height. Nordic break angle, and both CMJ and SJ height were significantly impaired 48 h post-trial when compared to pre-trial. The findings of the current study suggest surface dependent changes in eccKF PT which may have implications for recovery and subsequent performance after competition on AT.