Causal Inference and COVID: Contrasting Methods for Evaluating Pandemic Impacts Using State Assessments

In the spring of 2021, just 1 year after schools were forced to close for COVID-19, state assessments were administered at great expense to provide data about impacts of the pandemic on student learning and to help target resources where they were most needed. Using state assessment data from Colorado, this article describes the biggest threats to making valid inferences about student learning to study pandemic impacts using state assessment data: measurement artifacts affecting the comparability of scores, secular trends, and changes in the tested population. The article compares three statistical approaches (the Fair Trend, baseline student growth percentiles, and multiple regression with demographic covariates) that can support more valid inferences about student learning during the pandemic and in other scenarios in which the tested population changes over time. All three approaches lead to similar inferences about statewide student performance but can lead to very different inferences about student subgroups. Results show that controlling statistically for prepandemic demographic differences can reverse the conclusions about groups most affected by the pandemic and decisions about prioritizing resources.     

“Acquired” Equals Addition? Associating Verbs with Arithmetic Operations Impacts Word Problem Performance

Successful word problem performance often requires understanding the linguistic relations between characters and objects. However, the keyword method promotes associating specific words with mathematical operations while neglecting the situational context. Research has thoroughly investigated the detrimental effects of individuals associating relational terminology (e.g., “more”) with mathematical operations (e.g., “addition”). The current study expands upon this line of research by examining whether undergraduate students associate verbs with mathematical operations and if verbal associations affect word problem performance. Similar to relational terminology, the participants associated verbs with operations, which significantly impacted performance. The educational implications are discussed.