The impact of high-stakes testing on student proficiency in low-stakes subjects: Evidence from Florida's elementary science exam

An important criticism of high-stakes testing policies – policies that reward or sanction schools based on their students’ performance on standardized tests – is that they provide schools with an incentive to focus on those subjects that play a role in the accountability system while decreasing attention to those subjects that are not part of the program. This paper utilizes a regression discontinuity design to evaluate the impact of Florida's high-stakes testing policy on student proficiency in the low-stakes subject of science. We confirm prior results that students in schools facing more immediate sanctions under the policy made substantial gains in the high-stakes subjects of math and reading. Contrary to the crowding-out hypothesis, we find that students in these schools made substantial achievement gains in the low-stakes subject of science as well.     

Taking and teaching the test are not the same: a case study of first-year teachers’ experiences in high-stakes contexts

Background: Policymakers’ use of high-stakes exams to improve students’ academic achievement affects teachers and their tenure in the field at all levels of schooling. Novice teachers now being inducted into the field have been educated almost exclusively in these high-stakes learning environments. Yet, how their familiarity with these contexts combined with their experiences in their own classrooms affect novices’ induction into the field of teaching has not been fully examined. Aim: This article presents findings from an investigation into the experiences of two first-year teachers who were educated and trained to be teachers in the same high-stakes education system in which they taught. It examines how these first-year teachers viewed policymakers’ reforms affecting their teaching and tenure in the field. Methods: This qualitative case study centers on the experiences of two first-year teachers working in the same high-stakes standards-based accountability teaching context in which they were educated – the case. This study provides insight into the issue of how novices’ familiarity with high-stakes reform combined with their experiences in their own classrooms impacts their conceptions of their teaching and their tenure in the field. Outcomes: The findings from this case study reveal how policymakers’ high-stakes reforms impacted the development of these novice teachers in significant ways. Not only did they have to learn how to teach as they taught, but they also had to ensure they were teaching all of their students to pass the high-stakes exams. Their varied experiences also demonstrate how these high-stakes exams can ‘test’ beginning teachers out of the classroom. Conclusions: These findings demonstrate that first-year teachers’ familiarity with policymakers’ high-stakes reforms is not enough to prepare them for the expectation that they immediately improve students’ academic achievement on high-stakes exams. Such findings not only challenge what it means to be a educator in these contexts, but they also shed light on how larger political and economic forces impact the teaching and tenure of novices. To support new teachers, teacher educators and mentors should rethink the education and induction processes while helping novices understand as well as prepare for the role context plays in their teaching and development as professionals.     

What is at stake without high-stakes exams? Students’ evaluation and admission to college at the time of COVID-19

The outbreak of COVID-19 in 2020 inhibited face-to-face education and constrained exam taking. In many countries worldwide, high-stakes exams happening at the end of the school year determine college admissions. This paper investigates the impact of using historical data of school and high-stakes exams results to train a model to predict high-stakes exams given the available data in the Spring. The most transparent and accurate model turns out to be a linear regression model with high school GPA as the main predictor. Further analysis of the predictions reflect how high-stakes exams relate to GPA in high school for different subgroups in the population. Predicted scores slightly advantage females and low SES individuals, who perform relatively worse in high-stakes exams than in high school. Our preferred model accounts for about 50% of the out-of-sample variation in the high-stakes exam. On average, the student rank using predicted scores differs from the actual rank by almost 17 percentiles. This suggests that either high-stakes exams capture individual skills that are not measured by high school grades or that high-stakes exams are a noisy measure of the same skill.     

The impact of high-stakes testing on curriculum and pedagogy: a teacher perspective from Australia

Debates continue about how high-stakes testing regimes influence schools at all levels: their impact on teaching practices, distribution of resources and curriculum provision, and whether they achieve the intended increases in student achievement in targeted areas. In 2008, the Australian government introduced a national testing scheme, the National Assessment Program – Literacy and Numeracy (NAPLAN), in which all Australian students in Years 3, 5, 7 and 9 are required to participate, and a national website, MySchool, was established in 2010 to publish the results of all schools. This paper reports on the first national study of educators’ views on the impact of NAPLAN on Australian schools and students. Over 8000 educators from all states and territories participated in the study, and the findings indicate that the testing regime is leading to a reduction in time spent on other curriculum areas and adjustment of pedagogical practice and curriculum content to mirror the tests. The findings suggest that the modification of teaching and curricular practices is in response to concerns regarding the use and reporting of NAPLAN data and the potential impact on schools. This confirms findings of researchers in other countries on the capacity of high-stakes regimes to distort teaching practices, constrain the curriculum and narrow students’ educational experiences.     

‘It was,perhaps, the most important one’ students’ perceptions of national tests in terms of test-taking motivation

The aim of this study was to examine ninth-grade students’ perception of a Swedish national test in science (high-stakes for the students) and a field trial (low-stakes for the students), in terms of test-taking motivation. The study is based on semi-structured interviews with 12 students. Overall, the students had a positive perception of the national test; they were motivated to do their best, believed they did well and found the test important and good. Several students were nervous before the test and some during the test whereas others felt okay, or even enjoyed taking the national test. The students talked about less effort and test anxiety in the field trial and perceived the test as less important compared to the national test. The interviews provided a rich understanding of the different aspects of test-taking motivation as described by the expectancy-value theory. Furthermore, they indicated how the test administration could be adjusted so as to achieve higher test validity.     

Keyboarding compared with handwriting on a high-stakes writing assessment: Student choice of composing medium, raters’ perceptions, and text quality

This article examines the influence of keyboarding versus handwriting in a high-stakes writing assessment. Conclusions are based on data collected from a pilot project to move Old Dominion University's Exit Exam of Writing Proficiency from a handwritten format into a dual-option format (i.e., the students may choose to handwrite or keyboard the exam). This test case is used to demonstrate the complex implications for a writing assessment tool when universities aim to accommodate students’ various methods of physically creating a text. Discussion includes speculation about the format of the writing component on the SAT. It is argued that by analyzing the differences in high-stakes exams when students keyboard and when they handwrite, researchers can better understand how the material conditions for composing influence students and raters. It is concluded that if handwriting and keyboarding represent even slightly different composing skills and cognitive processes, then high-stakes writing assessments should aim to accommodate students by providing methods of textual production with which students are most comfortable, even as this accommodation produces new challenges for the creators and managers of large-scale writing assessments.     

USING A STATE TEACHER CERTIFICATION TEST TO ASSESS AN INQUIRY-BASED SCIENCE EDUCATION PROGRAM

In this study, we looked at the impact of our specially designed inquiry-based science courses for pre-service elementary teachers on their science content knowledge as measured by a high-stakes state certification test for elementary education. We conducted a pre/post-analysis of the certification test scores of 1,003 pre-service teachers. Cohort 1 consisted of 424 students who took the test prior to implementation of our reformed science courses. Cohort 2 consisted of 579 students taking the test after complete implementation. We examined overall test scores, science subscores, total science credits, age, and science transfer credits. We found that the overall test score dropped significantly from cohort 1 to cohort 2, but the science subscores remained unchanged. Our results showed that students who took all 3 of the science content courses vs. none scored significantly higher on the science portion, non-traditional students (older than 25) scored similarly to traditional-aged students, and there was a negative correlation with the number of science courses transferred. Of importance is the fact that students who took the minimal number of science courses and are less interested in science passed the science portion at the same rate (over 90%) as science majors and minors. We conclude that our pre-service elementary teachers can and do learn science content using inquiry as recommended by the National Science Education Standards (NRC, 1996).     

Using high-stakes grades to incentivize learning

I use a natural experiment in Norwegian high school to investigate how high-stakes grades affect students’ investment in schooling. By exploiting variation across space and time I compare the performance of students taking the same exit exam in compulsory school, but where the test is high-stakes for only a subset of students. Using a staggered triple-difference framework, I find that exam grades increase in the high-stakes setting if students have a sufficient number of prospective high schools within traveling distance. Results from low-stakes ability assessments suggest actual learning — and not test-taking strategy — could largely explain the effect.     

Enhancing Inquiry, Understanding, and Achievement in an Astronomy Multimedia Learning Environment

As an example of design-based research, this study refined an assessment strategy for simultaneously enhancing inquiry-based learning and supporting achievement on conventional assessment measures. Astronomy Village ^® : Investigating the Universe? is a software program designed to engage secondary science students in authentic and inquiry-based learning over core topics in astronomy. The software was enhanced with a 20-hour curriculum and three levels of assessment to ensure successful inquiry experiences and high-stakes achievement. The first year implementation of Astronomy Village^® yielded significant gains on a curriculum-oriented exam but not a standards-oriented test, and provided useful design insights that were integrated into the second year implementations. Significant gains were obtained on the test during the second year as well. It is expected that many existing inquiry-oriented science curricula might be similarly enhanced, and is suggested that a large-scale effort to do so might have a lasting impact on science education.     

Using multiple methods to investigate eleven-year-olds’ experiences of preparing for a high-stakes public examination in Trinidad and Tobago

This study was designed to uncover and evaluate unintended and indirect consequences of using the Secondary Entrance Assessment (SEA) in Trinidad and Tobago for high-stakes selection and placement. A major argument is that the test-taker is central to consequences, both intended and unintended. Data were obtained from eleven-year-olds preparing and taking the examination using multiple qualitative methods, which privilege agency and personal and social consciousness. Methods capturing the test-takers’ experience included diaries, focus groups, drawings, photovoice, and photo-elicitation. The findings were summarised by a theory of action and reported in integrated themes. Test preparation was mostly drill and practice. Both test preparation and test-taking were emotion-eliciting events requiring significant investment in positive social relationships to buffer negative impact. The costs and impact of early high-stakes examinations in Trinidad and Tobago might be higher than anticipated.     

大学生入学前后素质的“典型相关分析”

针对学生的大学成绩和高考成绩及学生自身的努力程度的相关关系问题,收集泉州经贸学院07级会计专业的某20名学生的相关数据,采用典型相关分析的方法,通过SAS9.1进行了分析。结杲显示,高考成绩及自身努力程度都对大学成绩产生影响,尤其是学生的个人努力程度对大学成绩中的学习成绩产生非常重要的影响。     

A promising science and literacy instructional model with Hispanic fifth grade students

Abstract

This study evaluated the Science and Literacy Instructional Model aimed at helping primarily Hispanic bilingual/English Learners (ELs) and economically disadvantaged fifth grade students with science achievement as measured by high-stakes standardized science achievement scores. The model combined purposeful planning, innovative academic vocabulary instruction, and a Lesson Design Lab. Difference-in proportions tests were used to determine if students at two school campuses showed positive achievement gains on a state science test. This study found statistically significant results with medium to large effect sizes at both campuses. Findings contribute to much needed research, practice, and policy in the area of effective models to assist both teachers and students in an era of high-stakes testing.     

High stakes testing and teacher access to professional opportunities: lessons from Indonesia

Abstract

High-stakes testing regimes, in which schools are judged on their capacity to attain high student results in national tests, are becoming common in both developed and developing nations, including the United States, Britain and Australia. However, while there has been substantial investigation around the impact of high-stakes testing on curriculum and pedagogy, there has been very little research looking at the impact on teachers’ professional opportunities. The current project used a case study approach to examine the impact a high-stakes national testing programme had on teachers’ access to professional learning and their teaching allocations in four Indonesian public schools. It found that better qualified teachers were allocated to classes that would be sitting for the national examinations, and that these teachers were given much more access to professional learning opportunities than those teaching non-examined year levels. This in turn impacted negatively on the staff morale of less qualified teaching staff and potentially on their career trajectories. Findings suggest that school leaders should be wary of targeting better qualified and/or more experienced staff to year levels sitting for high-stakes tests, as this may lead to staff stratification within schools, limiting opportunities for staff to learn from one another and reducing the morale of less qualified and less experienced staff. They also add support to a substantial body of research that suggests policy-makers should be wary of the flow-on effects of using performance in high-stakes tests as the key means of judging school effectiveness.     

A Quantum Measurement Paradigm for Educational Predicates: Implications for validity in educational measurement

Abstract

The outcomes of educational assessments undoubtedly have real implications for students, teachers, schools and education in the widest sense. Assessment results are, for example, used to award qualifications that determine future educational or vocational pathways of students. The results obtained by students in assessments are also used to gauge individual teacher quality, to hold schools to account for the standards achieved by their students, and to compare international education systems. Given the current high-stakes nature of educational assessment, it is imperative that the measurement practices involved have stable philosophical foundations. However, this article casts doubt on the theoretical underpinnings of contemporary educational measurement models. Aspects of Wittgenstein’s later philosophy and Bohr’s philosophy of quantum theory are used to argue that a quantum theoretical rather than a Newtonian model is appropriate for educational measurement, and the associated implications for the concept of validity are elucidated. Whilst it is acknowledged that the transition to a quantum theoretical framework would not lead to the demise of educational assessment, it is argued that, where practical, current high-stakes assessments should be reformed to become as ‘low-stakes’ as possible. This article also undermines some of the pro high-stakes testing rhetoric that has a tendency to afflict education.     

The impact of the scoring system of a large-scale standardized EFL writing assessment on its score variability and reliability: Implications for assessment policy makers

Using generalizability (G-) theory and rater interviews as research methods, this study examined the impact of the current scoring system of the CET-4 (College English Test Band 4, a high-stakes national standardized EFL assessment in China) writing on its score variability and reliability. One hundred and twenty CET-4 essays written by 60 non-English major undergraduate students at one Chinese university were scored holistically by 35 experienced CET-4 raters using the authentic CET-4 scoring rubric. Ten purposively selected raters were further interviewed for their views on how the current scoring system could impact its score variability and reliability. The G-theory results indicated that the current single-task and single-rater holistic scoring system would not be able to yield acceptable generalizability and dependability coefficients. The rater interview results supported the quantitative findings. Important implications for the CET-4 writing assessment policy in China are discussed.     

GI Forum v. Texas Education Agency: Observations for States

Seven conclusions for professionals who administer state assessment programs are drawn from the GI Forum v. Texas Education Agency ruling: (a) the American Educational Research Association, American Psychological Association, and National Council on Measurement in Education (1999) standards are appropriate to use; (b) items showing different p values for subgroups may be used if they are selected as adequate for sound educational reasons; (c) a cut score setting process should be educationally justified; (d) a high-stakes testing program can appropriately address unfair access to education; (e) multiple opportunities to pass satisfies the standard that a single test score should not be the sole basis for a high-stakes decision; (f) a conjunctive decision-making model can appropriately motivate both students and schools; and (g) an 80% pass rate criterion applied to eventual, as opposed to initial, success rates for subgroups is a reasonable threshold for adverse impact. Caution is recommended because circumstances in other states may not parallel those in Texas in important ways.     

Linking rights,needs, and fairness in high-stakes assessments and examinations

This paper presents a novel explanation for the continued absence of a children’s rights strategy within high-stakes educational assessment with reference to the competing purposes of high-stakes assessments and group-based constructions of fairness in assessment. We provide an original critique of group-based perspectives on the validity of assessment accommodations which supports an individual perspective on fair educational assessment. From this, the (almost forgotten) concept of ‘student assessment needs’ is (re-)introduced as a central axiom, to be constructed through feedback from, and dialogue with, students about their experience of high-stakes assessments, giving primacy to their purpose as an attainment demonstration opportunity for the student. To promote a new movement towards student participation in educational assessment processes and reforms, we propose ‘rights respecting assessment’, complementary to UNICEF’s ‘Rights Respecting Schools’ initiative, in which regular system-wide student representation would contribute to the development of high-stakes examination systems. In recognition of this aspiration, this paper is co-authored with a recent school-leaving age examination candidate.     

No Quick Fixes: Problems in the Development of A-level Students’ Writing (and Learning)

Over the past 15 years, there has been a range of standards-driven educational interventions in England: focused particularly on students’ writing, they have been targeted at particular students, short-term and based on the assumption that identifiable, quantifiable inputs would produce pre-identified, measurable outputs. This article explores one such intervention, aimed at students in Year 13 who were seen as having difficulty with academic writing. It looks closely at the work of one student, and raises questions about the effects of high-stakes testing upon pedagogy in schools and the damaging impact that this has upon student learning.     

The relationship between GEAR UP program involvement and Latina/o students’ performance on high-stakes tests

To assess the impact of a college preparation program, district-wide changes in high-stakes test pass rates are examined, followed by an examination of specific program elements that are associated with students’ score changes. Changes in pass rates for districts with and without services are compared for the Latina/o and overall student populations. This study also examines the relationship between hours of participation in various elements of the college preparation program and changes in test scores of low-income Latina/o students. Results indicate that key program elements related to improvements in test scores include participation in mentoring and summer programs.