Steps Toward More Effective Implementation of the Standards for Educational and Psychological Testing

The goal of the Standards for Educational and Psychological Testing is to improve testing practices, but their impact on practice appears spotty. Self-regulation clearly fails in some instances. The establishment of an external agency to oversee testing practices and adherence to the Standards would face substantial hurdles, and the ambiguity of many of the Standards would hobble such an organization if one were created. Many of the Standards are general statements of principle, and past controversies make clear that we in the field often disagree about the reasons for them, their applicability to specific cases, and their practical meaning in specific contexts. This paper argues that the field should follow two approaches to lessen this ambiguity. First, using journals, conferences, and other vehicles, we should foster more frequent and more protracted debate about the practical meaning of key Standards, such as 13.6 and 13.7, which mandate that a decision that will have a "major impact" on a student should not be based on a single score. Second, future revisions of the Standards should use concrete examples of testing practices to clarify the meaning of the Standards, much as the legal system uses case law to clarify the meaning of the general principles embodied in statutes .     

Validity of High‐Stakes Assessment: Are Students Engaged in Complex Thinking?

The validity of high-stakes assessments and accountability systems is discussed in relation to the requirements of No Child Left Behind (NCLB). The extent to which content standards and assessments are cognitively rich, the challenges in setting performance standards, and the impact of high-stakes assessments on instruction and student learning are addressed. The article argues for quality content standards, cognitively rich assessments, and a cohesive, balanced assessment system.     

Funding,reputation and targets: the discursive logics of high-stakes testing

This paper provides insights into teacher and school-based administrators’ responses to policy demands for improved outcomes on high-stakes, standardised literacy and numeracy tests in Australia. Specifically, the research reveals the effects of the National Assessment Program – Literacy and Numeracy (NAPLAN), and associated policies, in the state of Queensland. Drawing suggestively across Michel Foucault’s notions of disciplinary power and subjectivity, and Pierre Bourdieu’s concept of social fields, the research utilises interviews with teachers and school-based administrators to reveal how high-stakes, standardised testing practices served to discursively constitute performative teacher subjectivities around issues of funding, teacher and school reputation and target-setting within what is described as the ‘field of schooling practices’. The paper argues that the contestation evident within this field is also reflective and constitutive of more educative schooling discourses and practices, even as performative logics dominate.     

Construct-Irrelevant Variance in High-Stakes Testing

There are many threats to validity in high-stakes achievement testing. One major threat is construct-irrelevant variance (CIV). This article defines CIV in the context of the contemporary, unitary view of validity and presents logical arguments, hypotheses, and documentation for a variety of CIV sources that commonly threaten interpretations of test scores. A more thorough study of CIV is recommended.     

Using multiple methods to investigate eleven-year-olds’ experiences of preparing for a high-stakes public examination in Trinidad and Tobago

This study was designed to uncover and evaluate unintended and indirect consequences of using the Secondary Entrance Assessment (SEA) in Trinidad and Tobago for high-stakes selection and placement. A major argument is that the test-taker is central to consequences, both intended and unintended. Data were obtained from eleven-year-olds preparing and taking the examination using multiple qualitative methods, which privilege agency and personal and social consciousness. Methods capturing the test-takers’ experience included diaries, focus groups, drawings, photovoice, and photo-elicitation. The findings were summarised by a theory of action and reported in integrated themes. Test preparation was mostly drill and practice. Both test preparation and test-taking were emotion-eliciting events requiring significant investment in positive social relationships to buffer negative impact. The costs and impact of early high-stakes examinations in Trinidad and Tobago might be higher than anticipated.     

Assessing levels of student understanding in pre-service teachers using a two-cycle SOLO model

This paper reports the findings of an exploratory study concerning the development of higher-order conceptual understanding of Teaching Games for Understanding (TGfU) among physical education pre-service teachers. The development of such understandings has been seen as problematic among pre-service teachers (Randall, 2003 Randall, L. 2003. Preservice teachers' understanding of teaching games for understanding approach to content delivery. Avante, 9(1): 49–61.  [Google Scholar]). An analysis of the responses given in a high- stakes examination of 165 pre-service physical education teachers was used to investigate varying levels of understanding in TGfU pedagogy. This paper reports specific examples of student responses in a written high-stakes examination and justifies how a two-cycle structure of the observed learning outcome (SOLO) model can discriminate between the demonstrations of surface and deep conceptual understandings.     

Inquiry-based learning in China: Do teachers practice what they preach, and why?

China is undergoing an education reform that calls for a change from a rigid, fixed curriculum and didactic pedagogy to a more flexible, school-based curriculum and more inquiry-based pedagogy. This study investigated the extent to which Chinese middle and high school teachers (a) endorse an inquiry-based approach and underlying learning principles, (b) practice this mode of teaching, and (c) believe that the approach is practically viable in the current educational contexts in China. A structured survey was developed to solicit Chinese teachers’ responses to the above three issues. A total of 582 valid responses were collected, representing middle and high schools in different geographic locations. The results show that Chinese teachers are receptive to inquiry-based pedagogy but find practical constraints in fully implementing it. Several cultural and pragmatic reasons are explored. Policy implications are discussed with respect to teacher education/development, capacity building for the new pedagogy, and teaching/evaluation alignment. Finally cultural issues are discussed regarding using inquiry-based learning to enhance critical thinking and nurture independent thinkers.     

The meaning of teacher education in an exam-oriented education system: lessons from novice secondary teachers in Korea

This study investigated four novice secondary teachers’ experience and perceptions of teacher education in relation to their current work experience in a high-stakes testing context. The novice teachers commonly indicated that their preparation, which had focused on content expertise, turned out to have little significance in schools, as they mainly executed teaching to the test. Instead, their role as homeroom teachers, which was concerned with caring and supporting students, was found to have much more significance. Accordingly, they indicated that teacher education must more strongly emphasise preparing teachers for that role, which requires them to become mature, considerate, and autonomous educators. Based on this finding, this study suggests the need for a clearer conception of and emphasis on the subjectification function of teacher education that is grounded in the consideration of the fundamental vision, purpose, and meaning of teacher education in a society.     

The ethics of competition: accountability policy enactment in Chilean schools’ everyday life

ABSTRACT

Public policies have a moral order, an ethical horizon. They offer a vocabulary of imagined micro-policies. Using the case of Chile, this paper examines the ways in which accountability policies are reworked within schools and how they affect actors’ subjectivities. It adds new findings to the existing body of research on school accountability policies, offering in-depth evidence based on the case of Chile, which has a high-stakes testing model and a widespread competitive voucher system. The research is based on case studies of ten public and private subsidised schools, framed by a sociological perspective of policy enactment theory. The research findings show the ways that accountability policies are recreated, expanded, and intensified at the local level, permeating an ethic of competition. The analysis focuses on three qualitative trends: school actors’ sense-making of test scores and labels; zones of safety and risk for teachers under an accountability regime; and the emergence of a sticky web of persuasion, surveillance, and coercion among school members in order to improve performance. The practices examined are not understood as ‘secondary effects’ or an ‘implementation problem’, as if they occur unconnected from the policy rationale. The outcomes are consistent with the policy itself in interaction with school life.     

Seeking Redemption for Our Psychometric Sins

Many American children are currently receiving a reduced quality education because of the increasingly widespread misuse of educational tests. Employing a religious metaphor, the author argues that members of the educational measurement community are culpable, at least in part, for this calamity. During recent decades, our nation's assessment personnel have failed to speak out vigorously against the increasingly prevalent improper use of traditionally constructed achievement tests to appraise school quality. This absence of action, it is claimed, constitutes a nontrivial sin of omission. To secure absolution for that sin, it is contended that measurement specialists must promote widespread assessment literacy.