Using standards to evaluate accountability policy in context: School administrator and teacher perceptions of a teacher evaluation system

Accountability policy in the United States requiring school districts to design and implement state policy-directed teacher evaluation systems that utilize multiple measures, most often observations of professional practice and student achievement data, has been reflective of shifting international public discourse surrounding educational accountability, specifically related to teacher quality. Although the most recent wave of policy change provides states some flexibility in determining how teachers are evaluated, many are still mired in debate on how to improve teacher professional practice and increase student learning. This study examined the perceptions of school administrators and teachers in a large, suburban, public school district regarding their district’s new teacher evaluation system. Understanding the convergent and divergent views of participant groups regarding how (i.e., the standards by which) they defined and measured the effectiveness of the evaluation system is necessary to not only better understand their lived experiences but also to potentially inform policy change.     

An experimental evaluation of three teacher quality measures: Value-added,classroom observations,and student surveys

Nearly every state evaluates teacher performance using multiple measures, but evidence has largely shown that only one such measure—teachers’ effects on student achievement (i.e., value-added)—captures teachers’ causal effects. We conducted a random assignment experiment in 66 fourth- and fifth-grade mathematics classrooms to evaluate the predictive validity of three measures of teacher performance: value-added, classroom observations, and student surveys. Combining our results with those from two previous random assignment experiments, we provide additional experimental evidence that value-added measures are unbiased predictors of teacher performance. Though results for the other two measures are less precise, we find that classroom observation scores are predictive of teachers’ performance after random assignment while student surveys are not. These results thus lend support to teacher evaluation systems that use value-added and classroom observations, but suggest practitioners should proceed with caution when considering student survey measures for teacher evaluation.     

Predicting High School Graduation and College Enrollment: Comparing Early Warning Indicator Data and Teacher Intuition

Each year, more districts implement early warning systems (EWS). These EWS predict negative student outcomes, such as dropping out, before they occur. Predictions are then used to match at-risk students to appropriate supports and interventions. Research suggests that these systems are useful in ensuring educators respond to student needs early, generating conversation around specific students at risk of dropping out. However, no research considers what new information teachers gain from having a specific prediction for a student. This article bridges this gap by comparing teacher and EWS predictions of whether students will complete high school and enroll in college. Further, it assesses whether accuracy in teacher judgment stems from additional information not in models—especially related to academic tenacity—and biases like self-fulfilling prophecies. Generally, EWS can provide benefits both as organizational tools and by increasing the precision with which students are identified for supports and interventions.     

Alignment of Human Resource Practices and Teacher Performance Competency

In this article, we argue that human resource (HR) management practices are important components of strategies for improving student achievement in an accountability environment. We present a framework illustrating the alignment of educational HR management practices to a teacher performance competency model, which in turn is aligned with student achievement goals. We identify and illustrate the various HR practices that could be aligned to the performance competency model and to each other. These HR practices include recruitment, selection, induction, mentoring, professional development, compensation, performance management, and instructional leadership. We then describe HR practices in 2 districts where empirical links between teacher competency and student achievement were shown (Cincinnati and Washoe County) and evaluate how much alignment was in place. We discuss the importance of HR alignment analysis for diagnosing districts' teacher quality improvement efforts, and we present suggestions for future research on the strategic use of HR management in K-12 educational organizations.     

Developing High-Quality Teachers: teacher evaluation for improvement

As countries aim to ensure high achievement for all students, improving and sustaining the quality of the teacher workforce is a vital policy priority. Several studies have found that well-designed teacher evaluation systems, aligned with professional learning and development, can contribute to improvements in the quality of teaching and raise student achievement. This article provides an overview of research on teacher evaluation for improvement and suggests directions for policy and research to strengthen systems.     

How schools train principals to use new teacher evaluation systems: A comparative case study of charter school and traditional public school principals

Over the past decade U.S. policymakers, practitioners, and researchers have sought to examine if changing teacher evaluation policies and systems have resulted in changes in identifying quality teachers and/or increased student achievement. This research generally shows most states have experienced little change in how teachers are rated. Researchers are now exploring why, in many cases, teacher evaluation reforms have failed to produce the desired systematic changes of better identifying quality teachers and better distinguishing teacher performance. Embedded within this line of inquiry is how principals (and other evaluators) are trained to use new teacher evaluation systems. This comparative case study observed six principals (three charter school principals and three traditional public school principals) in the U.S. state of Michigan as they learned and enacted a new teacher evaluation system. Additionally, all principals were interviewed three times throughout the school year, in an effort to examine how their initial teacher evaluation training impacted their evaluation of teachers. The research questions that guided this work were: (1) how are principals initially trained when their school adopts a new teacher evaluation system?; (2) in what ways does the training received by charter school principals compare to that of traditional public school principals?; and (3) how does initial training impact how principals evaluate teachers? Results indicate principals are trained to navigate the logistics of new teacher evaluation systems, but are not trained to evaluate teacher performance. Implications for policy and practice are discussed.     

The administrative control of principals in effective school districts: The supervision and evaluation functions

In this article we examine the nature of administrative control of school principals in 12 especially effective districts in California. We combined two lines of inquiry, knowledge regarding control in organizations and information from the teacher effects and school effectiveness research, to examine types and patterns of control mechanisms in these districts. Although the work reported herein is exploratory in nature, we see evidence that, contrary to the norm, supervision and evaluation of site level administrators were rational and meaningful processes in these effective districts. We report further that the supervision and evaluation functions appeared to form important linkage mechanisms between schools and districts. In addition, we note that supervision and evaluation provide a strong base for the development of other important linkage functions, especially goal setting. Finally, we report that the superintendents often appear to be the key figures in the supervision and evaluation functions in these effective school districts.     

Examining the Relationship Between Teacher Evaluation and Student Assessment Results in Washoe County

In this article, we describe findings from an analysis of the relationship between scores on a standards-based teacher evaluation system modeled on the Framework for Teaching (Danielson, 1996) and student achievement measures in a large Western school district. We apply multilevel statistical modeling to study the relationship between the evaluation scores and state and district tests of reading, mathematics, and a composite measure of reading and mathematics. Using a value-added framework, the teacher evaluation scores were included at the 2nd level, or teacher level, of the model when other student and teacher-level characteristics were controlled. This study provided some initial evidence of a positive association between teacher performance, as measured by the evaluation system, and student achievement. The coefficients representing the effects of teacher performance on student achievement were positive and were statistically significant in 4 of 9 grade-test combinations studied.     

Revolution in Mental Retardation Research

In this article, I present the results of an analysis of the relationship between teacher evaluation scores and student achievement on district and state tests in reading, mathematics, and science in a large Midwestern U.S. school district. Within a value-added framework, I correlated the difference between predicted and actual student achievement in science, mathematics, and reading for students in Grades 3 through 8 with teacher evaluation ratings. Small to moderate positive correlationships were found for most grades in each subject tested. When these correlationships were combined across grades within subjects, the average correlationships were. 27 for science,. 32 for reading, and. 43 for mathematics. These results show that scores from a rigorous teacher evaluation system can be substantially related to student achievement and provide criterion-related validity evidence for the use of the performance evaluation scores as the basis for a performance-based pay system or other decisions with consequences for teachers.     

The value and feasibility of evaluation research on teacher development: contrasting experiences in Sri Lanka and Mexico

This article discusses the value and feasibility of carrying out evaluation research on teacher development and uses as points of reference the author's experiences in two countries, Sri Lanka and Mexico. In Sri Lanka, an evaluation study was designed to understand the effectiveness and costs of teacher development at the elementary level linking teacher preparation with classroom practice and student achievement. The study also evaluated costs and analyzed the possible impact of the results for future policy. The study in Mexico illustrates the challenges of doing evaluation research in an environment dominated by a central state and teacher union politics, and where systemic empirical research on teacher development has been rare. It constituted an initial attempt at looking at the content and the anticipated effects across different approaches to teacher development in Mexico. New calls for greater accountability and better understanding of the reach and limitations of general education worldwide are prompting systems to examine teacher development program effectiveness. In this analytical article, the author discusses strategies and possibilities in the emerging field of teacher development program evaluation.     

The elusiveness of teacher quality: A comparative analysis of teacher certification and student achievement in Gulf Cooperation Council (GCC) countries

In national education systems worldwide, teacher quality has become synonymous with education reform efforts, but a more elusive goal is empirically measuring teacher quality. One proposed measure of teacher quality, teacher licensing, also known as certification, is an increasingly ubiquitous component of national education systems and pre-service teacher education around the world. Rapidly developing national education systems, like those in the Arabian Gulf states, are actively seeking to measure teacher quality through teacher certification and to estimate teacher quality using scores on student achievement tests. This study synthesizes research literature and contextual data as a foundation for using hierarchical linear models to estimate the impact of teacher certification on student achievement in the Gulf Cooperation Council (GCC) countries. Despite the emphasis on improving teacher quality by establishing teacher certification requirements in GCC national education systems, the researchers found neither a direct nor a consistent association between teacher certification and student achievement in the GCC.     

Measuring the impact of competition in local government education markets on the cognitive achievement of students

In this paper, the role of interjurisdictional competition among school districts is examined. Using 1970 California school district data and controlling for a variety of education inputs, achievement scores are found to rise as the local education market becomes more competitive. This relationship continues until a competitive threshold is reached, beyond which additional districts have no impact on achievement. It is estimated that three to four school districts per county are necessary to create a competitive market in education. These findings are consistent with similar studies examining the effect of entry in the private sector. Additionally, the results show a variety of parental, school and student inputs to be significant in “producing” student achievement.     

Development and validation of Teacher Intentionality of Practice Scale (TIPS): A measure to evaluate and scaffold teacher effectiveness

Teacher evaluation instruments frequently fail to support teacher growth and improvement—particularly in critical constructs surrounding pedagogical content knowledge that promote student growth and achievement. The current study details the development and validation of the Teacher Intentionality of Practice Scale (TIPS), an observational protocol designed to measure intentional teaching practices and provide support for teacher growth over time. Findings suggest that TIPS provides a valid and highly reliable research-based teacher observational instrument that also serves as a professional development model to help scaffold the growth and improvement of K-12 educators. This paper details the theoretical foundations of TIPS, reports reliability and validity measures (including both Exploratory and Confirmatory Factor Analyses), and discusses applications and implications of TIPS for teacher evaluation and professional development.     

Teacher evaluation in Illinois: school leaders’ perceptions and practices

The aim of the current study is to assess school leaders’ perceptions and practices in the context of a new policy that emphasizes teacher evaluation. The study draws from survey data of 606 K-12 school leaders in the USA in a state implementing a new teacher evaluation model under Race to the Top. Findings illustrate that school leaders spent significant time on teacher evaluation. Some felt this was a good investment of time (e.g., rich conversations with teachers), increasing the ability for evaluations to improve instruction. Most, however, perceived the time demands as a cost (e.g., spending longer hours on the job, delegating more tasks to others). School leaders felt very confident in their ability to conduct observations and provide feedback to teachers. The opposite was found for student achievement data. Administrators’ reported preparedness, confidence, and beliefs suggest that barriers exist to using student achievement data to evaluate teachers. Findings suggest that school leaders need resources to build capacity, particularly in accommodating the new time demands of teacher evaluation. Administrators could benefit from professional development that addresses the use and value of student achievement if this measure continues to be a substantial component in teacher evaluation models.     

Educational evaluation schemes and gender gaps in student achievement

This paper investigates whether gender gaps in student achievement are related to evaluation schemes. We exploit different evaluations at the end of compulsory education in Norway in a difference-in-differences framework. Compared to the results at anonymously evaluated central exit exams, girls get significantly higher grades than boys when the same skills are assessed by their teacher. This gender grading gap in favor of the girls is found in both languages and mathematics. We find no evidence that the competitiveness of the environment can explain why boys do relatively better on the exam. We find some evidence that the gender grading gap is related to teacher characteristics, which indicates that the teacher–student interaction during coursework favors girls in the teacher grading.     

Principal holistic judgments and high-stakes evaluations of teachers

Results from a sample of 1,013 Georgia principals who rated 12,617 teachers are used to compare holistic and analytic principal judgments with indicators of student growth central to the state’s teacher evaluation system. Holistic principal judgments were compared to mean student growth percentiles (MGPs) and analytic judgments from a formal observation protocol. The correlations of a holistic principal rating with teacher MGPs and observation protocol scores were 0.22 and 0.32. Teachers selected as most successful at increasing student achievement had a mean MGP that was a full SD higher than did teachers selected as least successful, and a mean observation protocol score that was 1.35 SDs higher. Holistic principal judgments appear to be much more strongly influenced by observations of teachers’ classroom practices than they were by evidence of growth in student achievement.     

After-school tutoring in the context of no Child Left Behind: Effectiveness of two programs in the Pittsburgh Public Schools

The No Child Left Behind (NCLB) legislation has created pressure for districts to improve their students’ proficiency levels on state tests. Districts that fail to meet their academic targets for 3 years must use their Title I funds to pay for supplemental education services (SES) that provide tutoring or other academic instruction. Many districts, including the Pittsburgh Public Schools (PPS), have also adopted additional tutoring programs designed to help students reach proficiency goals. This paper examines student participation and achievement in two PPS tutoring programs—the NCLB-mandated SES program and a state-developed tutoring program. We examine the characteristics of students participating in each program, the effects of participation on student achievement, and the program features that are associated with improved achievement.     

Understanding science teaching effectiveness: examining how science-specific and generic instructional practices relate to student achievement in secondary science classrooms

ABSTRACT

In order to create conditions for students’ meaningful and rigorous intellectual engagement in science classrooms, it is critically important to help science teachers learn which strategies and approaches can be used best to develop students’ scientific literacy. Better understanding how science teachers’ instructional practices relate to student achievement can provide teachers with beneficial information about how to best engage their students in meaningful science learning. To address this need, this study examined the instructional practices that 99 secondary biology teachers used in their classrooms and employed regression to determine which instructional practices are predictive of students’ science achievement. Results revealed that the secondary science teachers who had well-managed classroom environments and who provided opportunities for their students to engage in student-directed investigation-related experiences were more likely to have increased student outcomes, as determined by teachers’ value-added measures. These findings suggest that attending to both generic and subject-specific aspects of science teachers’ instructional practice is important for understanding the underlying mechanisms that result in more effective science instruction in secondary classrooms. Implications about the use of these observational measures within teacher evaluation systems are discussed.     

The Structure and Substance of Teachers’ Opportunities to Learn about Teacher Evaluation Reform: Promise or Pitfall for Equity?

Despite growing momentum to overhaul teacher evaluation policies and practices, scant research examines how educators at the street level of such reform—principals and teachers—make sense of them, and almost no research examines the implications of current evaluation reforms for equity. This article provides findings based on a study of 14 districts implementing a new teacher evaluation policy in Connecticut. It focuses on how principals shaped teachers’ opportunities to learn about the new policy. We find that the majority of teachers’ opportunities to learn were formal and in whole group or one-on-one formats. We find important differences in the quantity and quality of learning opportunities at the district level, with districts serving greater shares of low-income students, students of color, and English language learners generally offering teachers fewer and lower quality opportunities to learn about the new reform than their counterparts. As such, this article builds on prior research illustrating the potential of new evaluation systems to exacerbate inequities and raises important cautions regarding the extent to which the unprecedented teacher evaluation reforms (currently underway) may exacerbate inequities among school districts.     

Questioning reform in the standards movement: professional development and implementation of common core across the rural South

The Common Core State Standards (CCSS) were adopted in an effort to create a consistent set of standards across schools, districts, and states in order to ensure equality of educational opportunities for all students in the country. Educational reform through standardization implies that across thousands of miles and millions of students, individual classroom teachers are able to create equitable instruction based on policy and curriculum mandates. For such reform efforts to be successful, standards must be implemented with fidelity, thoroughness, and strong support from stakeholders. The present study examines the role of teacher preparation for CCSS implementation, notably in rural and economically depressed districts in the American South. Teachers were surveyed about their CCSS professional development experiences and needs, CCSS instructional and assessment practices, and perceptions of student learning related to new standards. The analysis reveals that teachers do not feel fully prepared to teach according to CCSS, and highlights areas of professional development where districts failed to support effective teacher implementation. Findings from this study illustrate inequitable professional development for teachers and provide implications for professional development types, content, and quality that teachers perceive as essential for CCSS success.