Assessment of Teacher Reactions to a Standards-Based Teacher Evaluation System: A Pilot Study*

This article reports on teachers' reactions to a pilot implementation of a new standards-based teacher evaluation system in a medium-sized Midwestern school district. The system, based on the Framework for Teaching (Danielson, 1996), was intended to serve as the foundation for a new knowledge and skill pay plan for teachers. Teacher reactions were measured using both interview and survey methods. Teachers' overall favorableness toward the system, as measured by the survey, was correlated with acceptance of the teaching standards, perception of the evaluation process as fair and the evaluator as capable and objective, and perception that the evaluation system had a positive impact on their teaching. In interviews, teachers perceived both positive and negative aspects of the system, and for many these more or less balanced each other. While most teachers accepted the evaluation standards and the need for an evaluation system, many also perceived that the system added too much to teachers' workloads and did not provide enough feedback. Based on these results several changes were made in the evaluation system prior to full implementation. The results are discussed with reference to previous research and the Personnel Evaluation Standards.     

Criterion-Related Validity Evidence for a Standards-Based Teacher Evaluation System

This article reports evidence of the criterion-related validity of a standards-based teacher evaluation system implemented by the Cincinnati Public Schools. The analyses described provide evidence for a moderate level of criterion-related validity of the teacher evaluation ratings. The results indicate that teachers who received the lowest ratings on the Teaching for Learning domain had students who performed lower on state and district tests than predicted based on prior year test scores. Students of teachers who received the highest ratings on this domain generally performed better than predicted. Correlations (Kendall’s tau) between composite scores (the sum of four domain scores) and student achievement ranged from 0.28 to 0.37, depending on subject tested, and were statistically significant at conventional levels.A previous version of this paper was presented at the Annual Meeting of the American Evaluation Association, November 8, 2002 in Washington, D.C.     

Teacher evaluation and school improvement: An analysis of the evidence

In recent years, substantial investments have been made in reengineering systems of teacher evaluation. The new generation models of teacher evaluation typically adopt a standards-based view of teaching quality and include a value-added measure of growth in student learning. With more than a decade of experience and research, it is timely to assess empirical evidence bearing on the efficacy of this school improvement strategy. This paper examines the new generation of teacher evaluation along three lines of analysis: evidence on the magnitude, consistency, and stability of teacher effects on student learning, evidence on the impact of teacher evaluation on growth in student learning, and literature from the sociology of organizations on how schools function. Although the trend towards focusing on teacher evaluation is increasingly evident internationally, most of the empirical research evaluated in this paper is from the USA. This critical evaluation of the empirical literature yields two key conclusions. First, we conclude that the policy logic supporting this reform remains considerably stronger than the empirical evidence. Second, we suggest that alternative improvement strategies may yield more positive results and at a lower cost in terms of staff time and district funds.     

Examining the Relationship Between Teacher Evaluation and Student Assessment Results in Washoe County

In this article, we describe findings from an analysis of the relationship between scores on a standards-based teacher evaluation system modeled on the Framework for Teaching (Danielson, 1996) and student achievement measures in a large Western school district. We apply multilevel statistical modeling to study the relationship between the evaluation scores and state and district tests of reading, mathematics, and a composite measure of reading and mathematics. Using a value-added framework, the teacher evaluation scores were included at the 2nd level, or teacher level, of the model when other student and teacher-level characteristics were controlled. This study provided some initial evidence of a positive association between teacher performance, as measured by the evaluation system, and student achievement. The coefficients representing the effects of teacher performance on student achievement were positive and were statistically significant in 4 of 9 grade-test combinations studied.     

New teachers’ perceptions of a standards-based performance appraisal system

This study was conducted with first-year teachers. It provided insight into two key research questions: (1) What were the perceptions of new teachers regarding a standards-based performance appraisal system as it was implemented on their campus? (2) What factors contributed to the perceptions of new teachers regarding this system as it was implemented on their campus? This study was conducted in two phases. The first phase was an online survey consisting of 24 Likert-type, 4-point items designed to elicit responses regarding the individual teachers?? experience with the standards-based evaluation. These survey scores were then totaled and ranked to determine the top and bottom scoring participants. Extreme scoring respondents participated in individual interviews regarding the specific contributing factors behind their perceptions of the evaluation in the second phase. Interview data were then read and phenomena were coded to develop broad themes and subthemes of information that occurred during each interview. These patterns were analyzed to reveal factors that contributed to the negative and positive perceptions. This study found that the value a teacher placed on the evaluation process was influenced by the principal. In addition, teachers who felt well-prepared and well-supported by their peers and their principal viewed the experience positively, and teachers also placed a higher value on the process when they felt they received meaningful and timely feedback and/or were provided opportunity for self-reflection. Finally, this study found that the relationship teachers have with principals is not always a factor in their perception of the standards-based appraisal process.     

Whole-School Collaborative Sustained Professional Development and Science Teacher Change: Signs of Progress

The experiences of middle school science teachers at two schools in the southeastern US who participated in collaborative, sustained, whole-school professional development and their implementation of standards-based instructional practices are the focus of this study. Participants were involved in their second year of the professional development experience. The research question that was explored was “How are science teachers implementing standards-based instruction while participating in effective professional development experiences?” Data collected in the form of teacher interviews and classroom observations using an observation protocol. Findings indicate that even with effective professional development, teachers implement standards-based instructional practices at varying levels. The reasons for these differences are discussed and the role of teacher beliefs and the impact of beliefs, experience and support from administrators on professional development efforts are discussed. Implications for future science education reform stakeholders and professional development efforts are revealed.     

Teacher resistance against school reform: reflecting an inconvenient truth

In this article, the contexts, forms and consequences of teacher resistance against school reforms based on quality assurance policies are discussed. The problems are demonstrated by recurring on research devoted to the acceptance and/or resistance of (German) teachers against standards-based accountability policies. These results demonstrate that the majority in the teacher force ignores, misinterprets or misuses the feedback information from standards-based performance tests aiming at a data-driven development of classroom teaching. Only a small part shows an adequate usage of feedback information, especially those schools that already have developed a culture of internal evaluation. In general, the success or failure of educational innovations is difficult to estimate, because it is a matter of interpretation and post hoc reconstruction.     

Teacher Performance Ratings and Professional Improvement

Like other public workers, teachers typically receive high and compressed ratings that do little to differentiate them based on performance. Motivated by empirical evidence of substantial variation in effectiveness among teachers, there has been a recent push to develop more informative evaluation systems with greater ratings dispersion. We study one of the first of these new systems, implemented in Tennessee, in order to understand how teachers respond to the provision of new, more differentiated performance ratings. We focus on whether summative ratings influence teachers’ self-reported, self-directed professional improvement activities as measured by four items on a statewide teacher survey. Using a regression discontinuity design we find no evidence that teachers alter their time investments in professional improvement, or adjust their professional improvement activities based on evaluation feedback, in response to their ratings.     

Patchwork Quilt or Seamless Day? Parent, Teacher and Child Care Staff Views on Early Childhood Education Programs for Kindergarten-Age Children in Canada

In Canada, child care for kindergartners typically combines part-day school and child care. Exploratory research examined parents', teachers', and child care staff views on coordination between these programs. Community-based research surveyed these three groups. There was general agreement, and some key differences, among the groups in priorities for programs for kindergarten-age children. Social and language skills were generally considered most important; reading and writing least. Parents were particularly enthusiastic about a coordinated, school-based program. Teachers and child care staff were somewhat less enthusiastic about the model, with teachers' reactions the least positive. Teachers' views on the advisability of an integrated program were varied—some thought it would be of benefit to children, others disagreed. All three groups felt that a coordinated program would benefit by being school-based. Direct experience with integrated programs seemed to produce strong opinions about the value of program integration, with those teachers who had on-site child care programs giving the integrated programs the highest and the lowest ratings. Arguments are made for small-scale demonstration programs to expand provision of child care services using kindergarten as a foundation—and for rigorous evaluation of program impacts and outcomes.     

美国绩效标准本位教师教育课程理念与实践探析

美国绩效标准本位教师教育课程理念与实践的变革体现为四个维度:教师观,强调教师是从事复杂工作的专业者;知识观,认为教学知识是复杂的情景化知识;教学观,倡导教学要基于情境,以学生为中心展开;评价观,主张教学评价要注重个体发展和整体质量。美国教师教育相关院校和机构依据上述理念,基于自身特点选择并调适相应的教师教育标准,对教师的职责、教学理论及评价方式等进行多方面改革探索,实践中改革者在各维度所做的改变和尝试可为我国教师教育课程标准的优化和教师教育课程的构建与实施提供启示。     

How to activate teachers through teacher evaluation?

There is a general doubt on whether teacher evaluation can contribute to teachers’ professional development. Recently, standards-based teacher evaluation has been introduced in many countries to improve teaching practice. This study wants to investigate which teacher evaluation procedural, leadership, and teacher characteristics can stimulate teachers’ feedback utility perceptions and professional learning activities in the context of teacher evaluation. Data were collected in 32 secondary schools from 298 teachers and 32 school leaders, both quantitatively and qualitatively. We put forward a research model which is investigated through path analysis. Results nurture optimism about teacher evaluation indicating that teacher evaluation can stimulate teachers to undertake professional learning activities when they perceive feedback as useful. The important role of the school leader (through both instructional and transformational leadership) is stressed. Teacher participation and teacher collaboration also have a positive influence on teacher evaluation and its outcomes.     

Teacher effectiveness and teacher growth from student ratings: An action research of school-based teacher evaluation

Teacher effectiveness evaluations are considered an effective mechanism for promoting teacher growth. We implemented teacher effectiveness evaluations in three Chinese secondary schools over two years and (a) developed an action research framework for school-based teacher evaluations composed of five links, i.e., purpose and consensus, data collection and analysis, data interpretation and feedback, data use, and evaluation of the actions and (b) found that “school-based” instruments of student ratings have valid statistical properties, and teacher effectiveness can be regarded as a single-factor holistic concept in different schools. We also (c) analysed 2-year longitudinal data and identified changing trajectories of teacher effectiveness as an example for data feedback and use. Teacher effectiveness presents changing trajectories for different subgroups, representing different ecologies in three schools and provides insights for teacher growth. A (d) meta-evaluation of the actions shows that most teachers are positive about this evaluation, although differences are observed between schools.     

Teacher evaluation in China: latest trends and future directions

With the implementation of teacher performance pay in 2009 in China, teacher performance evaluation has become a heated topic. This research study follows up on two previous studies of teacher evaluation in China and continues the dialog by analyzing the latest trends in the context of teacher performance pay. There were two sources of information for this article: academic journal articles and teacher evaluation practices of two schools from Beijing. Seven themes were derived from a content analysis of the academic journal articles: (a) studies on teacher evaluation within the context of curriculum reform, (b) methods of teacher evaluation, (c) studies of miscellaneous responses, (d) studies on developmental teacher evaluation, (e) studies on teacher performance evaluation, (f) studies on teacher effectiveness, and (g) philosophical thoughts of teacher evaluation. The two cases of teacher evaluation practices revealed some changes since the implementation of teacher performance pay, such as a more comprehensive teacher evaluation system.     

从技能训练走向全人发展——美国绩效标准本位教师教育课程改革分析

在教师教育标准作为教师教育改革和保障教师教育质量的一个基本前提下,受到认知主义、建构主义学习理论和课堂评价的影响,美国推进了绩效标准本位教师教育课程的改革,改变过去过于注重技能训练的标准,从教师观、教学观、教学知识基础观和评价观上提出了新的标准,以期把教师培养成为一位专业的智力工作者,形成基于情境以学生为中心的教学观,让教师掌握多向度的、综合性的教学知识,注重教学整体质量的评价,促使高素质教师发展。这一改革构建了教学专业化的复杂性的理念,教师教育课程改革呈现出从技能训练走向综合发展、对个体和个体知识关怀的实践取向的全人发展观趋势,为优化我国教师教育课程改革提供了启示。     

Student Interest,Grading Leniency,and Teacher Ratings: A Conceptual Analysis

This study explored the extent to which student interest and grading leniency were predictive of teacher ratings. Unlike other teacher rating research, however, this study measured student interest at the beginning and end of seven sections of two courses; further, grading leniency was explicitly defined and measured, not inferred from expected grade and workload. Data indicate that precourse interest was positively associated with expected grade, but was not predictive of ratings, nor did it moderate the expected grade–rating association. Rather, interest change was positively associated with expected grade and predictive of ratings. Further, interest change and grading leniency provided incremental variance in ratings, beyond that provided by expected grade.     

Introducing a Teacher Evaluation System Based on Teacher Effectiveness Research: An Investigation of Stakeholders’ Perceptions

Teacher Effectiveness Research (TER) could be a foundation upon which a valid teacher evaluation system could be built. However, even a technically exceptional evaluation system may be doomed to fail if the political dynamics that influence its implementation are not examined. This study examines the extent to which a proposed teacher evaluation system based on TER is possible to gain acceptance from the main stakeholders of the Cypriot educational system. The extent to which stakeholders’ reactions to the proposed system are associated with their personal interests and concerns is also investigated. Both groups recognized that TER could be a foundation upon which a valid teacher evaluation system could be built but were critical of suggestions that might reduce their professional power. In order to reach consensus, policy makers should establish procedures to ensure a clear understanding among stakeholders of both the theoretical assumptions of the proposed system and the type of concerns that stakeholders might have against change.     

Teacher Evaluation, Teacher Effectiveness and School Effectiveness: Perspectives from the USA

This article provides historical overviews of the conceptual and research and development focus of teacher evaluation, teacher effectiveness and school effectiveness research in the USA. Pertinent literature is cited and arguments are made that these lines of inquiry have coexisted for nearly four without adequate integration. With the fourth stage of school effectiveness research in process, there is a recognition that within school context variables, particularly teacher effectiveness, have important effects on school improvement and school outcomes. Similarly, there is the recognition that findings from school effectiveness research have relevance for studies of teacher effectiveness and ongoing developments in teacher evaluation. Examples of: (a) new generation, learner-centered teacher evaluation systems in the USA that are informed by teacher and school effectiveness research; and (b) the fourth stage of school effectiveness research are described. It is proposed that these lines of research should be merged as completely as possible.     

The Impact of Reform: Challenges for Mathematics Teacher Preparation

The mathematics education community has been widely recognized as a leader in the standards-based reform movement. Despite the widespread interest and attention that reform documents such as the NCTM Standards have generated, what has yet to be fully understood is the impact of these types of reform documents on the preservice teacher preparation process. This paper examines the state of the standards-based reform effort in mathematics teacher education by presenting the results of a three-year study of six cohorts of secondary mathematics student teachers (n = 63). Examined in particular are the ways in which these student teachers' professed beliefs about and knowledge of the NCTM Standards contrast with their teaching practices. Possible explanations for the mismatch between belief and knowledge statements and teaching practices are examined, along with other implications for mathematics teacher preparation.     

The Validity of Teacher Ratings of Adolescents' Social Skills

The validity of teacher ratings of adolescents' social skills was examined in a follow-up study of a cohort sample of 395 students from seventh to ninth grade. The internal consistency of the teacher ratings on Social Skills Rating System (SSRS) was examined at two points in time, and the multidimensionality of the SSRS was confirmed through factor analyses. The findings indicated multi-informant consistency in ratings: teacher ratings consistently covaried with teacher ratings of problem behaviour in the classroom and academic competence, sociometric nominations by peer students, and grade point average in selected school subjects. Teacher ratings also produced construct-relevant group differences, with students referred to psychosocial and educational helping services receiving significantly lower ratings than their nonreferred peers.