Uncovering Multivariate Structure in Classroom Observations in the Presence of Rater Errors

We examine the factor structure of scores from the CLASS‐S protocol obtained from observations of middle school classroom teaching. Factor analysis has been used to support both interpretations of scores from classroom observation protocols, like CLASS‐S, and the theories about teaching that underlie them. However, classroom observations contain multiple sources of error, most predominantly rater errors. We demonstrate that errors in scores made by two raters on the same lesson have a factor structure that is distinct from the factor structure at the teacher level. Consequently, the “standard” approach of analyzing on teacher‐level average dimension scores can yield incorrect inferences about the factor structure at the teacher level and possibly misleading evidence about the validity of scores and theories of teaching. We consider alternative hierarchical estimation approaches designed to prevent the contamination of estimated teacher‐level factors. These alternative approaches find a teacher‐level factor structure for CLASS‐S that consists of strongly correlated support and classroom management factors. Our results have implications for future studies using factor analysis on classroom observation data to develop validity evidence and test theories of teaching and for practitioners who rely on the results of such studies to support their use and interpretation of the classroom observation scores.     

Examining Rater Effects on the Classroom Assessment Scoring System

The Classroom Assessment Scoring System (CLASS; Pianta et al., 2008) is a popular measure of teacher–child interactions. Despite its prominence, CLASS scores have fairly weak relations with various child outcomes (e.g., Zaslow et al., 2010). One potential reason for these findings could be systematic differences in observer severity. As such, the purpose of this study was to explore the scope and impact of rater effects on CLASS scores with a sample of 77 teachers who were rated by 13 observers. Results indicated significant rater effects across all three CLASS domains. Adjusting for these effects, however, did not improve relations between CLASS scores and child outcomes. Implications for the CLASS and related assessments are discussed.     

A Comparison of Teacher Effectiveness Measures Calculated Using Three Multilevel Models for Raters Effects

This study examines the use of cross-classified random effects models (CCrem) and cross-classified multiple membership random effects models (CCMMrem) to model rater bias and estimate teacher effectiveness. Effect estimates are compared using CTT versus item response theory (IRT) scaling methods and three models (i.e., conventional multilevel model, CCrem, CCMMrem). Results indicate that ignoring rater bias can lead to teachers being misclassified within an evaluation system. The best estimates of teacher effectiveness are produced using CCrems regardless of scaling method. Use of CCMMrems to model rater bias cannot be recommended based on the results of this study; combining the use of CCMMrems with an IRT scaling method produced especially unstable results.     

Measuring Parent Receptivity Towards the Classroom Environment Using a Rasch Measurement Model

This study investigated parent-reported receptivity towards the classroom environment and classroom outcomes. Classroom environment was based on a five-aspect model: (1) provision of information from the child; (2) beliefs about the school; (3) provision of information from teachers; (4) teachers' commitment to working with parents; and (5) confidence in communicating with teachers. Classroom outcomes were based on two aspects: (1) educational values (importance of schooling, involved with learning; seeing a future through learning, desire to learn, and importance of learning); and (2) learning outcomes (achieving, and views of child's engagement in school work). For each aspect, items were written in an ordered-by-difficulty pattern so that, for example, Item 2 involved Item 1 and ‘more’, making it conceptually ‘harder’ to agree with Item 2 than with Item 1. There were four Likert response categories (SDA, DA, A, and SA). Using the extended logistic model of Rasch, an interval-level, unidimensional scale was created with item difficulties for classroom environment aspects and classroom outcomes calibrated on the same scale as the receptivity measures. The sample consisted of 518 parents of students from three secondary schools in Western Australia. The item sample was 30. The proportion of observed variance considered true was 0.94. The items for each aspect were found to be ordered from ‘easy’ to ‘hard’ in line with the hypothesised model of receptivity and the data fitted the measurement model well. This revised version was published online in July 2006 with corrections to the Cover Date.     

Rater Effects on Essay Scoring: A Multilevel Analysis of Severity Drift,Central Tendency,and Rater Experience

This study examined rater effects on essay scoring in an operational monitoring system from England's 2008 national curriculum English writing test for 14‐year‐olds. We fitted two multilevel models and analyzed: (1) drift in rater severity effects over time; (2) rater central tendency effects; and (3) differences in rater severity and central tendency effects by raters’ previous rating experience. We found no significant evidence of rater drift and, while raters with less experience appeared more severe than raters with more experience, this result also was not significant. However, we did find that there was a central tendency to raters’ scoring. We also found that rater severity was significantly unstable over time. We discuss the theoretical and practical questions that our findings raise.     

提高语文课堂运用多媒体的有效性

在语文课堂中,适时、合理地运用多媒体技术,能够为语文教师提供非常丰富的教学资源,创设逼真、生动的教学情境,为学生增长语文知识、接受文学熏陶、开阔思维眼界、启迪进步思想创造更为便利的条件。多媒体技术为小学生语文教学效率的提高提供了有效的途径。作者根据教学经验,着重进行了创设情境、突破重难点、培养观察想象能力、搭建学生自主学习的平台等方面的尝试,取得了一定的效果。     

Evaluating the Validity of Classroom Observations in the Head Start Designation Renewal System

Classroom observations are increasingly common in education policies as a means to assess the quality of teachers and/or education programs for purposes of making high-stakes decisions. This article considers one policy, the Head Start Designation Renewal System (DRS), which involves classroom observations to assess the quality of Head Start programs in order to decide whether their funding is renewed. This article applies an argument-based approach for evaluating the validity of observational assessments that (a) explicates assumptions that underlie the presumed logic, leading from the collection of scores from observations of Head Start classrooms, to the inference that scores assess the quality of Head Start programs, to the decision to renew funding to Head Start programs, and (b) summarizes evidence that speaks to the plausibility of each assumption. There was limited evidence to support the plausibility of many assumptions, including those pertaining to score generalizability, predictive validity, and the cutoff scores set as minimum standards of quality. Implications for improving the validity of classroom observations and the accuracy and fairness of decisions in the Head Start DRS are discussed.     

Teacher and Classroom Context Effects on Student Achievement: Implications for Teacher Evaluation

The Tennessee Value-Added Assessment System (TVAAS) has been designed to use statistical mixed-model methodologies to conduct multivariate, longitudinal analyses of student achievement to make estimates of school, class size, teacher, and other effects. This study examined the relative magnitude of teacher effects on student achievement while simultaneously considering the influences of intraclassroom heterogeneity, student achievement level, and class size on academic growth. The results show that teacher effects are dominant factors affecting student academic gain and that the classroom context variables of heterogeneity among students and class sizes have relatively little influence on academic gain. Thus, a major conclusion is that teachers make a difference. Implications of the findings for teacher evaluation and future research are discussed.     

浅析课堂提问的有效性(英文)

众所周知,课堂交流在教学中非常重要,并且课堂提问是师生双边活动最重要的形式。教师在课堂上能否恰当地运用提问直接影响着课堂教学效果。有效的课堂提问可以激发学生的注意力、兴趣和爱好。此外,课堂提问还可以帮助教师及时了解不同层次学生的学习理解程度。因此,教师有必要明白提问在课堂教学中的重要性,关注目前课堂提问的现状及存在的问题并熟练掌握提问技巧与策略,尝试有效进行课堂提问,切实提高课堂教学效果。     

数学课堂教学中师生交往的有效化

数学课堂教学改革是数学课程改革的一项重要工作。数学课堂教学改革就是要用新课程理念改革过去被动式的课堂教学模式,使课堂教学成为“师生之间、学生之间交往互动与共同发展的过程”。要实现这一目标,建构课堂教学中真实的师生交往是关键。当前,在中小学数学课堂教学的师生交往中存在一些不真实的现象,这已对数学新课程的教学实施造成了一些不利的影响。因此,要正确认识数学课堂教学中师生交往的涵义,分析不真实交往的成因,明确师生之间真实交往的特征,建构数学课堂教学中有效的师生交往,从而推动数学课堂教学改革的健康发展。     

高校教师发展状况与课堂教学效果关系分析

高校教师存在发展空间受限,教师聪明才智难以充分发挥,严重影响高校课堂教学效果的问题。综合分析了教师教学方法不当,高校教师精力与兴趣点转移等问题的主要原因,并提出运用教学改革政策激励高校教师发展,鼓励教师全身心投入教育教学的策略,为提升高校课堂教学效果及质量提供理论依据。为实现高校教师发展与课堂教学效果的良性循环提供保障。     

高校教师发展状况与课堂教学效果关系分析

高校教师存在发展空间受限,教师聪明才智难以充分发挥,严重影响高校课堂教学效果的问题。综合分析了教师教学方法不当,高校教师精力与兴趣点转移等问题的主要原因,并提出运用教学改革政策激励高校教师发展,鼓励教师全身心投入教育教学的策略,为提升高校课堂教学效果及质量提供理论依据。为实现高校教师发展与课堂教学效果的良性循环提供保障。     

教师效能研究述评

教师效能研究一直备受学者的广泛关注.本文通过文献分析,回顾了国外教师效能研究的发展历史及其特点,总结了教师效能研究的最新发展状况,并在此基础上探讨了该领域今后的发展方向.     

国外英语课堂教师话语研究综述(英文)

该文对国外英语课堂教师话语研究进行梳理并简要综述。先从语言教学与研究的角度谈本话题的研究目的、意义、重要性,然后对国外主要经典研究进行概述,以期对我国的相关研究提供启示。     

AI全科教师主讲课程学习成效试验研究

智能技术的发展和全科教学的需求催生了AI全科教师主讲课程.本研究以广州市某小学四年级两个班79名学生为样本,采用深度访谈、课堂观察和问卷调查等研究方法,基于拟人化学习过程理论框架探究AI全科教师主讲课程学生学习成效.研究结果表明,AI全科教师主讲课程能显著增强学生学习动机(包括内在动机和外在动机)、增加学生学习投入(包括行为投入、认知投入和情感投入)和提升学生学习结果(包括知识与技能、过程与方法、情感态度与价值观).本研究发现证实了AI全科教师作为教学主体促进学生高效学习的重要作用,为解决师资匮乏、教师减负增能等问题提供了重要启发.     

国外英语课堂教师话语研究综述

该文对国外英语课堂教师话语研究进行梳理并简要综述.先从语言教学与研究的角度谈本话题的研究目的、意义、重要性,然后对国外主要经典研究进行概述,以期对我国的相关研究提供启示.     

Examining Rater Errors in the Assessment of Written Composition With a Many-Faceted Rasch Model

This study describes several categories of rater errors (rater severity, halo effect, central tendency, and restriction of range). Criteria are presented for evaluating the quality of ratings based on a many-faceted Rasch measurement (FACETS) model for analyzing judgments. A random sample of 264 compositions rated by 15 raters and a validity committee from the 1990 administration of the Eighth Grade Writing Test in Georgia is used to illustrate the model. The data suggest that there are significant differences in rater severity. Evidence of a halo effect is found for two raters who appear to be rating the compositions holistically rather than analytically. Approximately 80% of the ratings are in the two middle categories of the rating scale, indicating that the error of central tendency is present. Restriction of range is evident when the unadjusted raw score distribution is examined, although this rater error is less evident when adjusted estimates of writing competence are used     

Wearing My Identity: A Transgender Teacher in the Classroom

The term transgender is used by people whose gender identity or expression falls outside the boundaries of traditional gender expectations. In educational systems, transgender issues are becoming increasingly relevant as both students and staff "come out" as transgender, and as young people explore non-normative gender expression. In comparison to the empirical and theoretical discussions on gay, lesbian, and bisexual youth issues in education, research on transgender youth is sparse, and academic research on transgender teachers is non-existent. Like closeted gay, lesbian, and bisexual teachers, transgender teachers are isolated, hidden, and silent about their authentic identities. This exploratory study offers one transgender-identified teacher's story in an urban public school system. The issues addressed include gender dynamics in the classroom, relationships with students, the connections between sexual orientation and gender identity, and discrimination in the work environment.     

Teacher Evaluation, Teacher Effectiveness and School Effectiveness: Perspectives from the USA

This article provides historical overviews of the conceptual and research and development focus of teacher evaluation, teacher effectiveness and school effectiveness research in the USA. Pertinent literature is cited and arguments are made that these lines of inquiry have coexisted for nearly four without adequate integration. With the fourth stage of school effectiveness research in process, there is a recognition that within school context variables, particularly teacher effectiveness, have important effects on school improvement and school outcomes. Similarly, there is the recognition that findings from school effectiveness research have relevance for studies of teacher effectiveness and ongoing developments in teacher evaluation. Examples of: (a) new generation, learner-centered teacher evaluation systems in the USA that are informed by teacher and school effectiveness research; and (b) the fourth stage of school effectiveness research are described. It is proposed that these lines of research should be merged as completely as possible.