Assessing School Readiness: Validity and Bias in Preschool and Kindergarten Teachers' Ratings

As a part of efforts to evaluate and monitor the increasing public investment in early childhood education, teachers are being asked to assess children's school readiness. In this study, preschool teachers and kindergarten teachers rated children's skills in three areas (kindergarten readiness, academic skills, and communication skills), and these ratings were compared with direct assessments of the children's skills. Ratings by both groups of teachers tended to be more highly related to basic skills, such as counting and number naming, than to abilities such as solving applied problems and using expressive and receptive vocabulary. Preschool teachers' ratings had a lower association with children's observed skills and abilities than kindergarten teachers' ratings. Ratings of children attending Head Start were systematically inflated, but this relationship was mediated to a significant extent by the teachers' levels of education. More educated teachers rated children in a manner consistent with the children's directly assessed skills. Implications of these findings for informing future efforts to assess school readiness by using teacher ratings are discussed.     

What contemporary work are student ratings actually doing in higher education?

It is conventionally assumed that student ratings perform a significant function in driving improvement in pedagogical practices in higher education. As a result, this form of evaluation has gradually become institutionalised in recent decades as an essential proxy for understanding teaching and course quality in universities across the world. However, with the rise of market-based models in higher education and heightened expectations for accountability mechanisms, the role and functional purpose of ratings-based student evaluation have become increasingly confused. This rising ambiguity has created strong tensions between the seminal drive of student ratings as a tool of quality improvement, and the emerging demands for its use as a transparent accountability measure for the comparative assessment of academic performativity. So are student ratings now largely a tool of quality assurance or performance measurement, or do they remain a legitimate tool for pedagogical improvement? This paper reports on a study that responded to this critical question by considering the contemporary work of student ratings in a major Australian university. The outcomes of this research demonstrate that tension between improvement and accountability motives is causing considerable confusion and discord around the role and value of the student voice. It also reveals that academics are tending to discount the often critical insights of students on the implications of their pedagogical practices as a result of the elevating institutional role of student ratings as a proxy for teaching quality. In considering these outcomes, rising levels of academic dissonance around student ratings would suggest a necessity to consider broadened evaluative strategies that are able to more effectively capture the improvement potential offered by the student voice.     

Peer assessment of class participation: applying peer nomination to overcome rating inflation

Having students actively engaged with each other in discussions has become an increasingly important and common aspect of the classroom environment. This increased emphasis has also meant that instructors need to find ways to effectively and efficiently evaluate class participation. In this paper, we describe the most common method used for these assessments and highlight some of its inherent challenges. We then propose an alternative method based on peer nominations. Two case studies illustrate the advantages of this method; we find that it is both easy for students to complete and provide instructors with valuable diagnostic information with which to provide feedback and assign grades.     

CET-4作文评分人评分标准使用情况的研究

本研究采用混合研究法对CET-4作文评分人如何使用评分标准进行分析。26位CET-4作文评分人对30篇CET-4模拟作文评分,并提供3条按重要性排序的评分理由。研究结果显示：（1）虽然存在严厉度的差异,但是26位评分人之间的一致性比较好,且大部分评分人的自身一致性也较好。（2）部分评分人的评分理由呈现了单一化趋势。（3）评分人所给评分理由的71.91%体现了CET-4作文评分标准所规定的5个文本特征,说明大部分评分人对标准的理解和把握还是比较准确的。     

Who is the expert? Construct and criteria validity of student and teacher ratings of instruction

In this article, we examine the construct and criteria validity of student and teacher ratings as indicators of instructional features. As parsimonious measures, student and teacher reports are frequently used to assess aspects of the learning environment. From a methodological perspective, however, both approaches have been questioned. Whereas student ratings are occasionally criticised as being undifferentiated and easily influenced by personal preferences, teacher reports are sometimes considered to be biased by self-serving strategies or teaching ideals. Instead of pitting one method against the other, our study aimed at establishing the specific value of each approach. The analyses draw on data from a German extension to the 2003 PISA assessment, in which 288 mathematics teachers and their students completed questionnaires using identical items to tap various aspects of instruction. Mean student ratings were computed for each class. Exploratory and confirmatory factor analyses revealed specific conceptual structures for student and teacher ratings, with teachers elaborating on the use of tasks and methods, and students focusing on their teacher’s support in personal and learning matters. Three shared dimensions were also established: occurrence of classroom management problems, degree of cognitive autonomy during lessons, and tempo of interaction. Teacher/student agreement on these dimensions varied across constructs, with considerable agreement on classroom management, low agreement on cognitive autonomy, and no significant agreement on whether or not the interaction tempo was appropriate. Accordingly, correlations with external criteria (student motivation, teacher motivation, mathematical achievement scores, and characteristics of the tasks set in class) also varied across constructs. We conclude that student and teacher ratings are best suited to tapping different aspects of the learning environment, and recommend that care be taken in choosing a data source appropriate for the construct to be measured.

The Language of Schools: Curricular goals and classroom life

The experience of becoming a teacher involves more than learning to use the tools of the trade. Using the example of teaching questioning strategies in a curriculum methods seminar for student teachers, two teacher-educators present questions and dilemmas that arise out of the tension between the use of language as a tool, and the experience of language as an aspect of becoming teachers. Anecdotes from classrooms show how student teachers develop an awareness of the dynamic interplay of these language forms. We suggest that this dynamic interplay can be understood as aesthetic form, an aspect of teaching practice which enables teachers to sustain the pedagogical relation.     

我国公交车载移动电视发展现状及传播策略

面对竞争日趋激烈的媒介生态环境,公交车载移动电视因其独特的传播优势蕴涵着诱人的商业价值和广阔的市场前景。面对发展中的重重困境,公交车载移动电视要想获得经济效益和社会效益的双赢局面,需要进一步明确发展理念、确定产业标准、把握受众需求、实施自主创新,在收视环境和收视效果上开拓创新。     

Graduate teaching assistants: how well do their students think they do?

Graduate teaching assistants (GTAs) constitute a valuable and economical teaching force in many higher education undergraduate programmes. However, student satisfaction with their teaching has attracted little attention in the research literature. This study aimed at examining students’ evaluation of teaching of GTAs in discussion groups, as well as exploring the effects of group and GTA variables on these ratings. Data were collected using a questionnaire administered online and completed by 7078 undergraduate students. Participants were enrolled in classes taught by 278 GTAs from four faculties in a major Israeli university. Results indicated that ratings assigned to clarity of instruction were the most salient predictor of students’ overall evaluation. Generally, findings were consistent with those reported in the literature for other categories of instructors. Groups taught by GTAs in exact sciences and engineering were rated higher than those in social sciences and business management. Group size and the percentage of men students were inversely correlated with student ratings, while student attendance rate was positively correlated. Women GTAs and GTAs who taught more than one group tended to receive higher ratings. Overall student attendance rate was the most prominent predictor of student ratings. The implications of the findings are discussed.     

Student Ratings and Professor Self-Ratings of College Teaching: Effects of Gender and Divisional Affiliation

Twenty female and 23 male professors at a liberal arts college participated along with their 803 undergraduate students in a questionnaire study of the effects of professor gender, student gender, and divisional affiliation on student ratings of professors and professor self-ratings. Students rated their professors on 26 questions tapping five teaching factors as well as overall teaching effectiveness. Professors rated themselves on the same questions as well as on nine exploratory ones. On student ratings, there were main effects for both professor gender (female professors were rated higher than male professors on the two interpersonal factors) and division (natural science courses were rated lowest on most factors). These patterns were qualified by significant interactions between professor gender and division. Although professor self-ratings varied by division, there were few significant correlations between professor self-ratings and students' ratings. Implications for future research are discussed.Appreciation goes to Laura Capotosto and Julie Phelan for their research assistance.Suzanne Montgomery is now at Widener Law School, Philadelphia, PA, USA.     

Relationships between colleague ratings and faculty compensation

Rankings of graduate and professional programs have become commonplace since Cartter's study for the American Council on Education was published in 1966. Much of the research up to the present has focused on discovering proxy variables, such as departmental size, which correspond to departmental ratings, as well as observing how the ratings of departments change over time. This study examines the relationship between the peer ratings reported by Roose and Andersen in 1970 and resource allocation patterns, represented by average salary figures, among fifty major American universities. It also shows the relationship of these ratings and other resource allocation measures within one of the fifty institutions. Results indicate that among the fifty universities, average salaries are higher at the highly rated institutions than at the lower rated ones, with senior professors being the chief beneficiaries of the higher wage scales. Similarly, within one institution the senior professors of higher ranked departments are better paid, compared to their junior colleagues, than those of lower rated departments. The higher rated departments share commonalities as to proportion of senior professors and teaching load assignments as well. Both across and within universities, departments found to be prestigious by Roose and Andersen do tend to exhibit common characteristics concerning resource allocation procedures.Presented at the Annual Forum of the Association for Institutional Research, Houston, Texas, May 1978.