Student evaluations of teaching: effects of the Big Five personality traits,grades and the validity hypothesis

The purpose of the current study was to examine whether the Big Five personality traits and expected student grades relate to student evaluations of teachers and courses at the college level. Extraversion, openness, agreeableness and conscientiousness were found to be personality traits favoured in instructors, whereas neuroticism was not. A significant correlation was found between the students’ expected grades in the course and student evaluations of the course, but not the evaluations of the instructor. When the effect of students’ perceived amount of learning was taken into account, no significant effect of grades was found on teacher ratings. Personality explained variance in teacher and course evaluations over and above grades and perceived learning.     

Bias hypotheses under scrutiny: investigating the validity of student assessment of university teaching by means of external observer ratings

To advance the discussion on the validity of student evaluations of university teaching, student ratings of two teaching dimensions – student involvement and rapport – were compared with corresponding observer ratings. Seven potential bias variables were tested with regard to their impact on the students’ teaching assessment: three teacher characteristics (first impression, enthusiasm, humour) and four student characteristics (prior interest, expected grades, study experience, class attendance). Bias was defined as an impediment of the students’ assessment of teaching on course level. By means of bivariate correlations with course averages and two-level latent moderated structural equations, data of 1,716 students in 80 courses were analysed. Results showed that all three teacher characteristics were genuinely connected to rapport, and even explained variance of the student-rated variable when controlling for observer-rated rapport. The assessment of student involvement was not modified by the teacher characteristics except for teacher enthusiasm, which affected the student evaluation when controlling for observed involvement and, moreover, moderated the relation between the observed and the student-rated variable. For the examined student characteristics, no biasing effects were found – neither on rapport nor on student involvement.     

The Meaning and Utility of Institutional Teaching Evaluations

This study analyzes end of course institutional teaching evaluations (ITEs) done anonymously by students in a pre-service teacher education course over a 15-year span. The purpose was to determine if and how the ITE findings might inform practice and relate to teaching metaphors as a tool of study. Analysis revealed: (a) teacher effectiveness ratings were more meaningful when examined by metaphor periods and when outliers, which skewed the trends, were removed, (b) patterns in students’ optional narrative comments on ITEs indicated that students consistently saw instructor strength in caring and weakness in clarity, (c) there was no clear correlation between narrative comments and teacher effectiveness ratings, and (d) alignment with identified teaching metaphors was strongest with student narrative comment categories. In addition, doing this analysis reframed the way I thought of myself as a teacher and allowed me to make positive changes in how I related to students. Comments from the course in the semester following the analysis support this contention. This work informs the field by considering the usefulness of metaphors for long-term study of practice as well as modeling how teacher educators subjected to accountability measures can use such institutional measures to create meaningful study of practice.     

Students’ experience of peer assessment in a professional course

This paper reports a case study that investigated the practice of peer assessment in a professional course, with particular emphasis on students’ experience. It was found that peer assessment processes were beneficial to students’ learning and development as professionals. The case study pointed to several conditions for effective implementation of peer assessment, which also relate to assessment in general. These conditions include: (a) adequate and appropriate preparation for the use of peer assessment; (b) alignment of assessment, learning objectives and the broader purpose of the course (e.g. preparation as professionals); (c) the availability of assistance from a teacher throughout the peer assessment process; and (d) constructive discussions following peer assessment, sensitively handled by a teacher.     

Course evaluation and attitudes toward college teaching

Prior research suggests that attitudes may be one category of personal (student) variables that students bring to the task of evaluating college courses and their instructors. In a previous study, attitudes toward college teaching, in general, and the perceived progessivism of instructors, in particular, were associated with students' evaluations of courses. Perceived attitudes were a kind of evaluation. The results of the study raised the question as to when the attitude-evaluation relation was first formed in the course of a semester and whether a causal order between them could be established. The present study, using a repeated-measures design, replicated the major findings of the previous study on a cross-section of University of Haifa students attending 22 courses. While there is no clear evidence for the priority of either attitudes or evaluation, the findings indicate that their association is formed early in the course, with some further consolidation later on. Both attitudes and evaluations are remarkably stable during the course. In view of these findings it is suggested that classroom evaluations might be moved back to one of the first meetings of the course to allow for feedback and changes in the instructor's presentation. Moreover, perceived attitude measures could be developed into an indirect measure of teacher performance.     

The validity and students’ experiences of peer assessment in a large introductory class of gene technology

Although studies have examined the validity of peer assessment, research including students’ own experiences of peer assessment is scarce. The present study aims to improve assessment practices in a context with a highly traditional assessment culture. The aim is first to examine the validity of peer assessment by analysing the compatibility of student and teacher evaluation and explore the differences between minor and major students’ evaluations. Second, the study examines students’ experiences of peer assessment. Peer assessment was implemented in a large bioscience course with 79 student participants. After the peer assessment, the students provided feedback. The results indicate that student subject understanding can be supported through a proper assessment practice. Peer assessment was successful in an introductory class with minor and major students, and most students experienced it as supportive of their learning.     

Exploring graduate and undergraduate course evaluations administered on paper and online: a case study

The purpose of this study was to analyse the students’ evaluations of the course and instructor for all statistics courses offered during fall semester 2009 at a large university in the southern United States. Data were collected and analysed for course evaluations administered both online and on paper to students in both undergraduate and graduate courses. Unlike most previous studies on this subject, class section rather than student was treated as the unit of analysis. It was of specific interest to verify prior research findings that evaluation surveys administered online would not result in lower course and instructor ratings and lower response rates. The results showed that there is not sufficient evidence within the collected data to conclude that either course and instructor ratings or response rates are lower for evaluations administered online (online evaluations) than they are for evaluations administered on paper (paper evaluations). Of secondary interest was whether class ratings would be associated with students’ attendance and a comparison of variability among answers for undergraduate vs. graduate students. It was observed that class and teacher ratings were not related to students’ attendance and individual students did not tend to give the same answer for every question on their survey.     

Voice over distance: a case of podcasting for learning in online teacher education

This paper reports on a case study of an online pre-service teacher education course in a New Zealand university aimed at exploring the potential of student-generated podcasts as a form of interactive formative assessment at a distance. The study was part of a wider two-year funded project with the overall goal of documenting, developing, and disseminating effective and innovative e-learning practice. Findings from lecturer and tutor interviews, student focus group (FG) discussion, and course evaluations indicate that the podcasting task provided opportunities for the course lecturer, tutors, and students to learn and share ideas with one another. The experience empowered students to develop the skills and confidence to initiate more independent inquiry into technologies to support their pedagogical purposes. The study contributes to a better understanding of the skills, dispositions, and knowledge needed to prepare teacher candidates for teaching-learning contexts where information and communication technologies are increasingly pivotal.     

Reading achievement, metacognition, reading self-concept and interest: A study of German students in grades 3 and 4

This study examined the reading achievement, metacognitive knowledge related to reading and memory, reading self-concept, and interest in reading of 140 German school students in Grades 3 and 4. The results indicated significant correlations between the metacognitive variables, motivational variables, and amongst metacognitive and motivational variables, as well as with reading achievement variables. The examination of good and poor readers showed significant differences in metacognitive knowledge of reading and memory, reading self-concept, interest in reading, and in teacher judgements of reading achievement between the groups. Differences between Grade 3 and 4 students were found in their word decoding skills, metacognitive knowledge about reading and memory, and teacher judgements of reading achievement. Results of the regression analysis indicated that reading achievement was predicted by grade level, word decoding, and teacher evaluations of reading achievement. The causal modelling indicated that motivation and metacognition affect reading comprehension in different ways. The findings are discussed in terms of future research and their educational implications.     

THE ROLE OF THE TEACHER‐‐I

This study constitutes an assessment of the relation between the number of students leaving school (dropouts) and the magnitude of teacher life changes in conjunction with selected teacher performance variables in terms of pair‐wise and multiple correlation procedures.

Findings support the thesis that the selected teacher performance variables taken in conjunction with teacher life changes and the number of student dropouts are significantly associated.     

Determinants of teaching quality: What's important to students?

A method for using student evaluations to help faculty improve their teaching performance is presented. A survey of current methods of student evaluations of teaching identified a need to improve the statistical information obtained from these evaluations. An ordinary least squares framework is used to identify the factors that students feel are important in teacher and course ratings. This framework is used to estimate weights that students assign to various teacher and course attributes and to test whether students apply these weights consistently across teachers and courses. About 81 percent of the explained variation in teacher ratings was associated with attributes that contribute to student enjoyment of the learning process. Over 90 percent of the explained variation in course ratings was associated with attributes that measure how much a student learned in the course. Students were found to apply these attributes or weights consistently across teachers and courses. Implications for developing effective teaching strategies, faculty recruitment, and curriculum reform are discussed.     

Gender,ethnicity and teaching evaluations: Evidence from mixed teaching teams

This paper studies the effect of teacher gender and ethnicity on student evaluations of teaching at university. We analyze a unique data-set featuring mixed teaching teams and a diverse, multicultural, multi-ethnic group of students and teachers. Blended co-teaching allows us to study the link between student evaluations of teaching and teacher gender as well as ethnicity exploiting within course variation in a panel data model with course-year fixed effects. We document a negative effect of being a female teacher on student evaluations of teaching, which amounts to roughly one fourth of the sample standard deviation of teaching scores. Overall women are 11 percentage points less likely to attain the teaching evaluation cut-off for promotion to associate professor compared to men. The effect is robust to a host of co-variates such as course leadership, teacher experience and research quality, as well as an alternative teacher fixed effect specification. There is no evidence of a corresponding ethnicity effect. Our results are suggestive of a gender bias against female teachers and indicate that the use of teaching evaluations in hiring and promotion decisions may put female lectures at a disadvantage.     

Effect of sex,class standing,and academic performance on perceived importance of teacher behaviors

An implicit assumption made in most teaching evaluation instruments is that teaching behaviors are equally important to students. Using specific teacher behaviors which have appeared in a number of teaching assessment devices, the importance of teacher behaviors questionnaire (ITB) was constructed to measure students' perceptions of the importance of selected teacher behaviors. Data collected from 105 college students were utilized in the present study for the following purposes: (a) to determine if there are differences among students' ratings of teaching behaviors in terms of importance and (b) to determine if the ratings of importance given to selected teacher behaviors vary according to students' sex, class standing, or academic performance. Results of the study indicated that there were significant differences in perceived importance of selected teacher behaviors. Furthermore, it was found that ratings of some of these behaviors tended to vary across sex groups. Implications for utilizing importance information as a weighting component in teaching evaluations are discussed.     

Educational Research

Abstract

To assess the extent to which student evaluations of lecturers are accurate indicators of quality of instruction (teacher effectiveness), 128 students taking a multisection course in University Mathematics evaluated the performance of their particular lecturers. Initial tests indicated no difference among the students taking the different sections (same content) in either aptitude or expected performance. Hence, the fact that significantly positive correlations were obtained between average student performance and average student evaluations of their lecturers across the various sections suggests that student evaluations of this kind can be used as a measure of instructional effectiveness under appropriate conditions.     

Emotion in online college classrooms: examining the influence of perceived teacher communication behaviour on students’ emotional experiences

This research focused on teacher communication behaviour as an influential factor in students’ educational experiences. This study examined students’ perceptions of emotion (involving teachers’ emotional support, students’ emotional work and students’ positive emotional valence toward class and teacher) as influenced by a variety of predicting variables: perceptions of teacher affinity-seeking as well as teachers’ positive behaviour alteration techniques, verbal immediacy and teacher online presence (involving the three factors of instructional design, organisation facilitating discourse and direct instruction). Relying on survey data collected from undergraduate and graduate students across disciplines enrolled in course offerings at a large public university, quantitative analysis examined relationships among existing factors previously studied in educational and communication research. Although this study uncovered multiple significant relationships between variables in the data set, students’ perceptions of their teachers’ verbal immediacy and presence in their instructional design were found to be most predictive relative to students’ emotions in online classrooms. Ultimately this project addresses and emphasises the need to more fully examine students’ emotions and related social experiences with virtual teachers and course content in higher education.     

Class size and college students' evaluations of teachers and courses: A closer look

A systematic analysis of extant studies reveals, on average, a very weak inverse association between the size of class enrollment in a college course and students' overall evaluation of the course and its teacher, as well as between class size and evaluations of specific instructional dimensions pertaining to the instructor's skill in presenting material and communicating information. Larger inverse associations are typically found between class size and evaluations of specific instructional dimensions pertaining to the instructor's interactions and interrelationships with students. This pattern of findings is consistent with the accumulating evidence that the instructional dimensions of the first set have greater importance for students in forming their global opinions of teachers and courses than do those of the second set. The further finding in some studies of a negative curvilinear (roughly U-shaped) relationship between class size and evaluations is noted and discussed. Implications for the validity of teacher evaluations are considered and the related question of whether class size biases these evaluations is explored.     

Influences on peer evaluation in a group project: an exploration of leadership,demographics and course performance

Collaborative learning strategies are widely used in higher education to deepen learning, promote team-building skills and achieve course learning objectives. Using peer evaluation is an important strategy to ensure that engaged and active students are rewarded for their efforts, and to discourage loafing within groups. However, less is known about what biases may influence students’ peer evaluations. In this paper, we investigate what variables students may (consciously or unconsciously) use to evaluate their peers. We explore the role of sex, race, course performance and group leadership on peer evaluation. We also investigate whether these variables correlate with students’ final course grade. We found that students who reported being leaders in groups were evaluated higher than peers who reported being followers, and that course performance positively correlated with peer evaluations. White students received higher peer evaluations than students of colour. This difference reflects trends in group leadership and course performance, with more white students than students of colour reporting being leaders in groups and receiving higher grades.     

美国教师教育项目中教师品性评价的实施

为了培养优秀的教师候选人,美国教师教育项目开展了教师品性评价。该评价贯穿于教师候选人的整个学习和实习期间,旨在使教师候选人的品性在不断的反馈中得到充分发展。美国各高校教师教育项目对教师候选人的成功至关重要的品性评价内容进行界定。评价所使用的方法包括行为检核表、评价量表、访谈和品性反思文章。品性评价的实施涉及教师品性评价介绍课程、签订教师品性评价协议、四次主要评价、不定时的评价和评价结果的应用。教师品性的改进采用发挥教授和导师的模范作用、教师品性研讨会、品性改进课程和PDR-PREP方法等途径。     

Teacher support,student motivation,student need satisfaction,and college teacher course evaluations: testing a sequential path model

Self‐determination theory posits that individuals who have basic psychological needs satisfied while engaging in an activity will be more likely to value and persist in that activity. Scholars in this area have also posited that autonomy‐supportive social contexts are top‐down determinants of individual need satisfaction. To understand better the progression from social to motivational to outcome variables, we tested a four‐step path model within a classroom setting. Data were collected from 220 students in an introductory journalism course, which was subdivided into 14 lab sections. Students were surveyed regarding their feelings about the course, the autonomy support of the instructor, the motivation they felt to engage in the course, and the levels of need satisfaction they experienced. Theoretically‐guided structural equation modeling produced a best‐fitting model in which teacher autonomy support predicted more self‐determined student motivation, which along with teacher autonomy support predicted greater student psychological need satisfaction, which led finally to higher predicted course grades and higher teacher‐course evaluations. Overall, the results replicate and significantly extend previous findings regarding need satisfaction and teacher‐course evaluations. Implications for pedagogy and educational interventions are discussed.     

Mediating effects of teacher enthusiasm and peer enthusiasm on students’ interest in the college classroom

This study explored the mediating effects of students’ perceptions of teacher enthusiasm and of peer enthusiasm on the relation between students’ initial interest and their situational interest at the end of the semester. Students’ motivation for affiliation with their teacher and with peers was also measured to allow for potential associations between these variables and the students’ perceptions of teacher and peer enthusiasm. In path analysis, data from 455 undergraduates showed that perceptions of teacher enthusiasm and of peer enthusiasm had direct associations with two kinds of situational interest, both hold and catch interest. Also, the relations between initial interest and both hold and catch interest were mediated by perceptions of peer enthusiasm but not teacher enthusiasm. Students’ affiliative motivation with the teacher and peers had direct effects on their perceptions of teacher and peer enthusiasm. This study contributes to clarifying the importance of contextual factors as well as students’ own individual variables in understanding the mechanisms by which students’ interest in a course develops and is maintained throughout the semester.