Ratings of university teacher instruction: how much do student and course characteristics really matter?

Several student and course characteristics were examined in relation to student ratings of instruction. Students at a major Canadian university completed the Universal Student Ratings of Instruction instrument at the end of every course over a three‐year period, providing 371,131 student ratings. Analyses of between‐group differences indicate that students who attend class often and expect high grades provide high ratings of their instructors (p < .001). In addition, lab‐type courses receive higher ratings than lectures or tutorials, and courses in the social sciences receive higher ratings than courses in the natural sciences (p < .001). Regression analyses indicated, however, that student and course characteristics explain little variance in student ratings of their instructors (<7%). It is concluded that student ratings are more related to teaching instruction and behavior of the instructor than to these variables.     

Predicting student ratings: the relationship between actual student ratings and instructors’ predictions

This study examined the relationship between student ratings and instructors’ predictions of these ratings, taking into account other instructor, student, and course characteristics. Participants in the study were 198 instructors in the School of Education at a major teacher training college in Israel. Data corresponding to one randomly selected course per instructor were collected using student and instructor questionnaires and college records. Results indicate a systematic positive relationship between instructors’ predictions and actual student ratings with respect to overall ratings and the ratings of three dimensions of teaching. Results also demonstrate a systematic trend whereby low‐rated instructors tend to overestimate their student ratings, high‐rated instructors underestimated ratings, and moderately rated instructors gave accurate predictions. Results have implications for using predictions to motivate teaching improvement.     

Similarity of student ratings across instructors,courses, and time

This study raised three questions about the similarity or generalizability of student ratings of courses and instructors. First, how stable are student ratings of the same instructor giving the same course during twodifferent semesters? Second, how similar are student ratings of the same instructor in twodifferent courses? Third, how similar are student ratings of a given course being taught bydifferent instructors? Instances were identified in which student ratings on seven different factors were available for pairs of courses for each of these questions. For the case of the same instructor — same course—different semesters, student ratings were reasonably similar (median r for seven factors about 0.70). For the case of the same instructor—different courses, the median r was surprisingly low — about 0.40. For the case of the same course—different instructors, substantial correlations were obtained for some factors and insignificant correlations for other factors. Implications of these findings for practical use of student ratings and suggestions for further research in the area are discussed.     

Student ratings and various instructional variables from a within-instructor perspective

The relationship between class size, instructional method, course level, reason for enrollment, and student ratings of instruction was assessed from a within-instructor perspective. Two hundred fifty-four pairs of courses taught by the same instructor were correspondingly identified and subjected to a stepwise multiple regression procedure. Only class size was found to be a significant predictor of ratings once individual differences between instructors were controlled, hence underlining the importance of (1) taking cognizance of the size of the course when using student ratings of instructors as a measure of teaching effectiveness, and (2) controlling for systematic variation due to instructor idiosyncracies in instructional research.     

The generalizability of student ratings of instructors: Item specificity and section effects

A number of recent studies have used generalizability theory to examine the dependability of student ratings of instruction. This study extends this line of research by examining the consistency of ratings between different sections of a course taught in a given semester by the same instructor, and by comparing the performance of global- and attribute-type instructor rating items. Five samples of physics instructors, varying in size from 5 to 12 instructors, were rated by their students on a form containing two global and eight attribute items. Each instructor taught two sections of a course. The study found that the section effect was small (ratings of instructors were consistent across different sections of the same course), and that the generalizability of ratings was substantially influenced by item specificity. For summary purposes, one global item seemed sufficient.     

MULTILEVEL ANALYSIS OF THE EFFECTS OF STUDENT AND INSTRUCTOR/COURSE CHARACTERISTICS ON STUDENT RATINGS

Multilevel SEM was used to examine the extent to which student, instructor, and course characteristics affect student ratings. Data were gathered from 1867 students enrolled in 117 courses at a large teacher training college in Israel. Four alternative two-level models that differ in only the nature of the relationship among interest in the course subject, expected grade, and student ratings were tested. Two of the models were judged as less appropriate, one because it failed to support the spurious relationship assumed between expected grade and student ratings, and the other on grounds of poor model-data fit. The other two models were equally good both in terms of the model-data fit and the amount of variance in student ratings that is accounted for by each of them. Both models supported the mediation effect of expected grade in the relationship between interest in the course subject and student ratings.     

Students' evaluation and instructors' self-evaluation of university instruction

The purpose of the study was to investigate the relationship between students' evaluations of university instruction and self-ratings of instructors. The sample consisted of 52 instructors, from the School of Education at Tel-Aviv University, who taught 93 classes. The instructors, as well as their students, responded to a 20-item instructional-practices questionnaire. Instructor self-ratings had only a modest relationship with the ratings given by students (a median correlation of .28). Discrepancies between instructor ratings and ratings given by the students were further analyzed for (1) varying training in teaching - no difference was found; and (2) number of years of teaching experience - differences were noted; the self-ratings of less experienced instructors were closer to student ratings.     

Course ratings as measures of instructional effectiveness

This study assesses the validity of instructor ratings as measures of instructional effectiveness. Multiple regression analysis was used to assess the relationship between ratings of classroom practices, instructor characteristics and student commitment to the course and an overall rating of the instructor. Sixty-four percent of the variance in instructor ratings could be explained by variance in students' ratings for eleven course-related items. It was concluded that variance in instructor ratings reflected variance in classroom and instructional practices.     

Following student ratings over time with a catalog-based system

First, the longitudinal nature of student ratings of instructors has not received deserved emphasis from researchers. Second, the use of item banks for designing student rating questionnaires, especially for instructor feedback, has needed attention. These two factors are investigated in this study, which tracks 304 instructors over a four semester period. It was found that the type of questionnaire generated from the item bank led to statistically significant differences among designated groups. The longitudinal analysis, however, indicated only minor improvement over time, regardless of whether or not an instructor chose to use items yielding specific feedback on the instructional components of a course. Additionally, although main effect differences were noted between teaching assistants and regular faculty, other results were very similar.     

Similarities and Differences in the Play Activities of Children in Two Public Schools with Contrasting Environment

Based on previous work by H. G. Murray, this study examined relationships between overt classroom teaching behaviors of college instructors and student ratings of teaching effectiveness. One hundred seven students who enrolled in six sections of an introductory psychology course rated instructors on 12 measures of teaching effectiveness and reported the frequencies of 20 observable, inclass teaching behaviors. Relationships between teaching behaviors and student ratings were examined by simple, multiple, and canonical correlation procedures. A complex pattern of relationships emerged. The canonical analysis revealed a two-dimensional structure consisting of instructor clarity and student-teacher rapport dimensions. Teaching behaviors loading on these dimensions were examined.     

Discrepancy between instructor and student evaluations of instruction: Effect on instructor

To find out whether a discrepancy between the instructor's and the student's evaluations of teaching influence teaching, 13 introductory and educational psychology instructors and their students were given a Student Opinion Questionnaire (SOQ) twice: on the fourth week of a fall term and eight weeks after feedback sessions with the instructors. The instructors received feedback on the direction and amount of initial discrepancy. The results showed that the unfavorable discrepant instructors (instructor rating better than students) changed more on skill, feedback, rapport, general teaching ability, and the overall value of the course than the favorably discrepant instructors (student ratings better than instructor). The unfavorably discrepant instructors improved their teaching significantly more than the favorably discrepant instructors.This article is based on portions of a dissertation submitted in partial fulfillment of the requirements for the Ph. D. degree at the University of Michigan. The author wishes to acknowledge the help given by Wilbert J. McKeachie and Alvin F. Zander and thank them for their suggestions. An earlier version of the paper was presented at the 81st Annual Convention of the American Psychological Association, Montreal, Que., August 1973.     

Instructional practices in reformed undergraduate STEM learning environments: a study of instructor and student behaviors in biology courses

ABSTRACT

Theoretically, organisational culture, instructor training, and learning space design influence how faculty teach STEM courses. Previous studies have used classroom observation protocols to characterise the range of teaching practices in mostly teacher-centered, traditional STEM classrooms. In this study, we examined the classroom behaviour of 13 STEM faculty teaching biology courses in a reformed undergraduate STEM learning environment. Our findings indicate that instructors teaching in this reformed environment guided student learning (58.4?±?1.9%) almost three times more than they presented information (20.0?±?2.2%). Students worked individually or in groups and talked to the whole class (57.1?±?1.8%) 1.5 times more than they received information (35.5?±?1.9%). We found significant positive correlation between ‘instructor presenting’ and ‘students receiving’ information (r?=?0.743, p?=?1.4?×?10^?4) and ‘instructor guiding’ and ‘student working and talking’ in class (r?=?0.605, p?=?7.2?×?10^?5), suggesting that instructors can change their own classroom behaviours and expect concurrent change in their students’ behaviours. Finally, sequencing teaching practices in high active-engagement classrooms showed instructors move and guide student group work and lead whole class discussions before lecturing to students, which could lead to deeper learning of conceptual knowledge. We discuss insights from these findings that have implications for acculturating evidence-based teaching practices in STEM departments.     

The Effects of Different Instructions on Student Ratings of University Courses and Teachers

Abstract

The purposes of the present study were to investigate the influence of three sets of instructions, class level, and academic rank on teacher/course evaluation by student raters. Students did not differ in their teacher/course evaluation ratings when the instructions specified the evaluation results would be used: (a) only by the instructor, (b) by the administration, or (c) by students for course selection purposes. The evaluation of graduate courses did not differ from that of undergraduate courses. A statistical difference was found between the academic ranks examined. Specifically, graduate teaching assistants received higher ratings than did either assistant or full professors.     

Assessment & Evaluation in Higher Education Using the margin of error statistic to examine the effects of aggregating student evaluations of teaching

We proposed an extended form of the Govindarajulu and Barnett margin of error (MOE) equation and used it with an analysis of variance experimental design to examine the effects of aggregating student evaluations of teaching (SET) ratings on the MOE statistic. The interpretative validity of SET ratings can be questioned when the number of students enrolled in a course is low or when the response rate is low. A possible method of improving interpretative validity is to aggregate SET ratings data from two or more courses taught by the same instructor. Based on non-parametric comparisons of the generated MOE, we found that aggregating course evaluation data from two courses reduced the MOE in most cases. However, significant improvement was only achieved when combining course evaluation data for the same instructor for the same course. Significance did not hold when combining data from different courses. We discuss the implications of our findings and provide recommendations for practice.     

EDUCATIONISTS AND ACADEMICS: RATINGS OF COMMUNITY COLLEGE INSTRUCTORS

Relationships between student ratings, instructor pedagogical training, subject‐matter preparation in teaching field, and length of teaching experience were examined for community college university transfer instructors. Moderate but statistically signficant relationships were found between pedagogical training, amount of subject‐matter preparation in teaching area, and the rating subscale of Course Organization and Planning. Results indicated that students tend to give higher ratings to instructors with pedagogical training, while instructors with greater amounts of graduate‐level, subject‐matter preparation tend to receive lower ratings.     

Student evaluations of teaching: effects of the Big Five personality traits,grades and the validity hypothesis

The purpose of the current study was to examine whether the Big Five personality traits and expected student grades relate to student evaluations of teachers and courses at the college level. Extraversion, openness, agreeableness and conscientiousness were found to be personality traits favoured in instructors, whereas neuroticism was not. A significant correlation was found between the students’ expected grades in the course and student evaluations of the course, but not the evaluations of the instructor. When the effect of students’ perceived amount of learning was taken into account, no significant effect of grades was found on teacher ratings. Personality explained variance in teacher and course evaluations over and above grades and perceived learning.     

Understanding Student Evaluations of Teaching Quality: The Contributions of Class Attendance

Institutions of higher learning employ compulsory attendance policies with the expectation that these mandates enhance students' academic performance and perceptions of course quality. However, numerous empirical investigations demonstrate equivocal and often contradictory findings regarding the relationship between attendance and various markers of student achievement. The present investigation extends this research by exploring the utility of student ratings of the need to attend class in predicting their perceptions of teaching excellence after controlling for class size, instructor availability, and small-group interactions. As hypothesized, ratings of the need to attend predicted excellence while accounting for a significant 5.3% of the variance. Discussion and conclusions highlight the utility of class attendance in understanding students' evaluations of teaching and course quality.     

Effects of graduate teaching assistant attire on student learning,misbehaviors, and ratings of instruction

This study examined the influence of teaching assistant attire in the university classroom. Significant relationships were found between instructor attire and student cognitive learning, affective learning, and ratings of instruction. Analysis of variance indicated significant differences in these variables by levels of teaching assistant dress. Significant negative relationship was found between instructor attire and student likelihood of misbehavior. Analysis of variance revealed that student misbehaviors were less likely for teaching assistants with high professional dress.