Student knowledgeability,student sex,class size,and class level: Their interactions and influences on student ratings of instruction

Student knowledgeability, class size, and class level were found to significantly influence students' ratings of instruction. In general, the more knowledgeable the student in an area, the higher his ratings of courses and instructors in that area. Also, large courses and advanced courses were most highly rated by students. The effect of student sex on student ratings of instruction varied as a function of the particular aspect of instruction being evaluated. Significant interactions among the four main effects were also found across the judgmental dimensions students utilized in evaluating instruction, as assessed by factor analysis. Student knowledgeability and class size were found to be the main predictors of student ratings on these dimensions.     

Grading leniency, grade discrepancy, and student ratings of instruction

The purpose of this study was to examine how grading leniency and grade discrepancy (the difference between expected grades and deserved grades) were associated with various dimensions of student ratings of instruction. A sample of 754 undergraduate college students completed a student ratings of instruction instrument and provided responses to a number of other questions on topics such as course difficulty and workload. A series of multilevel regression analyses were conducted and results showed that an instructor's grading leniency, as perceived by students, was positively associated with student ratings on 11 of 12 dimensions of instruction examined. This finding suggests that more lenient instructors tend to receive higher student ratings. The second finding shows that grade discrepancy was negatively associated with most dimensions of instruction. This supports the self-serving bias hypothesis under attribution theory (Gigliotti & Buchtel, 1990) in that students tended to punish instructors with lower ratings when expected grades were lower than students believed they deserved, yet little evidence of a pattern of rewards existed in student ratings when students expected grades higher than they deserved.     

Perceived autonomy support,intrinsic motivation,and student ratings of instruction

Motivation theory suggests that autonomy supportiveness in instruction often leads to many positive outcomes in the classroom, such as higher levels of intrinsic motivation and engagement. The purpose of this study was to determine whether perceived autonomy support and course-related intrinsic motivation in college classrooms positively predict student ratings of instruction. Data were collected from 47 undergraduate education courses and 914 students. Consistent with expectations, the results indicated that both intrinsic motivation and autonomy support were positively associated with multiple dimensions of student ratings of instruction. Results also showed that intrinsic motivation moderated the association between autonomy support and instructional ratings—the higher intrinsic motivation, the less predictive autonomy support, and the lower intrinsic motivation, the more predictive autonomy support. These results suggest that incorporating classroom activities that engender autonomy support may lead to improved student perceptions of classroom instruction and may also enhance both student motivation and learning.     

Student ratings and various instructional variables from a within-instructor perspective

The relationship between class size, instructional method, course level, reason for enrollment, and student ratings of instruction was assessed from a within-instructor perspective. Two hundred fifty-four pairs of courses taught by the same instructor were correspondingly identified and subjected to a stepwise multiple regression procedure. Only class size was found to be a significant predictor of ratings once individual differences between instructors were controlled, hence underlining the importance of (1) taking cognizance of the size of the course when using student ratings of instructors as a measure of teaching effectiveness, and (2) controlling for systematic variation due to instructor idiosyncracies in instructional research.     

Faculty Views of Student Evaluation of College Teaching

The literature abounds with psychometric studies of course evaluation measures and articles debating the merits of student ratings of instruction, but little research has focused on faculty perceptions of this procedure. In the present study faculty perceptions are explored at a teachers' college where evaluation is carried out annually on a sample of courses. The sample includes 101 instructors who completed the research questionnaire. Faculty attitudes reflected a broad range of responses towards validity of student ratings, and their usefulness for improving instruction. Although overall attitudes were mildly positive, few instructors reported changing instruction as a result of student ratings. Moreover, few supported sending evaluation results directly to college administrators or publishing them for student consumption.     

Students' perceptions of teaching skills and overall effectiveness across instructional settings

This study examines the differences in students' perceptions of the amount of improvement in dimensions of teaching skills across levels of instruction, class sizes, and disciplines. The relationship between factor scores on the dimensions of teaching and students' overall ratings are also analyzed across instructional settings.Student ratings were collected in 2,816 classes in a Faculty of Commerce over three semesters. Using class means as a unit of analysis, the data were factor-analyzed. Factor scores were used as dependent variables in examining differences across course characteristics. Factor scores were further used to predict ratings of the overall quality of instruction. These predictions were subsequently compared across instructional settings.In the combined sample, differences in students' perceptions were found across levels of instruction for all factors, and across class size for two factors. When analyses were conducted on these variables within separate disciplines, the results varied. The relationship between factor scores on the dimensions of teaching and students' overall ratings varied across instructional settings.The results are discussed in relation to previous research on the dimensions of teaching and in relation to the concept of students' normative assumptions about teaching behavior. Some practical implications of the results are also described.     

Some effects of training on teaching effectiveness of untrained university teaching assistants

This paper presents an assessment of the effects which a brief training program had on teaching effectiveness of graduate teaching assistants (TAs). Twenty-two inexperienced and previously untrained university TAs from economics, geography, and business administration were assigned to a training or control group by a stratified random method with stratification based on TA departmental affiliation. Teaching experts rated two videotapes of each Ta's university class, one tape made before training and one following training. Ratings were obtained on two factors: (1) planning instruction to meet clear goals and organizing meaningful content in a logical fashion, and (2) involving students in instruction. Results from analyses of covariance indicate that the training group received significantly higher final ratings than the control group on the total score and on each of the two factors when final scores were adjusted for group differences in initial ratings. Teaching experience alone did not result in significantly higher ratings for control group TAs. Participants in training evaluated most topics and the overall program favorably both immediately after training and one semester later.     

Behavioral Attention: A Longitudinal Study of Whether and How It Influences the Development of Word Reading and Reading Comprehension Among At-Risk Readers

Abstract

The purpose of this study was to examine the extent to which teacher ratings of behavioral attention predicted responsiveness to word reading instruction in first-grade and third-grade reading comprehension performance. Participants were 110 first-grade students identified as at risk for reading difficulties who received 20 weeks of intensive reading intervention in combination with classroom reading instruction. Path analysis indicated that teacher ratings of student attention significantly predicted students’ word reading growth in first grade even when they were competed against other relevant predictors (phonological awareness, nonword reading, sight word efficiency, vocabulary, listening comprehension, hyperactivity, nonverbal reasoning, and short-term memory). Also, student attention demonstrated a significant indirect effect on third-grade reading comprehension via word reading but not via listening comprehension. Results suggest that student attention (indexed by teacher ratings) is an important predictor of at-risk readers’ responsiveness to reading instruction in first grade and that first-grade reading growth mediates the relationship between students’ attention and their future level of reading comprehension. The importance of considering ways to manage and improve behavioral attention when implementing reading instruction is discussed.     

Correlations,trends and potential biases among publicly accessible web-based student evaluations of teaching: a large-scale study of RateMyProfessors.com data

Student evaluations of teaching are widely adopted across academic institutions, but there are many underlying trends and biases that can influence their interpretation. Publicly accessible web-based student evaluations of teaching are of particular relevance, due to their widespread use by students in the course selection process and the quantity of data available for analysis. In this study, data from the most popular of these websites, RateMyProfessors.com, is analysed for correlations between measures of instruction quality, easiness, physical attractiveness, discipline and gender. This study of 7,882,980 RateMyProfessors ratings (from 190,006 US professors with at least 20 student ratings) provides further insight into student perceptions of academic instruction and possible variables in student evaluations. Positive correlations were observed between ratings of instruction quality and easiness, as well as between instruction quality and attractiveness. On average, professors in science and engineering disciplines have lower ratings than in the humanities and arts. When looking at RateMyProfessors as a whole, the effect of a professor’s gender on rating criteria is small but statistically significant. When analysing the data as a function of discipline, however, the effects of gender are significantly more pronounced, albeit more complex. The potential implications are discussed.     

Ratings of university teacher instruction: how much do student and course characteristics really matter?

Several student and course characteristics were examined in relation to student ratings of instruction. Students at a major Canadian university completed the Universal Student Ratings of Instruction instrument at the end of every course over a three‐year period, providing 371,131 student ratings. Analyses of between‐group differences indicate that students who attend class often and expect high grades provide high ratings of their instructors (p < .001). In addition, lab‐type courses receive higher ratings than lectures or tutorials, and courses in the social sciences receive higher ratings than courses in the natural sciences (p < .001). Regression analyses indicated, however, that student and course characteristics explain little variance in student ratings of their instructors (<7%). It is concluded that student ratings are more related to teaching instruction and behavior of the instructor than to these variables.     

Evaluating the quality of middle school mathematics teachers,using videos rated by college students

Using experts in teaching to evaluate pre-college mathematics teachers is both time consuming and costly. This study examines the potential of letting undergraduate mathematics students perform this task, comparing their ratings of two dimensions, richness of instruction and mathematical correctness, to those previously assigned by an expert. Using 85 undergraduates of two U.S. institutions, who independently watched short videos of teachers, we found that student ratings of teachers’ correctness were a good match to the expert ratings; student ratings of richness of instruction, less so. A "halo effect" was observed in that students did not fully differentiate between richness and correctness in their ratings. Moreover, male students gave harsher ratings than did female students. Whereas undergraduates showed promise in accurately evaluating teachers of younger students, improvements in terms of rating items, attention to bias, and explicit training on the teaching dimensions to be rated should be addressed.     

A cross-cultural comparison of instructor communication in American and French classrooms

This study addressed patterns and influence of instructor communication in American (US) and in French classrooms. Significant differences were found for instructor use of power bases, affinity-seeking strategies, and nonverbal immediacy. Significant differences were found in student affective learning, cognitive learning, affect toward instructor, and ratings of instruction between the two cultures. In both cultures, however, student affect toward instructor, cognitive learning, and rating of instruction were significantly correlated with instructor affinity-seeking and instructor nonverbal immediacy. Likewise, instructor use of expert power was significantly correlated with student cognitive learning and ratings of instruction in both cultures.     

The detection and correction of bias in student ratings of instruction

With the use of surveys of instructional effectiveness that use Likert rating scales, bias is a potential threat to the validity of interpretations. Simple summation of ratings or the use of larger samples are not methods for removing bias. In this study, a new model for scaling ratings is examined. The method both identifies and corrects for bias. Working with a database of student ratings of college instruction, the model was tested in terms of a variety of criteria. Results indicated that bias was detected and that it was large enough to warrant our concern. The statistical corrections were significant both in terms of order and magnitude of class means. Implications for future studies include the specification of more potential sources of bias, the interaction of some of these factors, and the development of more systematic evidence supporting the need to be attentive to bias. The many-faceted Rasch model used in this study needs more evaluation before we are convinced of its utility to study and correct for bias, but preliminary evidence is encouraging. Recommendations were offered for a theoretical rationale for studying bias in student ratings of instructional effectiveness and a program of research leading to the use of this model for reporting results for use in improving instruction and for promotion, tenure, and merit decisions. An earlier version of this paper was presented at the annual meeting of the American Educational Research Association, Atlanta April 1993.     

Improvement of instruction and teacher effectiveness: are student ratings reliable and valid?

This paper reviews the evidence on whether student ratings are reliable and valid enough to be used for the purpose of the improvement of instruction and teacher effectiveness. A consensus of evidence available in the literature tends to show a measure of consistency, stability and validity of student ratings. Several variables such as sex of rater and ratee, class size, mood of students, rank of instructors, grades students were expecting, time of the day courses are taught, to mention only a few, have been found to have a low to high positive relationship, with student ratings. It also appears that the use of student ratings leads to the improvement of instruction, provided the evaluation data are fed back to the instructor and that an expert or consultant provides assistance to the instructor.     

The effect of instructional quality on low‐ and high‐performing students

The study assessed the effects of the quality of reading and math instruction and classroom climate on the academic skills and engagement of 314 children in 245 classrooms at the end of third grade. All of the children in the study were from families with low incomes. On a classroom observation measure developed for the study, regression analyses revealed that relatively high ratings on the reading and math instruction subscales, which were scored after a half‐day observation, predicted better reading and math achievement test scores, respectively, for third graders who had previously had poor academic performance, but did not have a significant effect for relatively high‐performing students. High teacher ratings on the reading instruction and classroom climate observation scales predicted high levels of student engagement. The findings suggest the importance of high‐quality teaching for economically disadvantaged children who have poor academic skills.     

EFFECT OF CLASS SIZE,GRADES GIVEN,AND ACADEMIC FIELD ON STUDENT OPINION OF INSTRUCTION

This study reviewed the effect of class size, grades given, and academic field on student opinion of instruction. Data analysis showed that there were no significant correlations between variables of the three groups: (a) class size and student opinion of instruction; (b) grades given and student opinion of instruction, and (c) college and national academic field rankings and student opinion of instruction. These results leave teacher effectiveness as the most likely variable to explain ratings of student opinion of instruction.     

The Great Therapists Program: Computerized Learning of Counseling Theories

The authors examine the effectiveness and subjective enjoyment of a computer-assisted instruction program to teach neophyte counselors classic methods of psychotherapy. Software was developed that called for beginning counseling students to simulate dialogue with a client via the computer's keyboard. Students were able to compare their responses with those of classic therapists. Provisions for a printed record of the session were made. Questionnaires, knowledge tests, and students' choices of counseling styles provided investigators with feedback on learning and level of positive response to the teaching method. As with most computer-assisted instruction studies, no evidence supporting increases in learning was obtained; however, free responses and ratings indicated that the students positively evaluated the program as a learning experience.     

Regular Feedback from Student Ratings of Instruction: Do College Teachers Improve their Ratings in the Long Run?

The authors examined whether feedback from student ratings of instruction not augmented with consultation helps college teachers to improve their student ratings on a long-term basis. The study reported was conducted in an institution where no previous teaching-effectiveness evaluations had taken place. At the end of each of four consecutive semesters, student ratings were assessed and teachers were provided with feedback. Data from 3122 questionnaires evaluating 12 teachers were analyzed using polynomial and piecewise random coefficient models. Results revealed that student ratings increased from the no-feedback baseline semester to the second semester and then gradually decreased from the second to the fourth semester, although feedback was provided after each semester. The findings suggest that student ratings not augmented with consultation are far less effective than typically assumed when considered from a long-term perspective.     

Students' evaluation and instructors' self-evaluation of university instruction

The purpose of the study was to investigate the relationship between students' evaluations of university instruction and self-ratings of instructors. The sample consisted of 52 instructors, from the School of Education at Tel-Aviv University, who taught 93 classes. The instructors, as well as their students, responded to a 20-item instructional-practices questionnaire. Instructor self-ratings had only a modest relationship with the ratings given by students (a median correlation of .28). Discrepancies between instructor ratings and ratings given by the students were further analyzed for (1) varying training in teaching - no difference was found; and (2) number of years of teaching experience - differences were noted; the self-ratings of less experienced instructors were closer to student ratings.