Comparison of online and classroom-based student evaluations of instruction

In recent years, colleges have been moving from traditional, classroom‐based student evaluations of instruction to online evaluations. Because of the importance of these evaluations in decisions regarding retention, promotion and tenure, instructors are justifiably concerned about how this trend might affect their ratings. We recruited faculty members who were teaching two or more sections of the same course in a single semester and assigned at least one section to receive online evaluations and the other section(s) to receive classroom evaluations. We hypothesised that the online evaluations would yield a lower response rate than the classroom administration. We also predicted that there would be no significant differences in the overall ratings, the number of written comments, and the valence (positive/neutral/negative) of students’ comments. A total of 32 instructors participated in the study over two semesters, providing evaluation data from 2057 students. As expected, online evaluations had a significantly lower response rate than classroom evaluations. Additionally, there were no differences in the mean ratings, the percentage of students who provided written comments or the proportion of comments in the three valence categories. Thus, even with the lower response rate for online evaluations, the two administration formats seemed to produce comparable data.     

Relationship of changes in student motivation to student evaluations of instruction

Research has demonstrated that student evaluations of instruction are influenced by variables extraneous to the instructional procedures being evaluated. One of the most important of these is the student's motivation to take the course. The Instructional Development and Effectiveness Assessment (IDEA) system controls this variable by comparing a course evaluation to a norm group of courses having students with similar motivation. The present study examined the possibility that the IDEA procedure of having students rate their precourse motivation at theend of a course might be unacceptable, because the rating would be influenced by experiences within the course itself. The data indicated that postcourse ratings of precourse motivation do deviate somewhat from actual precourse ratings, but the deviation is not of an order of magnitude which would seriously distort the interpretation of the ratings.     

Do grades contaminate student evaluations of instruction?

The correlation between grades and student satisfaction has been interpreted as providing support for belief in a grading leniency bias hypothesis. That is, easy graders are assumed to receive better evaluations than hard gradersbecause they are easy graders. Howard and Maxwell have demonstrated that the relationship between grades and satisfaction might be viewed as an expected result of important causal relationships of other variables (student motivation and progress in the course) with satisfaction and grades, rather than simply evidence of contamination due to grading leniency. Eighty-three students in a research methodology course provided data at two points in a semester. Cross-lagged panel correlation analysis was employed to ascertain the direction of casuality in the relationship between student satisfaction and grades. The findings replicate the Howard and Maxwell path analytic results in finding no evidence that a grades-influencing-satisfaction interpretation is more likely than its opposite, namely, a satisfaction-causing-grades one. The weak relationship between grades and student satisfaction, and the study's inability to find evidence to impugn a satisfaction-causing-grades interpretation of that relationship, renders the grade contamination objection to student evaluations even less poignant.     

Relationship with instructor and other variables influencing student evaluations of instruction

This study examined some of the factors which influence college students’ evaluations of their instructors. Data were collected from 557 students enrolled in a basic speech communication course. Overall, a student's relationship with the instructor was the best predictor of the student's evaluation of the instructor. This study lends partial support to the idea that students evaluate male and female instructors differently. The variables examined in this study were better predictors of student evaluation of female than male instructors.     

Discrepancy between instructor and student evaluations of instruction: Effect on instructor

To find out whether a discrepancy between the instructor's and the student's evaluations of teaching influence teaching, 13 introductory and educational psychology instructors and their students were given a Student Opinion Questionnaire (SOQ) twice: on the fourth week of a fall term and eight weeks after feedback sessions with the instructors. The instructors received feedback on the direction and amount of initial discrepancy. The results showed that the unfavorable discrepant instructors (instructor rating better than students) changed more on skill, feedback, rapport, general teaching ability, and the overall value of the course than the favorably discrepant instructors (student ratings better than instructor). The unfavorably discrepant instructors improved their teaching significantly more than the favorably discrepant instructors.This article is based on portions of a dissertation submitted in partial fulfillment of the requirements for the Ph. D. degree at the University of Michigan. The author wishes to acknowledge the help given by Wilbert J. McKeachie and Alvin F. Zander and thank them for their suggestions. An earlier version of the paper was presented at the 81st Annual Convention of the American Psychological Association, Montreal, Que., August 1973.     

Exploring the beliefs surrounding student evaluations of instruction in criminology and criminal justice undergraduate courses

Student evaluations of instructors are given consideration in a number of important decisions concerning instructors such as hiring, merit pay, and in the tenure and promotion process. However, a number of popular beliefs surround the interpretation of these evaluations. Using student evaluations for the three research-oriented universities in the state of Florida as a database, a number of these beliefs are explored. Specifically, the effects of instructor attributes (such as gender, rank and grades) and course characteristics (such as class size, type, number of course meetings, and time of day offered) on the end of the semester student evaluation rankings are discussed.     

Expectancy violation and student rating of instruction

This study examined whether violations that are incongruent with student expectations are significantly different than congruent violations of expectancy in relation to student ratings of instruction. Analysis using the Scheffe' post ANOVA test revealed that college students having high expectations/high experiences evaluated teachers more favorably than students with low expectations/high experiences, low expectations/low experiences and high expectations/low experiences. Reasons why these findings did not coincide with the expectancy violation model are offered.     

Do examinations influence student evaluations?

This paper measures the impact of timing on student evaluations of teaching effectiveness, using a dataset of close to 3000 observations from Erasmus School of Economics. A special feature of the data is that students were able to complete on-line questionnaires during a time window ranging from one week before to one week after the final examination. This allows for the isolation of the effect of the examination on student evaluations. Among students who subsequently pass the exam, we find little difference between pre- and post-exam ratings. Among students who fail, evaluation scores are significantly lower after the exam on a number of items. Our evidence is compatible with a self-serving bias in student evaluations, but does not indicate that students seek revenge on instructors through lower ratings.     

Expectancy theory outcomes and student evaluations of teaching

As student evaluation of teaching (SET) instruments are increasingly administered online, research has found that the response rates have dropped significantly. Validity concerns have necessitated research that explores student motivation for completing SETs. This study uses Vroom's [(1964). Work and motivation (3rd ed.). New York, NY: John Wiley & Sons] expectancy theory to frame student focus group responses regarding their motivations for completing and not completing paper and online SETs. Results show that students consider the following outcomes when deciding whether to complete SETs: (a) course improvement, (b) appropriate instructor tenure and promotion, (c) accurate instructor ratings are available to students, (d) spending reasonable amount of time on SETs, (e) retaining anonymity, (f) avoiding social scrutiny, (g) earning points and releasing grades, and (h) being a good university citizen. Results show that the lower online response rate is largely due to students’ differing feelings of obligation in the 2 formats. Students also noted that in certain situations, students often answer SETs insincerely.     

Causes and consequences of student evaluation of instruction

Student evaluation of course quality was examined as an intervening variable in a path model with seven input variables and the following two output variables: (1) whether students would recommend the course to others, and (2) students' plans to take more courses in the discipline. Student evaluation of the lecturer was the most powerful input variable for the intervening variable course rating. Class format, defined as lecture or lecture with small groups, did not affect course ratingper se but did relate to students' attitudes toward taking more courses in the discipline. Lecturer and textbook evaluation had significant independent effects on whether the student would recommend the course to others. The major implication for university departments is that quality of instruction in introductory courses has impact on future enrollment plans.     

大学生学科竞赛模式的改革与实践

立足民族院校实情,结合学院办学目标,探索新的学科竞赛模式:从平台体系、保障及运行机制、与教学改革的联动创新机制3个方面入手,打造融普及、提高为一体的多层次学科竞赛平台。通过7年实践,人才培养效应日益凸显,一大批拔尖创新人才脱颖而出,教学改革成效显著,为民族高校大学生创新竞赛和高素质创新人才培养探索出了一条特色之路。     

Sources of variance in end-of-course student evaluations

Abstract

Evaluation of college instructors often centers on course ratings; however, there is little evidence that these ratings only reflect teaching. The purpose of this study was to assess the relative importance of three facets of course ratings: instructor, course and occasion. We sampled 2,459 fully-crossed dyads from a large university where two instructors taught the same two courses at least twice in a 3-year period. Generalizability theory was used to estimate unconfounded variance components for instructor, course and occasion, as well as their interactions. Meta-analysis was used to summarize those estimates. Results indicated that a three-way interaction between instructor, course and occasion that includes measurement error accounted for the most variance in student ratings (24%), with instructor accounting for the second largest amount (22%). While instructor - and presumably teaching - accounted for substantial variance in student course ratings, factors other than instructor quality had a larger influence on student ratings.     

The generalizability of student ratings of instructors: Item specificity and section effects

A number of recent studies have used generalizability theory to examine the dependability of student ratings of instruction. This study extends this line of research by examining the consistency of ratings between different sections of a course taught in a given semester by the same instructor, and by comparing the performance of global- and attribute-type instructor rating items. Five samples of physics instructors, varying in size from 5 to 12 instructors, were rated by their students on a form containing two global and eight attribute items. Each instructor taught two sections of a course. The study found that the section effect was small (ratings of instructors were consistent across different sections of the same course), and that the generalizability of ratings was substantially influenced by item specificity. For summary purposes, one global item seemed sufficient.     

Student opinion of the value of student evaluations

This is an empirical investigation of student attitudes towards evaluation of instruction questionnaires. A 13-item attitude questionnaire was administered to 440 students in the Schools of Law and Engineering at Tel-Aviv University. Results indicated that students generally considered evaluation of instruction of value and were willing to participate in subsequent evaluations. They felt their evaluations were significant in influencing faculty promotion and selection; however, they were uncertain of their effect on improving the level of instruction. Criteria relevant for evaluation were the course work-load and instructor's grading policy; the instructor's academic status or talent for entertaining was of little importance. Students' academic expectations or self-definition made little significant difference either in attitude toward evaluations or in choice of rating criteria. However, students with high academic expectations were less ready to make evaluations, were unsure whether evaluation was part of the students' domain, and granted greater weight to the instructor's rank - all of which could indicate their identification with the norms of the academic world.     

Grades,learning, and student evaluation of instruction

In the first experiment, the author employed three different grading standards, identified as stringent, moderate, and lenient, in separate sections of the course Psychology of Learning. Other aspects of the course were the same for each section. The different grading standards resulted in substantially different grade distributions in the anticipated direction. Evaluations of both the instructor's performance and the couse decreased as the stringency of the grading criteria increased. Every item on the evaluation questionnaire was systematically influenced by the grading criteria. The amount learned, as measured by performance on the objective part of the final test, increased as the stringency of the grading criteria increased. In the second experiment, the subjects were the students in two sections of the course Introductory Psychology, taught by the author during the same term. The same textbook, course notes, and grading criteria were employed. The manipulated variable was test frequency, with the two sections receiving either weekly or biweekly tests. Students in the section receiving weekly tests scored 11.9% higher, on the average, over all tests. This resulted in a substantial difference in grade distributions. The ratings of the section receiving lower grades were substantially lower on every item of the rating form. Students appear to rate instructors on the basis of a global impression (liking) which they form, which is strongly influenced by the grade the student receives.     

Understanding student evaluations of teaching in online learning

In this paper, we have developed a classification model for online learning environments that relates the Instructors Overall Performance (IOP) rating (according to students’ perceptions) with the course characteristics, students’ demographics and the effectiveness of the instructor in his/her teaching roles. To that end, a comprehensive Student Evaluation of Teaching (SET) instrument is proposed, which includes not only conventional teaching elements, but also items that encourage twenty-first century skills. The goal of the study is twofold: (i) to quantify the extent to which the selected variables explain the IOP rating, and (ii) determine which teaching and non-teaching variables most affect the IOP rating. The best performing classifier achieved a competitive accuracy, highlighting that the selected variables mainly determine the IOP values. Other important findings include: (i) the IOP value is mainly influenced by the effectiveness of the instructor in his/her teaching roles; (ii) teaching strategies that involve the cooperation between the technical and pedagogical roles should be promoted; (iii) the pedagogical role has the highest impact on the final IOP value; and (iv) the most influential demographic variable is the student’s status (working commitments and family responsibilities).