Effects of class size on alternative educational outcomes across disciplines

This is the first study to use self-reported ratings of student learning, instructor recommendations, and course recommendations as the outcome measure to estimate class size effects, doing so across 24 disciplines. Fixed-effects models controlling for heterogeneous courses and instructors reveal that increasing enrollment has negative and significant effects on student satisfaction in Sociology, Political Science, Computer Science and Engineering, and Mechanical and Aerospace Engineering. Educational outcomes in Linguistics, Psychology, Biological Sciences, Chemistry, Math, Physics, Cognitive Sciences, Visual Arts, History, and Philosophy are unaffected by class size. Other disciplines, including Economics, have inconclusive findings. No discipline benefits from increasing enrollment.     

Class size and teacher effects in higher education

Using student evaluations of their instructor as an outcome measure, we estimate and compare class size and teacher effects for higher education, with an emphasis on determining whether a comprehensive class size reduction policy that draws on the hiring of new teachers is likely to improve educational outcomes. We find that first time teachers perform significantly worse than their peers, and we find substantial class size effects. Hence higher education institutions face a tradeoff if they wish to increase admission. This tradeoff implies that as class size increases, at first the negative class size effect is smaller than that of introducing a first time teacher. However, beyond a certain level, the class size effect dominates and it is better to create a new class with a first time teacher.¹     

Estimating the impact of relative expected grade on student evaluations of teachers

Grade inflation over the past few decades has been a concern for many universities. Course evaluation scores are known to be positively correlated with students’ expected grades, and this paper tests whether or not there is an incentive for the instructor to “buy” higher evaluation scores by inflating grades. To test this hypothesis, I use unique data from the University of Washington's Office of Educational Assessment that includes a measure of each student's relative expected grade in the course. I find that there is an incentive for instructors to grade leniently after accounting for the potential endogeneity of the relative expected grade variable due to unobserved teacher productivity and unobserved heterogeneity of instructors and departments. Instructor fixed effects account for a significant part of the measured effect of relative expected grade on evaluations, and by not including them, the estimated impact of relative expected grade on evaluations is biased upwards.     

Measuring the impacts of administrative policies on student performance in higher education

In this study, we investigate the joint impacts of class size, modality of content delivery and instructor workloads on 1078 student performances from 2010 through 2014 in a required College of Business Administration Information Systems (IS) course at a public Midwestern university, controlling for 11 demographic and other factors that might impact student performance. Class size, modality of delivery and instructor assignments are strategic college decisions affected by and subject to administrative policies that can impact the probabilities of student performance. Using the Akaike Information Criteria (AIC) and Chi-square significance tests, we construct a parsimonious binary logit model of individual student performance in the IS course and examine the effects of the administratively controllable factors on student performance. We find that the class size and the assignment of instructors to teach the course significantly impact student performance while modality of delivery does not. We offer and evaluate administrative policy recommendations likely to improve student performances.     

Exploring the beliefs surrounding student evaluations of instruction in criminology and criminal justice undergraduate courses

Student evaluations of instructors are given consideration in a number of important decisions concerning instructors such as hiring, merit pay, and in the tenure and promotion process. However, a number of popular beliefs surround the interpretation of these evaluations. Using student evaluations for the three research-oriented universities in the state of Florida as a database, a number of these beliefs are explored. Specifically, the effects of instructor attributes (such as gender, rank and grades) and course characteristics (such as class size, type, number of course meetings, and time of day offered) on the end of the semester student evaluation rankings are discussed.     

Exploring graduate and undergraduate course evaluations administered on paper and online: a case study

The purpose of this study was to analyse the students’ evaluations of the course and instructor for all statistics courses offered during fall semester 2009 at a large university in the southern United States. Data were collected and analysed for course evaluations administered both online and on paper to students in both undergraduate and graduate courses. Unlike most previous studies on this subject, class section rather than student was treated as the unit of analysis. It was of specific interest to verify prior research findings that evaluation surveys administered online would not result in lower course and instructor ratings and lower response rates. The results showed that there is not sufficient evidence within the collected data to conclude that either course and instructor ratings or response rates are lower for evaluations administered online (online evaluations) than they are for evaluations administered on paper (paper evaluations). Of secondary interest was whether class ratings would be associated with students’ attendance and a comparison of variability among answers for undergraduate vs. graduate students. It was observed that class and teacher ratings were not related to students’ attendance and individual students did not tend to give the same answer for every question on their survey.     

A review of the research on student evaluation and a report on the effect of different sets of instructions on student course and instructor evaluation

An extensive review of the research concerning the effect of different variables on student ratings is presented. A study is then reported comparing the effects of different sets of instructions on student evaluations of the course and instructor. The results indicated that the students who were informed that the results of their ratings would be used for administrative decisions rated the course and instructor more favorably on all aspects than students who were informed that the results of their ratings would only be used by the instructor.The authors are indebted to Professor Robert A. Waller for cooperating in obtaining the data on his two history courses.     

The effects of class size in online college courses: Experimental evidence

Class size is a first-order consideration in the study of education cost and effectiveness. Yet little is known about the effects of class size on student outcomes in online college classes, even though online courses have become commonplace in many institutions of higher education. We study a field experiment in which college students were quasi-randomly assigned to either regular sized classes or slightly larger classes. Regular classes had, on average, 31 students and treatment classes were, on average, ten percent larger. The experiment was conducted at DeVry University, one of the nation's largest for-profit postsecondary institutions, and included over 100,000 student course enrollments in nearly 4,000 classes across 111 different undergraduate and graduate courses. We examine class size effects on student success in the course and subsequent persistence in college. We find little evidence of effects on average or for a range of course types. Given the large sample, our estimates are precise, suggesting that small class size changes have little impact in online settings.     

Student Psychological Need Satisfaction and College Teacher-Course Evaluations

Two studies examined student psychological need satisfaction as a predictor of positive teacher-course evaluations. In Study 1, 268 undergraduates recalled and rated the quality of a recent important college course, then rated their feelings of autonomy, competence, and relatedness within that course. Consistent with self-determination theory, all three ratings predicted instructor and/or course ratings. Study 2 found the same pattern in a sample of 179 introductory journalism students nested within 12 sections of a single course. Study 2 also evaluated instructor characteristics as predictors of mean levels of student need satisfaction across the 12 classes. Although instructor age and overall teaching experience were unrelated to students' need satisfaction, greater experience teaching their particular class negatively predicted student autonomy and relatedness need satisfaction. Implications for pedagogical practice are discussed.     

Understanding Student Evaluations of Teaching Quality: The Contributions of Class Attendance

Institutions of higher learning employ compulsory attendance policies with the expectation that these mandates enhance students' academic performance and perceptions of course quality. However, numerous empirical investigations demonstrate equivocal and often contradictory findings regarding the relationship between attendance and various markers of student achievement. The present investigation extends this research by exploring the utility of student ratings of the need to attend class in predicting their perceptions of teaching excellence after controlling for class size, instructor availability, and small-group interactions. As hypothesized, ratings of the need to attend predicted excellence while accounting for a significant 5.3% of the variance. Discussion and conclusions highlight the utility of class attendance in understanding students' evaluations of teaching and course quality.     

Administrative and structural changes in student evaluations of teaching and their effects on overall instructor scores

Using nine years of student evaluation of teaching (SET) data from a large US research university, we examine whether changes to the SET instrument have a substantial impact on overall instructor scores. Our study exploits four distinct natural experiments that arose when the SET instrument was changed. To maximise power, we compare the same course/instructor before and after each of the four changes occurred. We find that switching from in-class, paper course evaluations to online evaluations generates an average change of ?0.14 points on a five-point scale, or 0.25 standard deviations (SDs) in the overall instructor ratings. Changing labelling of the scale and the wording of the overall instructor question generates another decrease in the average rating: ?0.15 of a point (0.27 SDs). In contrast, extending the evaluation period to include the final examination and offering an incentive (early grade release) for completing the evaluations do not have a statistically significant effect on the overall instructor rating. The cumulative impact of these individual changes is ?0.29 points (0.52 SDs). This large decrease shows that SET scores are not comparable over time when instruments change. Therefore, administrators should measure and account for such changes when using historical benchmarks for evaluative purposes (e.g. appointments and compensation).     

Limited contracts,limited quality? effects of adjunct instructors on student outcomes☆

A growing literature documents significant differences in college student outcomes between adjunct and full-time instructors, although less is known regarding the causes of these differences. Many existing studies implicitly treat instructor quality—as measured by an instructor’s effect on student outcomes—as a fixed trait. However, instructor quality itself may be sensitive to employment rank, given that adjunct and full-time instructors work under very different conditions. Understanding mechanisms driving these differences is important for informing policy decisions aimed at improving student outcomes. Using panel data on first-semester students at public colleges in Arkansas, this paper first establishes that adjuncts have significantly worse student outcomes than full-time counterparts on a number of metrics. Next, I investigate mechanisms underlying differences in outcomes across instructor rank. I take advantage of the panel structure of the data and use an instructor fixed effects approach to provide evidence that within instructors, student outcomes improve when the instructor is full-time, compared to when the instructor is an adjunct. Results indicates improving student outcomes is about more than just getting better instructors, since instructor quality is not a fixed trait. Rather, factors like an instructor’s conditions of employment and teaching and working environment may affect the outcomes of their students.     

Online student evaluations of teaching: what are we sacrificing for the affordances of technology?

In the context of increased emphasis on quality assurance of teaching, it is crucial that student evaluations of teaching (SET) methods be both reliable and workable in practice. Online SETs particularly tend to raise criticisms with those most reactive to mechanisms of teaching accountability. However, most studies of SET processes have been conducted with convenience, small and cross-sectional samples. Longitudinal studies are rare, as comparison studies on SET methodological approaches are generally pilot studies followed shortly after by implementation. The investigation presented here significantly contributes to the debate by examining the impact of the online administration method of SET on a very large longitudinal sample at the course level rather than attending to the student unit, thus compensating for the inter-dependency of students’ responses according to the instructor variable. It explores the impact of the administration method of SET (paper based in-class vs. out-of-class online collection) on scores, with a longitudinal sample of over 63,000 student responses collected over a total period of 10 years. Having adjusted for the confounding effect of class size, faculty, year of evaluation, years of teaching experience and student performance, it is observed that the actual effect of the administration method exists, but is insignificant.     

Within-Class Variability in Student–Teacher Evaluations: Examples and Problems

Although student evaluation of instruction has been shown to produce reliable results over class averages, considerable within‐class variability exists that has not been investigated. This study looked at examples of student evaluations in which students diametrically differed in their evaluation of the same instructor. Patterns were noted. A survey was conducted to determine the self‐reported incident of students changing evaluations based on noninstructional factors. About 60% of students admitted that they had changed evaluations for reasons related to the instructor's personality, while 30% admitted to changing evaluations based on grades and rigor. Possible reasons and effects of such changes are discussed.     

Instructor Reputation and Student Ratings of Instruction

The purpose of this study was to examine the association between instructor reputation, as perceived by students, and student evaluations of the instructor and course. A total of 754 students from 39 classes participated in the study. Based on what students claimed to have heard about the instructor prior to enrolling in the course, they were classified into one of three groups: positive reputation, no information, and negative reputation. Using these groupings, two analyses were performed. In the first, mean overall ratings for the instructor and course were calculated and presented by class. In the second, both instructor and course ratings were modeled using multilevel regression. Results show large mean differences in both instructor and course ratings between the positive and negative reputation groups. More specifically, students who heard positive information regarding the instructor's reputation rated both the instructor and course higher than students who heard negative information about the instructor.     

The stability of teacher performance in the same course over time

Traditional reliability studies report the stability of teaching performance, as measured by student course/instructor evaluations, as ranging from .40 to .90, depending upon the study. Recently, generalizability theory has been applied to these rating data to more specifically pinpoint sources of error in student course evaluations. The purpose of the present study is to (1) familiarize the reader with some of the work that has been done with student rating data using generalizability theory and (2) to illustrate the use of the theory on a problem that in the past has been examined using traditional reliability estimates. Specifically, the paper addresses the stability of teacher performance across time in the same course.     

Students choosing colleges: Understanding the matriculation decision at a highly selective private institution

This paper provides an econometric analysis of the matriculation decisions made by students accepted to Williams College, one of the nation's most highly selective colleges and universities. Using data for the Williams classes of 2008 through 2012 to estimate a yield model, we find that—conditional on the student applying to and being accepted by Williams—applicant quality as measured by standardized tests, high school GPA and the like, the net price a particular student faces (the sticker price minus institutional financial aid), the applicant's race and geographic origin, plus the student's artistic, athletic and academic interests, are strong predictors of whether or not the student will matriculate.     

The effect of perceived class mean on the evaluation of instruction

A total of 232 college students in six different courses in three departments participated in a study to examine the effect of perceived course mean on course and instructor evaluations. Following a midsemester exam, students were given their actual earned exam scores and a manipulated class mean that was either ten percentage points higher or lower than the actual class average on the exam. Participants then completed an evaluation of the course and instructor. It was hypothesized that students scoring above the manipulated mean would rate the course and instructor more highly than students scoring below the manipulated mean. It was further hypothesized that students who were told that the class mean was higher would rate the course and instructor more highly than students who were told that the mean was lower. Results supported the first hypothesis. However, hypothesis two was not supported. Students receiving the lower manipulated class mean rated instructors more favorably. Results suggest the need to consider both individual exam scores and class averages in understanding the grade-teaching evaluation relationship.     

Positive Effects of Converting a Food and Bioprocessing Analysis Course to an Inquiry‐Guided Approach

Abstract: Food science laboratory courses are traditionally taught as a series of preplanned laboratories with known endpoints. In contrast, inquiry‐guided (IG) laboratories allow students to ask questions, think through problems, design experiments, then adapt and learn in response to unexpected results. This study examined the effects of converting the course, “Analytical Techniques in Food and Bioprocessing Sciences” from a traditional approach (2008 to 2010 data) to an IG approach (2011 data) by assigning teams of 2–3 students a food and a set of 5 analyses to conduct over the course of the semester. Students were required to choose and justify the use of specific methods for each analysis, as well as to develop a supply list and a budget for the semester‐long project. During the semester, students were required to post and discuss their weekly progress with the instructor, teaching assistants, and the rest of the class using an online discussion forum. At the end of the semester, students were required to present the results of their analysis in both oral and written formats. Overall course grades were significantly higher (P≤ 0.05) using IG in 2011 compared to 2010 and 2009, but not to 2008 grades. Numerical course evaluations for the instructor, overall course, and lab, as well as written course evaluations all significantly (P≤ 0.05) improved. This suggests that an IG approach may measurably improve student performance in terms of course grades and the ability to complete semester long projects. It may also increase student satisfaction with the course, as measured by numerical and written end of semester surveys.     

Assessment & Evaluation in Higher Education Using the margin of error statistic to examine the effects of aggregating student evaluations of teaching

We proposed an extended form of the Govindarajulu and Barnett margin of error (MOE) equation and used it with an analysis of variance experimental design to examine the effects of aggregating student evaluations of teaching (SET) ratings on the MOE statistic. The interpretative validity of SET ratings can be questioned when the number of students enrolled in a course is low or when the response rate is low. A possible method of improving interpretative validity is to aggregate SET ratings data from two or more courses taught by the same instructor. Based on non-parametric comparisons of the generated MOE, we found that aggregating course evaluation data from two courses reduced the MOE in most cases. However, significant improvement was only achieved when combining course evaluation data for the same instructor for the same course. Significance did not hold when combining data from different courses. We discuss the implications of our findings and provide recommendations for practice.