Estimating the impact of relative expected grade on student evaluations of teachers

Grade inflation over the past few decades has been a concern for many universities. Course evaluation scores are known to be positively correlated with students’ expected grades, and this paper tests whether or not there is an incentive for the instructor to “buy” higher evaluation scores by inflating grades. To test this hypothesis, I use unique data from the University of Washington's Office of Educational Assessment that includes a measure of each student's relative expected grade in the course. I find that there is an incentive for instructors to grade leniently after accounting for the potential endogeneity of the relative expected grade variable due to unobserved teacher productivity and unobserved heterogeneity of instructors and departments. Instructor fixed effects account for a significant part of the measured effect of relative expected grade on evaluations, and by not including them, the estimated impact of relative expected grade on evaluations is biased upwards.     

The impact of Covid-19 on student achievement: Evidence from a recent meta-analysis

This work attempts to synthetize existing research about the impact of Covid-19 school closure on student achievement. It extends previous systematic reviews and meta-analyses by (a) using a more balanced sample in terms of country composition, (b) considering new moderators (type of data and research design), and (c) including studies on tertiary education students in addition to primary and secondary education students. Our meta-analysis findings show that the pandemic had, on average, a detrimental effect on learning. The magnitude of this learning deficit (about 0.19 standard deviations of student achievement) appears to be roughly comparable to that suffered by students who have experienced a significant disruption in their schooling due to a major natural disaster (e.g., Hurricane Katrina). Students are also found to have lost more ground in math/science than in other subjects. Additionally, one year or more after the first lockdown, students seem to have been unable to catch up on unfinished learning from the pandemic. This result suggests that more efforts should be made to ensure students recover their missed learning in order to avoid negative long-term consequences for them and society.     

Linking student evaluations to institutional goals: a change story

For the past 30?years, beginning with the seminal work of Herbert Marsh in Australia and New Zealand, institutions of higher education have developed internal practices and procedures to collect and analyse student evaluations of teaching and learning. However, the question remains: has this development resulted in the achievement of institutional goals that maximise learning across all teaching contexts? As is the case in many other countries, a recent review of Australian national student evaluation data failed to conclude that student evaluations have improved overall teaching. However, these data have made student perceptions of teaching and learning more salient in the minds of tertiary educators. Certainly, teaching staff are aware of the impacts of student evaluations on informed decision-making, such as the continuation or discontinuation of courses, and on their promotion processes. This paper will review student evaluation practices according to criteria used in change theories, such as the transtheoretical change model (TTM). TTM construes organisational change as a process involving progress through a series of five stages: precontemplation, contemplation, preparation, action and maintenance. This paper will focus on organisational and behavioural outcomes that can be linked to the use of student data. It will recommend strategies for better aligning evaluation results to the stages of change.     

The effect of kindergarten classroom size reduction on second grade student achievement: Evidence from California

Because classroom size reduction (CSR) and standards based testing were implemented at the same time in California during the mid-1990s, it is difficult to isolate the effects of classroom size on outcomes from the effects of curriculum changes. As a result, the main comparison in this paper is very specific – the effect of reduced sized kindergarten classrooms on test performance in second grade. The main result of the paper is that the effect of classroom size reduction is small, especially when compared to the other determinants of student achievement at the second grade level. There is a very small effect of classroom size reduction on student achievement in reading and math, and no effect in language and spelling. Of the two offsetting effects of CSR controlling for spending – the direct effect of smaller classes and the indirect effect on teacher quality, the class-size effect is larger. Over time, the negative effect on teacher quality is reduced as new teachers gain experience and credentials.     

The impact of school choice and public policy on segregation: Evidence from Chile

Advocates argue that vouchers can make improved educational opportunity available to disadvantaged students. Critics contend that vouchers increase the risk of stratification. Researchers have found that Chile's voucher program has lead to increased socioeconomic school segregation. What has been overlooked, however, is segregation between schools within a sector and variation within private for-profit and non-profit school sectors. I find that public schools are more likely to serve disadvantaged students than private voucher schools. I also find that disadvantaged students are more segregated among private voucher schools than among public schools. While between and within sector segregation levels vary across private voucher school types, the differences are not always consistent with theory. The data also suggest that policies can either mitigate or exacerbate the stratifying effects of educational vouchers.     

The impact of a universal class-size reduction policy: Evidence from Florida's statewide mandate

Class-size reduction (CSR) mandates presuppose that resources provided to reduce class size will have a larger impact on student outcomes than resources that districts can spend as they see fit. I estimate the impact of Florida's statewide CSR policy by comparing the deviations from prior achievement trends in districts that were required to reduce class size to deviations from prior trends in districts that received equivalent resources but were not required to reduce class size. I use the same comparative interrupted time series design to compare schools that were differentially affected by the policy (in terms of whether they had to reduce class size) but that did not receive equal additional resources. The results from both the district- and school-level analyses indicate that mandated CSR in Florida had little, if any, effect on student achievement.     

Subjective evaluations of intelligence and academic self-concept predict academic achievement: Evidence from a selective student population

The study examined the relationship between implicit theories, goal orientations, subjective and test estimates of intelligence, academic self-concept, and achievement in a selective student population (N = 300). There was no direct impact of implicit theories of intelligence and goal orientations on achievement. However, subjective evaluations of intelligence and academic self-concept had incremental predictive value over conventional intelligence when predicting achievement accounting for more than 50% of its variance. The obtained pattern of results is presented via structural equation models and interpreted within a dynamic regulative systems framework suggesting the importance of further studying complex sets of achievement predictors that include ability, personality and mediating constructs.     

The effect of same-gender teacher assignment on student achievement in the elementary and secondary grades: Evidence from panel data

We utilize information from a rich administrative panel dataset following the universe of test-taking public school students in Florida over a period of five years to estimate the relationship between same-gender teacher assignment and student achievement. We estimate how a student's achievement changes as he/she is assigned to teachers of different genders throughout his/her academic career, holding constant both observed and unobserved factors related to academic outcomes. We also provide estimates from models that evaluate how the relative performance of male and female student assigned to the same teacher or in the same classroom relates to the gender of the teacher. We find no statistically distinguishable relationship between same-gender teacher assignments and student math or reading achievement in elementary school. We find a statistically significant relationship between being assigned to a female teacher and student achievement in middle and high school, however the magnitude of the effect is small.     

The impact of labor market work and educational tracking on student educational outcomes: Evidence from Taiwan

This study empirically investigates how working while enrolled in high school affects educational outcomes, while accounting for self-selected educational tracking. Using a longitudinal survey of Taiwanese youth and exploiting the inter-zip-code geographic variations in youth-preferred industries, we find a negative effect of school-year work on educational achievement, and the negative marginal impact is much stronger for academic-track than for vocational-track students. An exogenous increase in school-year hours worked of 10 hours per week lowers college entrance scores by a 0.117 (0.083) standard deviation for academic-track (vocational-track) students. The negative impact of school-year work tends to be overstated if the endogeneity arising from educational tracking is not acknowledged—with upward biases as large as 16% and 30% for academic- and vocational-track students, respectively. Among subjects, math scores suffer most from working during the school year.     

The impact of online tutorials on course completion rates and student achievement

Providing learning support for students has been the most challenging issue for Universitas Terbuka (UT) in Indonesia since most students live in rural areas within which qualified tutors are hard to find. Attending face‐to‐face meetings within regular scheduled times is not feasible for most working students. Similarly, disseminating academic administration and information on time has also been very difficult to do since students are scattered throughout the country. These unresolved problems are believed to have resulted in low course completion. The recent development of ICT as well as its infrastructure in Indonesia has opened up a possibility for UT to provide its students with a fast, reliable and affordable two‐way communication channel. In accordance with this, UT piloted the use of ICT (specifically the Internet) to provide online tutorials to its students. It was expected that students taking advantage of the online tutorials would contribute to higher completion rates and achievement than those who did not. The project was carried out with the aid of a grant from the International Development and Research Centre, Ottawa, Canada. This paper reports the findings of that pilot project, which show that student participation in online tutorials does indeed improve course completion rates and achievement.     

The impact of school policy and stakeholders' actions on student learning: A longitudinal study

This paper proposes a theoretical framework on how school policy can promote student learning. School policy is considered to have an indirect effect on student achievement by changing school stakeholders' actions toward improving the School Learning Environment (SLE) and teaching practice. A reciprocal relationship between school policy and stakeholders' actions is also considered. A longitudinal study was conducted to test the framework's main assumptions. A stratified sample of 64 primary schools was selected and students' achievement in Mathematics at the beginning of Grade 4 and at the end of the next three consecutive school years was measured, alongside the school policy and teachers' actions with regards to issues associated with teaching and the SLE. The results of multilevel structural equation modelling analyses supported the main assumptions of the framework. Implications for the development of school policy are drawn and suggestions for further research are provided.     

Changes in student effort and grade expectation in the course of a term

Students were surveyed and asked to self-evaluate their performance and time-on-task in six engineering courses (eight sections). Surveys were conducted four times over the course of a 10-week quarter. Students with the highest grade point averages (GPAs) (2.88 mean; 4.00 mode) indicated ‘no change’ while students with lower GPAs (2.67 mean; 1.83 mode) indicated a ‘negative change’ in grade expectations by the end of the quarter, indicating that better performing students are better self-evaluators. Students reduced time-on-task 2–4 hours/week, especially early in the quarter. Students who changed their time-on-task during each survey had a GPA mode of 4.00, indicating that ‘master’ students readily adapt to course demands. This study shows that most students over-predict their grades and their level of commitment to a course and lose confidence in their abilities as the term progresses. It is suggested that instructors obtain student grade predictions and use them to provide timely and appropriate feedback.     

The effects of learning partners and retests on pretest-posttest scores,final course grades,and student attitudes

The present study investigated the effect of three peer monitoring and retest conditions on student quiz scores, attitudes, study time, final course grades, and independent pretest-posttest scores. Three introductory special education classes were randomly assigned to treatments. Students in Group One used a learning partner system similar to one used in a study reported earlier by Fraser, Beaman, and Kelem (1972). Group Two used a system which permitted an unlimited number of retests. The students in Group Three were assigned a combined learning-partner-retest system. All three treatments resulted in significant posttest gains. However, there were highly significant differences in the distribution of course grades. Based on this data it is possible that some behavioral techniques serve more to inflate grades than actually increase overall mastery. Results are discussed in terms of the effectiveness of the learning partners procedure and the implications of this study on the choice of dependent variables for future studies comparing innovative instructional procedures. Several aspects of the data such as study time and student attitudes which tend to contradict results reported earlier by Fraser, Beaman, and Kelem (1972) are documented and discussed.     

Teacher support,student motivation,student need satisfaction,and college teacher course evaluations: testing a sequential path model

Self‐determination theory posits that individuals who have basic psychological needs satisfied while engaging in an activity will be more likely to value and persist in that activity. Scholars in this area have also posited that autonomy‐supportive social contexts are top‐down determinants of individual need satisfaction. To understand better the progression from social to motivational to outcome variables, we tested a four‐step path model within a classroom setting. Data were collected from 220 students in an introductory journalism course, which was subdivided into 14 lab sections. Students were surveyed regarding their feelings about the course, the autonomy support of the instructor, the motivation they felt to engage in the course, and the levels of need satisfaction they experienced. Theoretically‐guided structural equation modeling produced a best‐fitting model in which teacher autonomy support predicted more self‐determined student motivation, which along with teacher autonomy support predicted greater student psychological need satisfaction, which led finally to higher predicted course grades and higher teacher‐course evaluations. Overall, the results replicate and significantly extend previous findings regarding need satisfaction and teacher‐course evaluations. Implications for pedagogy and educational interventions are discussed.     

Evaluating the impact of first-year writing course grades on college-student persistence

Informed by Tinto’s model of persistence, this large-scale evaluation study (N = 8916) centers on intersections of first-year composition (FYC), grades, and persistence at a career-focused, polytechnic university in a midwestern U.S. state. It presents a replicable design exploring a pathway through Composition 1 and Composition 2 to graduation and suggests theoretical and practical implications for early-alert, student-support intervention. Results of binary logistic regression indicate that, accounting for demographic and enrollment factors, grades significantly predict whether students take Composition 2 immediately after Composition 1, whether they persist to Composition 2 at all, and whether they graduate. We propose that the linked sequence of writing courses may allow students to build skills over a year-long sequence, enhancing self-efficacy and grade-reinforced senses of belonging, while factors related to ethnic identity and first-generation status pose barriers at critical points along the first-year sequence. We conclude with suggested points of student-support intervention and future research on grades, first-year writing coursework, and college-student persistence related to writing self-efficacy and senses of belonging.     

Effects of differentiated school vouchers: Evidence from a policy change and date of birth cutoffs

This paper studies the effects of an increase in school choice by examining a 2008 reform that made the value of Chile’s (previously flat, universal) school voucher a step function of student income. This policy increased the number of private schools that low income children could access free of charge. I identify the impact of the policy by combining its introduction with variation from a date of birth enrollment cutoff. I show that the differentiated voucher lowered, but only slightly, the probability that students used public schools. Students more likely to move to private schools experienced better school characteristics but no increase in test scores. Further analysis suggests a rise in test scores for students most likely to stay in public schools. These results suggest that the effects of the policy on test scores were caused by responses from public schools, instead of by the re-sorting of students into private schools.     

Exploring the impact of faculty reflection on weekly student evaluations of teaching

This exploratory study considered Larrivee’s assessment of teachers’ reflective practice levels by using a formative, weekly, online student evaluation of teaching (SET) tool through a virtual learning environment (VLE) as a means to encourage reflective practice. In‐depth interviews were conducted with six faculty members in three departments at a university college in the UK. The study found that: (a) faculty who experienced surface‐level reflection were more likely to have a reactive reflection style; and (b) faculty who experienced higher levels of reflection were more likely to have a proactive reflection style. Overall, the tool was found to be an efficient means of encouraging reflection by all participants and demonstrated that reflective practice could come about as a result of these weekly formative SETs. The study concludes with suggestions for academic development and future research on reflection that could be conducted using SETs via a VLE.     

The impact of high-stakes testing on student proficiency in low-stakes subjects: Evidence from Florida's elementary science exam

An important criticism of high-stakes testing policies – policies that reward or sanction schools based on their students’ performance on standardized tests – is that they provide schools with an incentive to focus on those subjects that play a role in the accountability system while decreasing attention to those subjects that are not part of the program. This paper utilizes a regression discontinuity design to evaluate the impact of Florida's high-stakes testing policy on student proficiency in the low-stakes subject of science. We confirm prior results that students in schools facing more immediate sanctions under the policy made substantial gains in the high-stakes subjects of math and reading. Contrary to the crowding-out hypothesis, we find that students in these schools made substantial achievement gains in the low-stakes subject of science as well.     

The impact of student misconceptions on student persistence in a MOOC

Massive Online Open Courses (MOOCs) provide opportunities to learn a vast range of subjects. Because MOOCs are open to anyone with computer access and rarely have prerequisite requirements, the range of student backgrounds can be far more varied than in conventional classroom-based courses. Prior studies have shown that misconceptions have a huge impact on students' learning performance; however, no study has empirically examined the relationship between misconceptions and learning persistence. This study of 12,913 MOOC-takers examines how students' misconceptions about the upcoming course material affect course completion. Using a survival analysis approach, we found that, controlling for the score in a pre-course test, students holding more misconceptions had a higher dropout rate at the start of the course, an effect that diminished over time. Other student variables were found to have a positive impact on survival that persisted throughout the entire course: U.S. location, higher age, an intention to complete, better English skills, prior familiarity with the subject, motivation to earn a certificate, and score and time spent on the previous problem set (homework). By contrast, student gender, education level, number of previous MOOCs completed, and motivation to participate in online discussion forums did not affect survival.