Investigating substantive and consequential validity of student ratings of instruction

This study aimed at exploring faculty and student perspectives on student evaluations, as well as identifying their perceptions of the usefulness and appropriateness of the ratings for evaluating teaching effectiveness. More specifically, the study aimed at identifying the consequences, both intended and unintended, of using the evaluations, in addition to better understanding the process students used in responding to evaluations and what use faculty members made of them. Two surveys were developed and placed on the website of the Office of Institutional Research and Assessment. Emails were sent to all students and faculty participating in evaluations soliciting their cooperation and requesting their input. Faculty and student perceptions were compared qualitatively and statistically. Results revealed that students and faculty believe in the effectiveness and usefulness of the system with the need to overcome some negative consequences and biases inherent in its application at the University. Recommendations for system improvement are provided.     

A path analysis of factors often found to be related to student ratings of teaching effectiveness

The relationships among several variables outside of the instructor's classroom control and student ratings of teaching effectiveness are investigated in a causal network. Student ratings are relatively independent of external variables. Students may be able to take into account more factors than generally assumed when they rate their instructors.     

The factor invariance of student ratings of instruction under three sets of directions

This research addressed itself to the issue of the validity of student ratings of instruction, describing and comparing the factor patterns obtained under three different sets of directions. Graduate education students (n=414) were randomly assigned to one of three conditions: administrator use, instructor use, and student use. Subjects in each condition received a different cover letter which explained the purported use to which their ratings were to be put and which asked them to rate their course and instructor on a 33-item rating scale. Data were factor analyzed using principal-axes factor method followed by an oblique transformation. The factor patterns obtained were then compared using the coefficient of congruence. While two clusters (organization/structure and rapport/interaction) emerged across all three conditions, a third cluster appeared which was unique to each condition. The coefficients of congruence obtained generally indicated that the factors could not be considered invariant across the three conditions.     

Student ratings of college teaching: a comparison of faculty and their students

Based on the review of student ratings myths by Aleamoni (1987, 1999), a survey research design was used to analyse differences between college students' (n = 968) and faculty's (n = 34) perceptions. Generally, students held stronger beliefs in these myths, in that they believed faculty with excellent publication records were better qualified to evaluate teaching and that student ratings on single general items are accurate measures of teaching effectiveness. On the other hand, faculty believed that student ratings were invalid and unreliable. Further examination of student characteristics revealed that male students held stronger beliefs in these myths. Finally, students' beliefs in these myths were correlated with their actual ratings of nine dimensions of the Student Evaluation of Educational Quality. A discussion as well as suggestions for using student ratings is provided.     

Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes

The contribution examines theoretical foundations, factorial structure, and predictive power of student ratings of teaching quality. Three basic dimensions of teaching quality have previously been described: classroom management, cognitive activation, and supportive climate. However, student ratings, especially those provided by primary school students, have been criticised for being biased by factors such as teacher popularity. The present study examines ratings of teaching quality and science learning among third graders. Results of multilevel confirmatory factor analyses (N = 1556 students, 89 classes) indicate that the three-dimensional model of teaching quality can be replicated in ratings of third graders. In a longitudinal study (N = 1070 students, 54 classes), we found ratings of classroom management to predict student achievement, and ratings of cognitive activation and supportive climate to predict students' development of subject-related interest after teacher popularity is controlled for. The analyses show that student ratings can be useful measures of teaching quality in primary school.     

An empirical test of the validity of student evaluations of teaching made on RateMyProfessors.com

The present article examined the validity of public web‐based teaching evaluations by comparing the ratings on RateMyProfessors.com for 126 professors at Lander University to the institutionally administered student evaluations of teaching and actual average assigned GPAs for these same professors. Easiness website ratings were significantly positively correlated with actual assigned grades. Further, clarity and helpfulness website ratings were significantly positively correlated with student ratings of overall instructor excellence and overall course excellence on the institutionally administered IDEA forms. The results of this study offer preliminary support for the validity of the evaluations on RateMyProfessors.com.     

The significance of circumstances for college students' ratings of their teachers and courses

Although firm generalizations and conclusions cannot yet be drawn from the extant research on the effects on teacher and course ratings of the circumstances surrounding these evaluations, at least some studies have shown that college students' ratings of their teachers and courses are somewhat higher when students remain anonymous rather than identifying themselves, when the purported use of the ratings is an official or administrative one for use in salary, promotion, or tenure considerations rather than otherwise, and when the instructor is present rather than absent during the rating session. (The differences between each of these contrasted circumstances are usually rather small and do not inevitably appear across studies.) Certain variations in rating format have been found to make a difference in the ratings obtained, whereas other have not. From limited evidence, the exact timing (or occasion) of evaluation appears not to be important to ratings. Variability in sampling procedures, as it affects the composition of students available to complete rating forms, may or may not turn out to be a generally important element in ratings (as directly relevant data are collected). The analysis concludes with a discussion of (1) the presumed bias in ratings produced in certain of the rating conditions and (2) the more general issue of the comparability of ratings made in different circumstances of evaluation.     

Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, Cohen, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.     

Course characteristics and college students' ratings of their teachers: What we know and what we don't

From showing in a general way that there is room for course context to influence class (average) ratings of instruction, this review proceeds to a search for specific course characteristics that are associated with these ratings. Extant research has centered around five such characteristics: class size, course level, the electivity of the course, the particular subject matter of the course, and the time of day that the course is held. Although statistically significant zero-order relationships do not appear in every piece of research located for review, such relationships are more likely to be found than not for the first four of these characteristics. The associations may not be particularly strong, but rather clear-cut patterns do emerge. Of the studies reporting an association between size of class and class ratings, most find it to be inverse, although several studies show a curvilinear (U-shaped) relationship. Teacher (and course) ratings tend to be somewhat higher for upper division courses and elective courses. Compared to other instructors, those teaching humanities, fine arts, and languages tend to receive somewhat higher ratings. The possible reasons for these relationships are many and complex. A precise understanding of the contribution of course characteristics to the ratings of teachers (and the courses themselves) is hampered by two circumstances. Studies in which relevant variables are controlled are far fewer in number than are the studies in which only the zero-order relationships between course characteristics and ratings are considered. More importantly, existing multivariate studies tend to underplay or ignore the exact place of course characteristics in a causal network of variables.     

The validity of student ratings of teaching quality: Factorial structure,comparability, and the relation to achievement

Student ratings are widely used to assess teaching quality in primary and secondary education. This necessitates a thorough examination their validity and reliability in such contexts. To this end, we present validity evidence for using student ratings of teaching quality based on their factorial structure, comparability across grades and subjects, and relation to achievement. Based on data from the 2019 Trends in Mathematics and Science Study (TIMSS) and its national extensions in Norway (N = 3951 5th graders and N = 4575 9th graders), we could confirm a theoretically assumed three-dimensional factor structure (i.e., classroom management, cognitive activation, and supportive climate). Moreover, classroom management and supportive climate were more comparable across grades and subjects then cognitive activation. Finally, classroom management was associated with achievement across all groups, whereas other dimensions showed mixed results, especially across grades. Overall, the findings indicate that students, especially in Grade 9, can provide a valid assessment of teaching quality.     

Bias hypotheses under scrutiny: investigating the validity of student assessment of university teaching by means of external observer ratings

To advance the discussion on the validity of student evaluations of university teaching, student ratings of two teaching dimensions – student involvement and rapport – were compared with corresponding observer ratings. Seven potential bias variables were tested with regard to their impact on the students’ teaching assessment: three teacher characteristics (first impression, enthusiasm, humour) and four student characteristics (prior interest, expected grades, study experience, class attendance). Bias was defined as an impediment of the students’ assessment of teaching on course level. By means of bivariate correlations with course averages and two-level latent moderated structural equations, data of 1,716 students in 80 courses were analysed. Results showed that all three teacher characteristics were genuinely connected to rapport, and even explained variance of the student-rated variable when controlling for observer-rated rapport. The assessment of student involvement was not modified by the teacher characteristics except for teacher enthusiasm, which affected the student evaluation when controlling for observed involvement and, moreover, moderated the relation between the observed and the student-rated variable. For the examined student characteristics, no biasing effects were found – neither on rapport nor on student involvement.     

高职院校学生管理工作创新研究

近年来,我国的高职教育迅速发展,高职院校的学生管理工作日益成为人们关注的焦点。在实践中创新高职学生管理,探索适应高职院校发展规律的学生管理工作之路,对高职院校的发展、稳定及人才的培养起着非常重要的作用。本文试从创新高职学生管理工作的必要性、高职学生管理工作创新之构想、创新高职学生管理工作的途径等三个方面对高职学生管理工作的创新进行探讨。     

Teacher professionalism and student evaluation of teaching: will better teachers receive higher ratings and will better students give higher ratings?

The use of student evaluations of teaching performance has been an important but controversial tool in the improvement of teaching quality during the past few decades. Although student evaluations of teaching are implemented in many faculties, not everyone is convinced of the desirability and utility of these ratings. In this paper, we present the results of a study with regard to the existence of a higher‐order factor that might influence students' perceptions of teaching and, thus, explain the variance in teacher rating scales. A second question concerns the effect of students' grades on teacher ratings and of other factors influencing this relationship (for instance, students' overall grades).     

A description of effective college teaching in five disciplines as measured by student ratings

Student rating of teachers in five disciplines (science and math, education, social sciences, humanities, and business) were analyzed to determine which teacher attributes were important in predicting ratings of teaching effectiveness. Rating results from 1,439 courses taught at Southern Illinois University, Carbondale from 1973 to 1974 were used as data for this study. The results indicated that the instructor attributes rated as characteristic of effective instruction were highly consistent across disciplines, and the effective instructor was described as (1) knowing when students understood him, (2) increasing students' appreciation of the subject matter, (3) answering impromptu questions satisfactorily, (4) achieving the objectives of the course, and (5) giving several examples to explain complex topics.     

Using the sampling margin of error to assess the interpretative validity of student evaluations of teaching

We present an equation, derived from standard statistical theory, that can be used to estimate sampling margin of error for student evaluations of teaching (SETs). We use the equation to examine the effect of sample size, response rates and sample variability on the estimated sampling margin of error, and present results in four tables that allow users to assess the interpretative validity (IV) of a SET score for a specific evaluation context. In this framework, a small margin of error, e.g. 3% of the range, suggests a greater precision, or IV in a score, whereas a large margin of error, e.g. 10% of the range, suggests a lower IV. We review the SET literature and summarise five ways that low response rates may be ameliorated. We also compare our results to previous published studies. Our findings matched, but greatly extended, prior results.     

How large is the course effect? A note on Romney's course effect vs. teacher effect on students' ratings of teacher competence

Romney (1977) presented data from which he concluded that within student ratings of college instruction, the course that an instructor teaches is as important a determiner of resulting ratings as the instructor himself. Reanalysis of his data indicates that the course effect is actually quite small, a result that is consistent with earlier studies.     

素质教育下对女大学生空竹教学的价值研究

在当前素质教育下,如何实现对当代女大学生体育教学与健康教育的有效结合成为一个不容回避的课题。本文通过采用文献法、数理统计法以及逻辑分析法等积极探索了空竹教学在大学生素质教育中的理论依据,结合当代大学的实际情况,探索了空竹运动教学对女大学生的体质健康水平的影响程度,为提高女大学生的素质教育的效果和改善女大学生的心理素质等提供了一些有用的教学建议,使针对女大学的体育教学同健康教育有效的结合在一起。     

中国大学生资助政策:对H大学的个案研究

The research findings have made it clear that Chinese college student aid policy has several characteristics. Generally speaking, student financial aid is increasing up to some extent. The government financial aids focus mostly on improving equality of educational opportunity in higher education. However, aids from the government are inadequate, especially national student loans. Besides, all kinds of aids were decreasing from 2004 to 2006 and the aid programs were short of stability between various years. In order to improve equal access to higher education through student aid policy, it is necessary to enlarge the government aids and reform the aid system. __________ Translated from Jiaoshi Jiaoyu Yanjiu 教师教育研究 (Teacher Education Research), 2008, (2), 59–63     

Influence of improved teaching practices on student satisfaction ratings for two undergraduate units at an Australian university

To track the quality of instruction delivered at the University of Western Australia, the university surveys all units using its Students’ Unit Reflective Feedback (SURF) metrics, and faculties use these metrics to benchmark student satisfaction. Consequently, teaching staff are actively encouraged to adopt teaching practices that will increase the average levels of these metrics. Using a ‘before-after’ approach, we compared these metrics before and after the implementation of improved teaching practices that addressed specific weaknesses identified through student responses for two undergraduate units. Despite the implementation of improved teaching practices, SURF scores did not increase significantly for the two units assessed, due possibly to the components of the academic programme covered by the SURF questions, the field-based nature of the units, and the difficulty in measuring an increase in the SURF scores of units which already score highly. This poor sensitivity of the SURF metrics to the implementation of improved teaching practices could have implications for their use for performance assessment by the university.