Student Ratings and Professor Self-Ratings of College Teaching: Effects of Gender and Divisional Affiliation

Twenty female and 23 male professors at a liberal arts college participated along with their 803 undergraduate students in a questionnaire study of the effects of professor gender, student gender, and divisional affiliation on student ratings of professors and professor self-ratings. Students rated their professors on 26 questions tapping five teaching factors as well as overall teaching effectiveness. Professors rated themselves on the same questions as well as on nine exploratory ones. On student ratings, there were main effects for both professor gender (female professors were rated higher than male professors on the two interpersonal factors) and division (natural science courses were rated lowest on most factors). These patterns were qualified by significant interactions between professor gender and division. Although professor self-ratings varied by division, there were few significant correlations between professor self-ratings and students' ratings. Implications for future research are discussed.Appreciation goes to Laura Capotosto and Julie Phelan for their research assistance.Suzanne Montgomery is now at Widener Law School, Philadelphia, PA, USA.     

An empirical test of the validity of student evaluations of teaching made on RateMyProfessors.com

The present article examined the validity of public web‐based teaching evaluations by comparing the ratings on RateMyProfessors.com for 126 professors at Lander University to the institutionally administered student evaluations of teaching and actual average assigned GPAs for these same professors. Easiness website ratings were significantly positively correlated with actual assigned grades. Further, clarity and helpfulness website ratings were significantly positively correlated with student ratings of overall instructor excellence and overall course excellence on the institutionally administered IDEA forms. The results of this study offer preliminary support for the validity of the evaluations on RateMyProfessors.com.     

Improving instruction of veteran physics professors

In spite of the abundance of publications describing university faculty development programs and activities, little is known about the effectiveness of such programs on specific disciplines or subject areas. The fact that differences have been identified in the dimensions on which students of different university departments rate their teachers suggests that instructors of different departments need different types of programs for teaching improvement. This article describes a study that has looked into methods for improving instruction of university physics full professors with many years of teaching experience. Two methods for this aim were examined for effectiveness: a workshop and individualized consultation, both augmenting students' midterm ratings of their instructors. Analysis of pre- and postworkshop questionnaires reveals impressive improvement on the majority of items, particularly those of specific teaching techniques discussed in the workshop, but not on the global ratings of the teacher. The special consultation procedure has been shown to bring about substantial increase on overall teaching performance. We conclude that veteran teachers are often unable to improve significantly their overall teaching performance when provided with midterm feedback from students' questionnaires or when participating in a workshop for teaching improvement. Improving their instruction requires substantial and continuous expert consultation as well as investing substantial time and efforts of their own.     

Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, Cohen, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.     

Students' Evaluation of Teachers and Instructional Quality--Analysis of Relevant Factors Based on Empirical Evaluation Research

This paper explores the factors that determine the student evaluation of teachers. Unlike many previous studies, the effect of potential biases on global ratings is examined in the context of teaching behaviour (that should in fact--in contrast to the potential biases--have an influence on global ratings). Our research takes into account that biases like the students' interest in the subject or their liking for the teacher may be a result of good teaching behaviour and may not be considered a mere bias of student ratings. Furthermore, it also concentrates on the students' attitudes towards evaluating their teachers. The empirical results are based on qualitative interviews with 40 students at Austrian commercial colleges and on a quantitative survey of 2,121 students who were asked to evaluate their Accounting teachers. Structural equation models show that the students' global ratings of teachers mainly depend on their teaching behaviour. However, global ratings are also affected by the students' attitudes towards evaluating their teachers, as well as by the students' liking for their teacher and their interest in the subject of Accounting.     

The challenge of Improving Teaching in a Globalizing World

The USA has served as a beacon of hope for thousands of foreign students and academics with its diverse and superior opportunities created by a system of meritocracy unparalleled in the world. In keeping with other industries, academia is increasingly becoming a global village and foreign‐born professors constitute a large proportion of university faculties. As higher education is increasingly accessible to students with varying levels of academic preparedness, faculties have become more aware of the importance of the opinions of students – the system’s consumers – on teaching. Bearing in mind the ‘similarity‐attraction paradigm’, this study sought to examine whether teaching evaluations are affected by cultural similarity or difference between students and instructors. Our data are based on teaching ratings from the largest Israeli public college. The analysis relates to 42,874 teaching ratings of 768 instructors, of whom 602 are Israeli‐born and 166 are immigrants from the Former Soviet Union (FSU). The findings indicate that FSU immigrant students awarded higher evaluations to FSU immigrant faculty members than their native‐born peers. Similarly, Israeli‐born students awarded higher evaluations to Israeli‐born faculty than the FSU immigrant students. We conclude by discussing the educational and managerial implications of these findings for higher education institutions with ethnically diverse faculty and students.     

THE RELATIONSHIP BETWEEN CEQ RATINGS AND INSTRUCTOR'S RANK, CLASS SIZE, AND COURSE LEVEL

A study was conducted to determine if the tendency for faculty members of higher rank to receive the highest ratings on the Illinois Course Evaluation Questionnaire (CEQ) remained when variables such as class size and course level were taken into account. The relationship between CEQ ratings and instructor's rank, class size, and level of course was examined by means of multivariate analysis of variance (MANOVA). Dependent variables were the six subscales of the CEQ. As hypothesized there were no significant differences in ratings assigned by students in small (1-20 students), medium (21-40 students), and large (over 40 students) classes, or received by teaching assistants, instructors, and assistant, associate, and full professors. Highly significant differences, however, were found in ratings assigned by students in freshman, sophomore, junior, senior, and graduate level courses. In addition, significant size by level and size by rank interaction effects were found. Discriminant functions computed for effects found to be significant yielded information concerning the extent and direction of these significant differences.     

College students' views of male and female college teachers: Part II—Evidence from students' evaluations of their classroom teachers

Although a majority of studies have found that male and female college teachers do not differ in the global ratings they receive from their students, when statistically significant differences are found, more of them favor women than men. Across studies, the average association between gender and overall evaluation, while favoring women (averager=+.02), is so small as to be insignificant in practical terms. Considering specific instructional dimensions of evaluations, female teachers receive very slightly higher ratings on their sensitivity to and concern with class level and progress than do men (averager=+.12). On other specific dimensions, men and women either do not differ or the differences are trivial in size (or, for two dimensions, while nontrivial, based on too few studies to be generalizable with any degree of certainty). Students tend to rate same-gendered teachers a little higher than opposite-gendered teachers. Although interaction effects on evaluations have also been found between gender of teacher and other factors (academic rank of the teacher, academic area, class level of the course, difficulty of the teacher or course, and the teacher's pedagogical orientation or personality characteristics), they are inconsistent across studies. Moreover, ratings of teachers are sometimes enhanced by gender-typical, and sometimes by gender-atypical, attributes, behaviors, and positions. The findings are discussed in terms of the expectations or demands of students and whether or not student ratings are biased by the gender of the teacher.     

Understanding the characteristics of effective professors: the student's perspective

Increasingly, higher education institutions are realising that higher education could be regarded as a business-like service industry and they are beginning to focus more on meeting or even exceeding the needs of their students. Recent research findings suggest that the factors that create student satisfaction with teaching (‘teaching satisfiers’) may be qualitatively differently from the factors that create dissatisfaction with teaching. Thus, this research uses the Kano methodology to reveal the characteristics of professors that students take for granted (‘Must-be factors’) and that have the potential to delight them (‘Excitement factors’). Kano questionnaires containing 19 attributes of effective professors taken from previous studies and focus group discussions were handed out in two marketing courses to 63 postgraduate students enrolled in a service marketing course. The Kano results corroborate previous US findings that revealed the importance of personality in general and support studies that stress the importance of professors creating rapport with their students in particular.     

British and Commonwealth Counselors: An Investigation of Counselor Characteristics and Theoretical Views

This study investigated personal characteristics, espoused theoretical orientation, counseling response style, and tendency toward variety as related to general guidance, and counseling competence, and to home or overseas origin of 32 (16 males and 16 females) students enrolled in the Guidance Unit at the University of Reading, England. Subjects were drawn from the United Kingdom (Home students) and from 11 overseas countries (Overseas students). Instruments administered were the Edwards Personal Preference Schedule (EPPS), the Porter Counseling Inventory (PCI), and the Similies Preference Inventory (SPI). The subjects' theoretical orientation was determined by the ratings of six theories of counseling. These ratings were given weighted scores for Insight and Action orientation, i.e., London's Insight-Action dichotomy. Guidance and counseling competency was based upon ratings by three professors using a 5-point scale. Results showed large differences between the groups on the EPPS and PCI, but EPPS and PCI scores were unrelated to the subjects' theoretical orientations. Competency ratings were related to theoretical preference, with Home students espousing Action approaches receiving lower ratings and Overseas students espousing Insight approaches receiving lower ratings. Results were discussed in terms of different cultural demands on counselors and with relevance to the use of the Insight-Action dichotomy in research with American counselor samples.     

The Educational Achievement of Pupils in Differentiated and Undifferentiated Groups

Responses on a Likert-type instrument, the Eight College Professors (Form B), from 664 education juniors and seniors in five small colleges were analyzed to examine the preferential rankings given four types of social science professors as an instructor. While some suggestive variations were observed, the overall order of preference was Teacher—Socialite—Researcher—Administrator (the least preferred) . This was in contrast to earlier findings on university students who placed Researcher ahead of Socialite. Female professors were consistently less preferred to male professors by college students of both sexes.     

大学的治理

美国现行大学治理体制中的四个主要参与方——校董、学术领袖、教授和学生各有其优势和弱点,"共享的治理"的成功取决于上述四方的互信与合作。美国现行大学治理体制运行正常,没有对其进行系统性变革的必要。但这种"共享的治理"也存在一些亟待改进与完善之处,如有必要改善校董的遴选方式,吸引最受尊敬和负责任的教授参与治理,在学术领袖和教授之间建立起一种更加相互信任与携手合作的和谐关系等等。     

The different impact of a structured peer-assessment task in relation to university undergraduates’ initial writing skills

This article presents an empirical evaluation of the validity and reliability of a peer-assessment activity to improve academic writing competences. Specifically, we explored a large group of psychology undergraduate students with different initial writing skills. Participants (n?=?365) produced two different essays, which were evaluated by their course peers and professors using a specific grading instrument (rubric). The validity of the task was demonstrated by a high inter-grader agreement and a strong degree of consistency between the ratings of the peers and professors. Although all students did not improve their writing skills between their first and second essays, the peer-assessment activity enhanced the writing abilities of low but not high-achieving participants. The pedagogical implications of these results are discussed.     

Student evaluations and the university professor: Caveat professor

Research evaluating student ratings of professors reveals continued controversy. Interpretations of student ratings of professors in terms of face validity are marred by halo affects, the apparent inability of even skilled raters to judge complex behaviors adequately, the salience of personality features in judging tasks, and a host of other variables. Research shows student ratings to be reliable, but design flaws for simple, first order, predictions usually omit the teacher as a cause. Interpretations of research are confusing because of justifications that indiscriminately involve nomological and applied models. Rating scale peculiarities, questionable validity, and scholastic homogeneity lead to diverse professional attitudes towards student opinions of professors, with a learner or consumer emphasis occupying the extremes. Several evaluation schemes are noted along with behaviors that tend to produce favorable student opinions.     

Teaching Spoken English to Refugees

Abstract

Two studies examined college students' perceptions of professors' ethical responsibilities. Students agreed that professors must demonstrate respect for students, teach objectively, and grade honestly, and they should not tolerate cheating or plagiarism. Results indicate that students expect professors to act with professionalism, to employ a vast base of content knowledge, and to show concern for student welfare. Many view professors as exemplars of scholarship and professional behavior. Professors must be cognizant of student expectations and should reflect on their behavior both in and out of the classroom.     

Understanding the perceived quality of professors’ teaching effectiveness in various disciplines: the moderating effects of teaching at top colleges

Measuring the perceived quality of professors’ teaching effectiveness is a critical issue in higher education. This study involves a large-scale data exploration with a sample of 16,802 professors, in which each professor had received at least 20 ratings from the RateMyProfessors website. We find that perceived difficulty (from the students’ perspective) has a significantly negative effect on perceived quality. However, when professors teach more difficult courses at top colleges, the decline in perceived quality is relatively small when compared to other colleges. In other words, whether professors come from top colleges has a moderating effect on the relationship between perceived quality and perceived difficulty. Furthermore, through a consideration of the characteristic differences among disciplines in terms of the relationship between perceived quality and perceived difficulty, we obtain three specific groups of disciplines. These findings facilitate a better understanding of quality for professors from different disciplines. We suggest that the measurement of teaching effectiveness should avoid the use of a single criterion because differences in courses, disciplines or schools can influence the measurement results, and these factors are beyond the control of professors.     

Correlations,trends and potential biases among publicly accessible web-based student evaluations of teaching: a large-scale study of RateMyProfessors.com data

Student evaluations of teaching are widely adopted across academic institutions, but there are many underlying trends and biases that can influence their interpretation. Publicly accessible web-based student evaluations of teaching are of particular relevance, due to their widespread use by students in the course selection process and the quantity of data available for analysis. In this study, data from the most popular of these websites, RateMyProfessors.com, is analysed for correlations between measures of instruction quality, easiness, physical attractiveness, discipline and gender. This study of 7,882,980 RateMyProfessors ratings (from 190,006 US professors with at least 20 student ratings) provides further insight into student perceptions of academic instruction and possible variables in student evaluations. Positive correlations were observed between ratings of instruction quality and easiness, as well as between instruction quality and attractiveness. On average, professors in science and engineering disciplines have lower ratings than in the humanities and arts. When looking at RateMyProfessors as a whole, the effect of a professor’s gender on rating criteria is small but statistically significant. When analysing the data as a function of discipline, however, the effects of gender are significantly more pronounced, albeit more complex. The potential implications are discussed.     

Access and perceived ICT usability among students with disabilities attending higher education institutions

An increasing number of students with disabilities are attending higher education. These students might face various difficulties coping with academic skills and with learning methods compared to students without disabilities. Integrating information and communication technologies (ICTs) in academic studies may be effective and constructive for students with and without various disabilities, as ICTs can provide students with adaptive ways to compensate for disabilities and enable them to improve learning. The present study examined students’ knowledge of and accessibility to ICTs and it examined students’ perceptions of the ICTs used by professors teaching in a face-to-face traditional postsecondary educational institute (in Canada) and a distance/blended learning higher education institute (in Israel). The sample included 309 Canadian students and 963 Israeli students who completed questionnaires regarding ICT usage, accessibility, and perceived use by professors. Findings reveal that Israeli students reported higher use and greater accessibility of ICTs and they also reported higher use of ICTs by professors. For both groups of students – those with and without LD/ADHD - accessibility to ICTs was predicted by self-reported knowledge and use of ICTs, professors’ ICT use, gender and nationality. The study’s findings and its implications are likely to be important for promoting access to ICTs for students with and without disabilities in both the traditional higher education modality and in distance/ blended learning contexts.     

The Effects of Different Instructions on Student Ratings of University Courses and Teachers

Abstract

The purposes of the present study were to investigate the influence of three sets of instructions, class level, and academic rank on teacher/course evaluation by student raters. Students did not differ in their teacher/course evaluation ratings when the instructions specified the evaluation results would be used: (a) only by the instructor, (b) by the administration, or (c) by students for course selection purposes. The evaluation of graduate courses did not differ from that of undergraduate courses. A statistical difference was found between the academic ranks examined. Specifically, graduate teaching assistants received higher ratings than did either assistant or full professors.