The Relative Merits of Different Types of Overall Evaluations of Teaching Effectiveness

An unresolved issue in student evaluations of teaching effectiveness (SETE) literature is what type of overall evaluation of teaching effectiveness should be used in personnel decisions. The objective of this study is to compare the merits of: (a) an overall evaluation made by students, (b) a weighted average overall evaluation with the weights determined by students, (c) a weighted average overall evaluation with the weights being determined by the individual instructors teaching their respective classes, (d) an unweighted average overall evaluation, and (e) a second-order factor that proxies for an overall evaluation. Our results indicate that: (a) all of these overall evaluations were very highly intercorrelated, (b) the unweighted and weighted average overall evaluations measured virtually the same thing, and (c) the second-order factor that served as an overall evaluation was most highly correlated with the other overall evaluations and had the advantage of being most understandable to the faculty.     

“Top of the Mark”

The present study investigated differences in the effectiveness of instructors from a variety of departments who taught the same course in both intensive and traditional formats within the same year, while controlling for many confounding variables. Results indicated that intensive courses did not significantly differ from traditional courses in overall instructor ratings on student evaluations of teaching effectiveness when confounding variables were taken into account. Conversely, intensive courses received significantly higher overall course ratings on student evaluations than did traditional courses, even after controlling for class size and probable grade in course. These findings provide further evidence that negative beliefs concerning intensive courses may be unjustified, and intensive courses may be as or more effective than those presented in traditional formats.     

How do we rate? An evaluation of online student evaluations

This paper analyses the popular RateMyProfessors (RMP) website where students evaluate instructors in higher education. A study was designed to measure (1) the awareness and utilisation of the RMP website, (2) the internal and external validity of the RMP ratings in measuring teaching effectiveness, and (3) variation in the above across disciplines. It is concluded that the category of ratings, created by the website, establishes an anti‐intellectual tone that manifests itself in comments about instructors’ personality, easiness of workload and entertainment value rather than knowledge attained.     

The Development and Validation of the Student Evaluation of Online Teaching Effectiveness

Abstract

Sponsored by a Preparing Tomorrow's Teachers to Use Technology(PT3) grant, we participated in various activities designed to help us learn to infuse technology in our teacher education courses. The purpose of this paper is to describe the specific impact of the PT3 project activities on our change process-including the formation, evolution, and efforts of our curriculum design team-and to share the activities and products that we developed through participation in the grant activities.     

学生评教的研究现状及启示

学生评教是高校改进教师教学和教学管理工作的重要手段,也是近年来高等教育研究者关注的一个焦点.介绍了国内外学生评教的发展历程,系统地梳理了学生评教的研究重点,即对学生评教的可靠性、有效性、影响因素、指标体系的研究,分析了学生评教实践中存在的诸多问题,得出对学生评教实践的启示,以促进学生评教工作的良性发展.     

Student evaluations of teaching: combining the meta-analyses and demonstrating further evidence for effective use

There is a plethora of research on student evaluations of teaching (SETs) regarding their validity, susceptibility to bias, practical use and effective implementation. Given that there is not one study summarising all these domains of research, a comprehensive overview of SETs was conducted by combining all prior meta-analyses related to SETs. Eleven meta-analyses were identified, and nine meta-analyses covering 193 studies were included in the analysis, which yielded a small-to-medium overall weighted mean effect size (r = .26) between SETs and the variables studied. Findings suggest that SETs appear to be valid, have practical use that is largely free from gender bias and are most effective when implemented with consultation strategies. Research, teaching and policy implications are discussed.     

Honesty on student evaluations of teaching: effectiveness,purpose, and timing matter!

Student evaluations of teaching (SETs) are an important point of assessment for faculty in curriculum development, tenure and promotion decisions, and merit raises. Faculty members utilise SETs to gain feedback on their classes and, hopefully, improve them. The question of the validity of student responses on SETs is a continuing debate in higher education. The current study uses data from two universities (n = 596) to determine whether and under what conditions students are honest on in-class and online SETs, while also assessing their knowledge and attitudes about SETs. Findings reveal that, while students report a high level of honesty on SETs, they are more likely to be honest when they believe that evaluations effectively measure the quality of the course, the results improve teaching and benefit students rather than the administration, and when they are given at the end of the term. Honesty on evaluations is not associated with socio-demographic characteristics.     

Web-Based Social Work Courses: Guidelines for Developing and Implementing an Online Environment

Nationwide, the percentage of faculty who are tenured (or in tenure-earning positions) is declining, with proportionate increases in the amount of instruction provided by adjunct and other part-time instructors, including doctoral students. These trends are mirrored within academic social work and have given rise to some concerns about the potentially adverse effects this could have on the quality of instruction provided to MSW and BSW students. A review of the social work literature, however, failed to locate any systematic investigations published on the topic of evaluating the quality of teaching provided by adjuncts or doctoral students. The authors undertook a comprehensive analysis of the course evaluations obtained from a large urban school of social work in the Southeast over a 3-year period, covering 294 courses (61% BSW, 39% MSW). Of these, 181 classes were taught by regular faculty, 63 classes were taught by community-based adjuncts, and 50 classes by social work PhD students. Inferential tests found no statistically significant differences in the global course evaluations earned by regular faculty or adjuncts. However, there were statistically significant differences in the course evaluations earned by regular faculty and doctoral students, favoring the former, but the effect size was small and of little practical import. In general, the authors found no strong evidence that adjuncts or doctoral students provided less positively evaluated teaching than did regular faculty. The authors' results are limited to one university setting but their research design can be readily adopted by other programs, given the widespread use of quantified student-completed course evaluations generally maintained on university databases. More serious attention needs to be given to determining whether the course evaluations commonly used by universities are truly valid.     

Electronic in-class course evaluations and future directions

This study sought to determine whether an evaluator conducting electronic course evaluations in person in a classroom increased response rates. At a large public university in the United States, course evaluations were being conducted electronically, attaining a response rate of <50% over several semesters. At the end of the Spring 2015 at this university, an evaluator entered several classrooms having a total enrollment of 273 students and guided students through the process of completing course evaluations using their mobile devices. A response rate of 78.4% was achieved. The process described in this study provides evidence that response rates can be increased in similar circumstances. In addition, several ways the technology described could be used in the future for course evaluation are discussed.     

Student evaluations of teaching: perceptions of faculty based on gender,position, and rank

The current study explores the feelings and thoughts that faculty have about their student evaluations of teaching (SET). To assess the perceptions of SETs, all teaching faculty in one college at a western Land Grant University were asked to complete an anonymous online survey. The survey included demographic questions (i.e. gender; rank such as assistant, associate, and full professor; and positions like non-tenure track, tenure track, and tenured) as well as questions related to faculty's feelings while reading their SETs. While minimal differences were found in responses based on rank or position, several differences were found based on faculty gender. Overall, female faculty appear to be more negatively impacted by student evaluations than male faculty. These gender differences support previous research that suggests males and females receive and react differently to personal evaluation. Resultant suggestions include modifying surveys from anonymous to confidential and offering professional development training for faculty.     

Stability and correlates of student evaluations of teaching at a Chinese university

This paper examines the stability and validity of a student evaluations of teaching (SET) instrument used by the administration at a university in the PR China. The SET scores for two semesters of courses taught by 435 teachers were collected. Total 388 teachers (170 males and 218 females) were also invited to fill out the 60‐item NEO Five‐Factor Inventory together with a demographic information questionnaire. The SET responses were found to have very high internal consistency and confirmatory factor analysis supported a one‐factor solution. The SET re‐test correlations were .62 for both the teachers who taught the same course (n = 234) and those who taught a different course in the second semester (n = 201). Linguistics teachers received higher SET scores than either social science or humanities or science and technology teachers. Student ratings were significantly related to Neuroticism and Extraversion. Regression results showed that the Big‐Five personality traits as a group explained only 2.6% of the total variance of student ratings and academic discipline explained 12.7% of the total variance of student ratings. Overall the stability and validity of SET was supported and future uses of SET scores in the PR China are discussed.     

Assessing the reliability of student evaluations of teaching: choosing the right coefficient

Many of the studies used to support the claim that student evaluations of teaching are reliable measures of teaching effectiveness have frequently calculated inappropriate reliability coefficients. This paper points to three coefficients that would be appropriate depending on if student evaluations were used for formative or summative purposes. Results from the present study indicated that students had very low absolute inter-rater reliability, but somewhat higher consistency inter-rater reliability.     

What does ratemyprofessors.com actually rate?

This research looks closely at claims that ratemyprofessors.com creates a valid measure of teaching effectiveness because student responses are consistent with a learning model. While some evidence for this contention was found in three datasets taken from the site, the majority of the evidence indicates that the instrument is biassed by a halo effect, and creates what most accurately could be called a ‘likeability’ scale.     

School social work and early childhood student's attitudes toward gay and lesbian families

In this discussion paper we seek to challenge prevailing wisdoms in higher education regarding the value of measuring teaching quality, prescribing standards for professionalism and using student satisfaction as an indicator of teaching effectiveness. Drawing on the literature, we explore and probe four wisdoms in an attempt to identify and problematise popular assumptions about teaching and professionalism. We suggest that externalising procedures for assessing quality can be counter-productive to effective teaching and learning and propose core values we see as central to enhancing higher education practice: collegial reflection on practice, consideration of ethical issues and risk-taking.     

如何引导新生适应大学英语教学

分析了大学一年级学生在学习英语中存在的问题，从而提出如何帮助和引导新生更快地适应大学英语教学的建议。     

源于实践,归于实践——读王升教授的新著《主体教学有效性探索》

王升教授的新著《主体教学有效性探索》源于实践又归于实践。王升教授在书中首先从形而上的层面探讨了主体教学有效性的目标定位与主体教学运行的动力,确定了主体教学有效性的视阈;然后从形而下的层面制定了主体教学有效性的标准,并探索了确保主体教学有效性的策略;最后通过实验研究验证了这些策略的科学性与合理性。     

Assessment & Evaluation in Higher Education Using the margin of error statistic to examine the effects of aggregating student evaluations of teaching

We proposed an extended form of the Govindarajulu and Barnett margin of error (MOE) equation and used it with an analysis of variance experimental design to examine the effects of aggregating student evaluations of teaching (SET) ratings on the MOE statistic. The interpretative validity of SET ratings can be questioned when the number of students enrolled in a course is low or when the response rate is low. A possible method of improving interpretative validity is to aggregate SET ratings data from two or more courses taught by the same instructor. Based on non-parametric comparisons of the generated MOE, we found that aggregating course evaluation data from two courses reduced the MOE in most cases. However, significant improvement was only achieved when combining course evaluation data for the same instructor for the same course. Significance did not hold when combining data from different courses. We discuss the implications of our findings and provide recommendations for practice.     

Assessing academic program and department effectiveness using student evaluation data

The use of aggregated student evaluations of their courses and course elements (e.g., subject functionality, affect, difficulty, graded assignments) is suggested as an efficient and useful means of obtaining program and department assessments. Given that the instruments used to collect student evaluations are valid (if they are not, they should not be used for any purpose), then averaging class data is likely to provide a valid and reliable index of program and department effectiveness as evaluated by students.Program and department assessment data are presented and discussed for a large northeastern professional school. Large and significant differences in the ratings of program elements were found. Although many of the elements designed into the program by the administration and faculty were perceived as operational by the students, some discrepancies between the design and student perceptions existed. Substantial departmental differences were also found which indicated areas of strength and weakness both within and across departments. The potential usefulness of the assessment for internal change and development is discussed.     

Evaluate to improve: useful approaches to student evaluation

Many teachers in higher education use feedback from students to evaluate their teaching, but only some use these evaluations to improve their teaching. One important factor that makes the difference is the teacher’s approach to their evaluations. In this article, we identify some useful approaches for improving teaching. We conducted focus groups with award-winning university teachers who use student evaluations to improve their teaching, and we identified how they approach their evaluation data. We found that these teachers take a reflective approach, aiming for constant improvement, and see their evaluation data as formative feedback, useful for improving learning outcomes for their students. We summarise this as the improvement approach, and we offer it for other teachers to emulate. We argue that if teachers take this reflective, formative, student-centred approach, they can also use student evaluations to improve their teaching, and this approach should be fostered by institutions to encourage more teachers to use student evaluations to improve their teaching.