The relative importance of teaching evaluation criteria from the points of view of students and faculty

Abstract

The student evaluation of teaching (SET) tool is widely used to measure student satisfaction in institutions of higher education. A SET typically includes several criteria, which are assigned equal weights. The motivation for this research is to examine student and lecturer perceptions and the behaviour of the students (i.e. ratings given by them to lecturers) of various criteria on a SET. To this end, an analytic hierarchy process methodology was used to capture the importance (weights) of SET criteria from the points of view of students and lecturers; the students' actual ratings on the SET were then analysed. Results revealed statistically significant differences in the weights of the SET criteria; those weights differ for students and lecturers. However, analysis of 1436 SET forms of the same population revealed that, although students typically rate instructors very similarly on all criteria, they rate instructors higher on the criteria that are more important to them. The practical implications of this research is the reduction of the number of criteria on the SETs used for personnel decisions, while identifying for instructors and administrators those criteria that are perceived by students to be more important.     

Competencies of postsecondary education personnel serving students with learning disabilities

This paper presents the results of a survey designed to identify the needed competencies of both administrative and direct service personnel in directing and implementing postsecondary support programs for students with learning disabilities. Data were collected from a national sample of 299 practitioners. In addition to information about respondent characteristics, results include ratings for all items on the survey. Competency areas perceived as most desired by learning specialists were assessment skills, cognitive interventions, and instructional skills, while administrative personnel rated management/leadership skills as most desired. Implications for professional development activities are addressed. The need for strengthening linkages between secondary and postsecondary personnel to foster effective transition planning for students with learning disabilities is also explored.     

Workplace learning: principles and practice

Students in part‐time courses were interviewed about their perceptions of good teaching and tutoring. The perceptions differed markedly between those with reproductive conceptions of learning and students holding self‐determining ones. The former preferred didactic teaching but disliked interaction, whereas the latter had almost diametrically opposite perspectives by finding student‐centred approaches consistent with their conceptions of learning. The findings have implications for the evaluation of teaching, as ratings are likely to be influenced by the predominant conceptions of learning of a class. It is common for individual instructors to be regularly evaluated by teacher evaluation questionnaires, which often have a teacher‐centred bias, and for the ratings to be used for appraisal. It is argued that this leads to conservatism as teachers fear that students with reproductive conceptions of learning will reduce their ratings if they innovate in their teaching. As the degree of bias from this ratings‐lowering phenomenon may be quite large, the findings are a caution against the common practice of using absolute rating values from both teacher evaluation questionnaires and programme‐level evaluation by instruments such as the Course Experience Questionnaire. Results need to be interpreted together with other evidence and take into account contextual factors including students' conceptions of learning.     

Students' Evaluation of Teachers and Instructional Quality--Analysis of Relevant Factors Based on Empirical Evaluation Research

This paper explores the factors that determine the student evaluation of teachers. Unlike many previous studies, the effect of potential biases on global ratings is examined in the context of teaching behaviour (that should in fact--in contrast to the potential biases--have an influence on global ratings). Our research takes into account that biases like the students' interest in the subject or their liking for the teacher may be a result of good teaching behaviour and may not be considered a mere bias of student ratings. Furthermore, it also concentrates on the students' attitudes towards evaluating their teachers. The empirical results are based on qualitative interviews with 40 students at Austrian commercial colleges and on a quantitative survey of 2,121 students who were asked to evaluate their Accounting teachers. Structural equation models show that the students' global ratings of teachers mainly depend on their teaching behaviour. However, global ratings are also affected by the students' attitudes towards evaluating their teachers, as well as by the students' liking for their teacher and their interest in the subject of Accounting.     

What do students consider useful about student ratings?

Although measures of student ratings of instruction have been developed and examined empirically, students' attitudes about the utility of these ratings have yet to be meaningfully quantified. Using survey responses from 1229 students at a major Canadian university, we developed a psychometrically sound measure of the usefulness of student ratings. A principal components analysis with oblique rotation was conducted and three factors (Instructor characteristics, Course characteristics and Instructor's relative ranking) were obtained that explained 64% of the variance in six iterations. The usefulness of these three types of ratings information differs according to several student and course characteristics. The present results provide evidence for both reliability and validity of the instrument as an indicator of the utility of teaching evaluation for students.     

Student evaluation of instruction in higher education: exploring issues of validity and reliability

Many personnel committees at colleges and universities in the USA use student evaluation of faculty instruction to make decisions regarding tenure, promotion, merit pay or faculty professional development. This study examines the construct validity and internal consistency reliability of the student evaluation of instruction (SEI) used at a large mid‐western university in the USA for both administrative and instructional purposes. The sample consisted of 73,500 completed SEIs for undergraduate students who self‐reported as freshman, sophomore, junior or senior. Confirmatory factor analysis via structural equation modelling was used to explore the construct validity of the SEI instrument. The internal consistency of students' ratings was reported to provide reliability evidence. The results of this study showed that the model fits the data for the sample. The significance of this study as well as areas for further research are discussed.     

层次分析法在《绩效管理》课程的教学思路及案例解析

《绩效管理》课程是人力资源管理专业的专业核心课程之一,该课程的重点内容是利用定性和定量方法对组织内的人员进行绩效考核与评价。由于此部分内容涉及较多的统计学的综合评价分析原理及步骤,在教学过程中师生普遍反映有一定的难度。究其原因,主要在于学生未能将综合评价分析方法与绩效考核流程相结合。文章归纳总结了综合评价方法在绩效考核课程的运用思路,其中关键点是构建评价指标体系和估算权重,并通过案例展示了层次分析法(AHP)在绩效考核中的教学思路及步骤以便于教学。     

南开大学学生评教工作的开展及有效性研究

本文介绍了近4年来南开大学基于学生信息员队伍的学生评教工作开展的基本情况、多元机制和评教信息反馈模式,同时论述了通过学生信息员工作进行随堂评教的有效性及其在促进学校教学质量提升方面的意义。针对学校进一步推进学生评教工作,让学生参与学校教学管理工作,充分发挥其在教学质量监控体系中的重要作用,本文提出了几点建议。     

Single comments or average ratings: which elements of RateMyProfessors.com™ shape university students’ judgments and course choice intentions?

The use and abuse of course and lecturer rating websites such as RateMyProfessors.com? is a highly relevant topic for universities’ evaluation and assessment policies and practice. However, only a few studies have paid attention to the actual influence of teaching evaluation websites on the students themselves—that is, their perceptions of a certain course and their course choice intention at university. Findings point to the fact that positive comments on the website about professors improve students’ evaluations. However, professor evaluation websites contain two types of information: single student comments and average ratings. Research on exemplification effects has shown that single cases often have a stronger influence on recipients than more valid base rate information. We test this assumption in an experiment (n?=?126) using a professor evaluation website stimulus. Results show that single comments strongly influence opinions and course choice intentions but that they are moderated by the valence of the average rating.     

Performance Appraisal: Issues of Validity

Performance ratings are probably the most widely used criteria in personnel evaluation. To determine the psychometric quality of performance ratings, practitioners generally evaluate ratings in terms of reliability and validity. This article presents generalizability theory as a measurement procedure that can be used to assess and improve the reliability and validity of ratings of employee job performance.     

Student assessment of teaching in higher education

Plans to introduce campus-wide assessments of college or university teaching which are largely dependent on student ratings are seen as a threat to academic freedom in those institutions with little or no experience of this form of evaluation. While regular student evaluations of teaching are very common in North America, their introduction is only now being considered in colleges and universities in a number of other countries. Research on the reliability and validity of student ratings indicate that they are capable of providing valuable information about the quality of teaching. Depending on the survey used, this type of evaluation may be used to provide evidence of teaching ability to staffing committees or to suggest ways of improving teaching. The paper concludes with a set of recommendations for higher education institutions which are considering the regular assessment of all teachers by their students.     

会计人员学习能力和职业判断能力的培养

会计职业能力是会计职业者从事会计职业应具备的基本素质。本文主要探讨知识经济时代会计人员面临的挑战和要求,论述如何在会计教育中从知识传授向能力培养转变,使学生具备终身自我学习的能力;树立会计职业判断意识,从而具备会计职业判断能力。     

Factors associated with staff perceptions towards inclusive education in Singapore

In this study, we sought to examine the perceptions of teachers and other school professionals towards the inclusion of secondary school students with special educational needs (SEN), and the associated factors. The Sentiments, Attitudes and Concerns about Inclusive Education Revised scale (SACIE-R) was completed by 131 teachers and school professionals from two mainstream secondary schools in Singapore. The findings revealed an overall neutral attitude towards inclusion. Together, confidence in teaching students with SEN, the level of training SEN support, as well as experience teaching students with SEN account for a large proportion of the variance in ratings of inclusive perceptions. Further analyses revealed that confidence in teaching or supporting students with SEN was found to be a significant predictor of inclusive perceptions. The findings suggest that schools seeking to engage in inclusive practices should focus on ways to develop the confidence of personnel to support students with SEN. Professional development and mentorship were suggested as possible avenues.     

SELF AND STUDENT EVALUATION OF ACADEMIC STAFF

At the University of Queensland a questionnaire to students for their evaluation of tutors’ and lecturers’ classroom presentation and classroom management has been in (increasing) use since 1982. The introduction of the evaluation questionnaire and the management of the evaluation system are based on the research lieterature on change. Student evaluations per se do not induce change. However, self evaluations focus staffs attention on their own perception as teachers, and possible discrepancies between self and student evaluation may then motivate staff to change.

Therefore self evaluation has been used in the evaluation of teaching schemesto facilitate change. Overall, there was no relationship between student ratings and staff self ratings on the question, “All things considered, how would you rate this staff member's overall effectiveness as a university teacher?” Both highly and poorly rated lecturers showed large discrepancies between their self perception and student perception. This emphasises the importance of using more than one source of evaluative information for decision makeing. An interview study found that nearly all those evaluated had implemented changes and felt positive about evaluation.     

The role of scales in student ratings

Questions used in course evaluations should mainly measure the quality of teaching, and students' answers to those questions should not be influenced by other factors. This paper investigates how seemingly neutral rating scales and multiple-choice questions might have an impact on the results of such evaluations. In several studies, it has been shown that the way in which a scale is constructed may strongly influence the answers it elicits in surveys and tests. Whether and to what extent common course-related ratings of students are also affected by the kinds of scales used is the main topic of this paper. Four studies examined the influence of scale polarity (unipolar vs. bipolar scales), the role of different ranges in time scales, and the impact of ordering choices in a certain way. Participants' ratings and answers were strongly influenced by all these manipulations. It is recommended to pay special attention to the role of scales when constructing questionnaires for course evaluations and when interpreting course evaluation reports.     

Criterion-Related Validity Evidence for a Standards-Based Teacher Evaluation System

This article reports evidence of the criterion-related validity of a standards-based teacher evaluation system implemented by the Cincinnati Public Schools. The analyses described provide evidence for a moderate level of criterion-related validity of the teacher evaluation ratings. The results indicate that teachers who received the lowest ratings on the Teaching for Learning domain had students who performed lower on state and district tests than predicted based on prior year test scores. Students of teachers who received the highest ratings on this domain generally performed better than predicted. Correlations (Kendall’s tau) between composite scores (the sum of four domain scores) and student achievement ranged from 0.28 to 0.37, depending on subject tested, and were statistically significant at conventional levels.A previous version of this paper was presented at the Annual Meeting of the American Evaluation Association, November 8, 2002 in Washington, D.C.     

论冯友兰的教育思想及其民族内涵

冯友兰认为，生产社会化使教育打破了家庭、民族和社会的界限，各民族平等受教育而共同进步成为可能。生产社会化所带来的细密分工和专业化，使学生道德品质的培育主要依靠社会之“化”。社会对专业人才的需求，要求学生只有根据自己的兴趣、才学进行学习方可取得大的成就。     

Exploring graduate and undergraduate course evaluations administered on paper and online: a case study

The purpose of this study was to analyse the students’ evaluations of the course and instructor for all statistics courses offered during fall semester 2009 at a large university in the southern United States. Data were collected and analysed for course evaluations administered both online and on paper to students in both undergraduate and graduate courses. Unlike most previous studies on this subject, class section rather than student was treated as the unit of analysis. It was of specific interest to verify prior research findings that evaluation surveys administered online would not result in lower course and instructor ratings and lower response rates. The results showed that there is not sufficient evidence within the collected data to conclude that either course and instructor ratings or response rates are lower for evaluations administered online (online evaluations) than they are for evaluations administered on paper (paper evaluations). Of secondary interest was whether class ratings would be associated with students’ attendance and a comparison of variability among answers for undergraduate vs. graduate students. It was observed that class and teacher ratings were not related to students’ attendance and individual students did not tend to give the same answer for every question on their survey.     

Student evaluations of teaching and courses: Student study strategies as a criterion of validity

Recent developments in higher education are likely to lead to increased evaluation of teaching and courses and, in particular, increased use of student evaluation of teaching and courses by questionnaire. Most studies of the validity of such evaluations have been conducted in terms of the relationship between traditional measures of how much students learn and their ratings of teaching and courses. But there have been few if any studies of the relationship between students' rating of teaching and the quality of student learning, or in how the students approached their learning.For the evaluation of teaching and courses by questionnaire to be valid we would expect that (1) those students reporting that they adopted deeper approaches to study would rate the teaching and the course more highly than those adopting more surface strategies and, more importantly, (2) those teachers and courses which received higher mean ratings would also have, on average, students adopting deeper strategies.In the paper we report the results for eleven courses in two institutions. The results, in general, support the validity of student ratings, and suggest that courses and teaching in which students have adopted deeper strategies to learning also have higher student ratings.An earlier version of this paper was presented at the 1989 Annual Conference of the Higher Education Research and Development Society of Australia.