高等学校学生评教行为偏差研究——基于南京仙林大学城若干大学的分析

学生评教是高校教学质量保障体系的重要组成部分.目前,学生在评教过程中存在评教行为偏差,主要表现为引致评教、恶意评教、随意评教和放弃评教.其直接原因在于高校在学生评教管理中的若干强制规定和院系对学生评教的不当引导;根本原因则是由学生评教定位偏差以及学生评教运作无关学生利益造成的.因此,高校需要科学定位学生评教,极力完善学生评教的制度设计,构建学生评教和学生利益紧密关联的基点,以便促成教师根据学生评教改进教学和服务学生、满足学生学习需要这一根本目的的实现.     

Evaluation of university teaching as sound performance appraisal

As research on student evaluations of teaching (SET) has dominated our understanding of teaching evaluation, debate over SET implementation has turned attention away from basic principles for appraising employee performance as established in human resources literature. Considering SET research and its practical significance in isolation from relevant human resources literature not only risks unlawful remedies for issues such as bias in SET but also risks replacing one form of bias with another. Meanwhile, the full potential of human resources tools to support consistent evaluation of teaching remains unrealized. To address this gap, this article clarifies how teaching evaluation can be conducted as sound performance appraisal by deploying SET and peer review of teaching within a larger framework of established human resources techniques. A review of recent literature articulates prominent themes in research on SET and peer review of teaching and outlines key principles for performance appraisal and performance management. Those principles are used to analyze representative faculty evaluation policies and procedures and clarify the weaknesses of both traditional and recently revised approaches to teaching evaluation. The final section elaborates performance appraisal techniques relevant to teaching evaluation. These include planful use of results and/or behavior approaches to performance appraisal, robust rating instruments for behavioral performance appraisal, targeted collection of information from multiple stakeholders, and job analysis. Efforts to de-emphasize quantitative SET data to address issues such as bias can be strengthened through the incorporation of performance appraisal tools that clearly articulate performance criteria and standards and that gather both qualitative and quantitative data on employee performance.     

Acquiescence,instructor’s gender bias and validity of student evaluation of teaching

Abstract

The validity of student evaluation of teaching (SET) scores depends on minimum effect of extraneous response processes or biases. A bias may increase or decrease scores and change the relationship with other variables. In contrast, SET literature defines bias as an irrelevant variable correlated with SET scores, and among many, a relevant biasing factor in literature is the instructor’s gender. The study examines the extent to which acquiescence, the tendency to endorse the highest response option across items and bias in the first sense affects students’ responses to a SET rating scale. The study also explores how acquiescence affects the difference in teaching quality (TQ) by instructor’s gender, a bias in the latter sense. SET data collected at a faculty of education in Ontario, Canada were analysed using the Rasch rating scale model. Findings provide empirical support for acquiescence affecting students’ responses. Latent regression analyses show how acquiescence reduces the difference in TQ by instructor’s gender. Findings encourage greater attention to the response process quality as a way to better defend the utility of SET and prevent potentially misleading conclusions from the analysis of SET data.     

高校学生评教效果的影响因素研究

学生评教作为评价教师教学的一种重要手段受到了高校的普遍重视,并被广泛运用到高校的各项管理之中.但是学生评教是否真正能从本质上反映教学质量一直存在争议,教师因素、学生因素及课程自身因素都有可能影响到学生评教的效果.对学生评教效果影响因素的分析和探讨,有助于高等院校优化学生评教机制,促进教学质量的提高.     

试论信息不对称与高等学校学生评教

学生评教是高等学校教学质量管理的关键举措.在其运作过程中存在诸多信息不时称现象,它会造成学生和教师的逆选择及败德行为.根据信息不对称理论对学生评教的分析,学生、教师的逆选择和败德行为的发生不仅与评教过程中的信息不对称有关,还与目前学生评教制度运作的不科学有关.所以,完善学生评教制度,需要促成教学信息在管理方、教师和学生之间的对称配置,同时应使学生评教制度回归到服务学生和实现学生利益这一基点上.     

Making sense of module feedback: accounting for individual behaviours in student evaluations of teaching

Quantitative student evaluations of teaching (SET) and assessments are widely used in higher education as a proxy for teaching quality. However, SET are a function of individual rating behaviours resulting from student background, knowledge and personalities, as well as the learning experience being rated. SET from three years of data from a science department at a Russell Group University in the UK were analysed to highlight issues of sample size in relation to variable perceptions of modules, and develop a statistical model of feedback incorporating individual rating behaviours across modules. Key results are that sample size and individual rating behaviours have the potential to significantly affect summary module ratings, especially for <20 respondents or if individuals have heterogeneous views. A new approach is suggested, to interpret and compare quantitative module ratings, acknowledging uncertainty, variability and individual rating behaviours. This has implications for the interpretation of SET in many aspects of academic life, including university league table positions, the identification of good teaching practice with respect to student satisfaction, and the weight given to SET in individual academics’ promotion applications.     

Whose feedback? A multilevel analysis of student completion of end-of-term teaching evaluations

Student evaluation of teaching (SET) is now common practice across higher education, with the results used for both course improvement and quality assurance purposes. While much research has examined the validity of SETs for measuring teaching quality, few studies have investigated the factors that influence student participation in the SET process. This study aimed to address this deficit through the analysis of an SET respondent pool at a large Canadian research-intensive university. The findings were largely consistent with available research (showing influence of student gender, age, specialisation area and final grade on SET completion). However, the study also identified additional influential course-specific factors such as term of study, course year level and course type as statistically significant. Collectively, such findings point to substantively significant patterns of bias in the characteristics of the respondent pool. Further research is needed to specify and quantify the impact (if any) on SET scores. We conclude, however, by recommending that such bias does not invalidate SET implementation, but instead should be embraced and reported within standard institutional practice, allowing better understanding of feedback received, and driving future efforts at recruiting student respondents.     

高校学生评教与教师选择制度——基于“以足投票”理论的分析

学生评教制度是高校教学质量管理体系的有机组成部分,但从实际情况看,它受到了学生的漠视和教师的批评,问题的症结可能在于当前学生评教的运作与学生利益并无关系。根据“以足投票”理论,结合学生评教制度设计现状分析认为,高校需确立学生自由选择教师的制度安排,公开学生评教信息,控制学生规模,改革有关学生学籍管理的制度。     

Online student evaluations of teaching: what are we sacrificing for the affordances of technology?

In the context of increased emphasis on quality assurance of teaching, it is crucial that student evaluations of teaching (SET) methods be both reliable and workable in practice. Online SETs particularly tend to raise criticisms with those most reactive to mechanisms of teaching accountability. However, most studies of SET processes have been conducted with convenience, small and cross-sectional samples. Longitudinal studies are rare, as comparison studies on SET methodological approaches are generally pilot studies followed shortly after by implementation. The investigation presented here significantly contributes to the debate by examining the impact of the online administration method of SET on a very large longitudinal sample at the course level rather than attending to the student unit, thus compensating for the inter-dependency of students’ responses according to the instructor variable. It explores the impact of the administration method of SET (paper based in-class vs. out-of-class online collection) on scores, with a longitudinal sample of over 63,000 student responses collected over a total period of 10 years. Having adjusted for the confounding effect of class size, faculty, year of evaluation, years of teaching experience and student performance, it is observed that the actual effect of the administration method exists, but is insignificant.     

高校如何提升学生评教效度? ——澳大利亚高校学生评教最大差异量规及其启示

学生评教是高校汇集学生反馈、监测教育教学质量的重要手段。传统的学生评教量规大多由管理者、教学论专家开发,学生依据现成量规的评教指标打分,完成对授课教师的评价,其优点是指标体系明确、操作方便、统计简单,问题是由管理者、教学论专家拟定的教学评价指标不能涵盖学生认为重要的指标。针对上述问题,澳大利亚学者提出最大差异量规,从课程与教学维度评价高校教师教学质量,采用最好-最差配对选择,使评教差异最大化,包括对高校教师课程与教学绩效的评价,还可以对评价量规本身的绩效-重要性程度进行评价,其结果能更有效地反映授课教师的教学质量,动态地反映学生本身的学习发展需求,提升高校学生评教的效度和价值,对提高我国高校学生评教的科学性有借鉴价值。     

Teachers’ perceptions of student evaluations of teaching

Evaluation of teaching in higher education has drawn much attention due to the need for greater accountability and improvement in student learning. Our review of literature on Student Evaluation of Teaching (SET) surveys suggests that considerable controversy and criticism have surrounded its use, fairness, and validity. Yet, many universities in Hong Kong are depending primarily on student feedback in SET surveys to provide evidence for their improvement of teaching and learning, and for their decisions made on teachers' pay-rise and tenure. If universities are to continue to use SET surveys as the main source of mechanism for measuring teaching effectiveness, teachers will have to understand and be aware of its purposes. But how much do we know about teachers' perceptions of teaching and learning and, consequently, their opinions of using SET surveys as the primary standardized evaluation mechanism of teaching for promotion and tenure and for teaching effectiveness?     

The performance of a student evaluation of teaching system

Student evaluation of teaching (SET) is now commonplace in many universities internationally. While much effort has been devoted to examining the statistical validity of SET instruments, there has been limited examination of the methodological and consequential validity (together referred to as ‘utility’) of the ways in which SET data are used. This paper examines the SET system at Deakin University from the perspective of utility. It draws on publicly available SET results for an entire annual cycle of unit offerings. Consideration is given to the representativeness of the data produced, and to the utility of the data reported, by the system. While this investigation focuses on the SET system currently employed at Deakin University, it offers both an analysis methodology and conclusions that can be applied more generally.     

Academics’ Perceptions of Good Teaching: Assessing the Degree of Parity with Student Evaluation of Teaching Questionnaires

ABSTRACT

There is a dearth of studies on academics’ perceptions of good teaching in transitional economies such as Uganda and the degree of parity between academics’ conceptions of good teaching and the items in the student evaluation of teaching (SET) questionnaires. Against this backdrop, the article reports on a study that explored how academics at Makerere University, Uganda, perceive good teaching and compared the resultant perceptions with the items in the SET questionnaires. The study employed a qualitative approach and data was collected by using semi-structured interviews and reviewing documents. Thematic analysis was employed to analyse the data from the interviews while the data from the documents was analysed using content analysis. The findings showed that academics perceive good teaching as: being knowledgeable; being student-centred; demonstrating good communication skills; undertaking research-based teaching; demonstrating professionalism; being approachable; and being organised. Finally, the findings demonstrated a convergence between academics’ perceptions of good teaching and most of the items in the SET questionnaires.     

Predicting student evaluations of teaching using decision tree analysis

Abstract

This study uses decision tree analysis to determine the most important variables that predict high overall teaching and course scores on a student evaluation of teaching (SET) instrument at a large public research university in the United States. Decision tree analysis is a more robust and intuitive approach for analysing and interpreting SET scores compared to more common parametric statistical approaches. Variables in this analysis included individual items on the SET instrument, self-reported student characteristics, course characteristics and instructor characteristics. The results show that items on the SET instrument that most directly address fundamental issues of teaching and learning, such as helping the student to better understand the course material, are most predictive of high overall teaching and course scores. SET items less directly related to student learning, such as those related to course grading policies, have little importance in predicting high overall teaching and course scores. Variables irrelevant to the construct, such as an instructor’s gender and race/ethnicity, were not predictive of high overall teaching and course scores. These findings provide evidence of criterion and discriminant validity, and show that high SET scores do not reflect student biases against an instructor’s gender or race/ethnicity.     

Beyond Average: Contemporary statistical techniques for analysing student evaluations of teaching

Student evaluations of teaching (SETs) have been used to evaluate higher education teaching performance for decades. Reporting SET results often involves the extraction of an average for some set of course metrics, which facilitates the comparison of teaching teams across different organisational units. Here, we draw attention to ongoing problems with the naive application of this approach. Firstly, a specific average value may arise from data that demonstrates very different patterns of student satisfaction. Furthermore, the use of distance measures (e.g. an average) for ordinal data can be contested, and finally, issues of multiplicity increasingly plague approaches using hypothesis testing. It is time to advance the methodology of the field. We demonstrate how multinomial distributions and hierarchical Bayesian methods can be used to contextualise the SET scores of a course to different organisational units and student cohorts, and then show how this approach can be used to extract sensible information about how a distribution is changing.     

国外高校学生评教研究进展

对国外高校学生评教导致学生评教产生偏误的非教学因素,以及学生评教实施过程中的保密性、匿名性、信息采集手段、结果的解读和利用等问题作一个综述。研究发现国外非常重视评教中心理层面因素研究以及科学的研究设计和统计方法的应用。提出应合理解读及利用学生评教结果,注意学生评教实施过程中的保密性,将学生评教与其他手段结合起来形成全面的质量信息体系等建议。     

基于学生视角的中小学评教工具系统构建

教师专业发展需要倾听学生的声音,但是当前国内中小学学生评教系统主要是满意度测评或基于专家评教改编而来的学生评教问卷。由于其设计理念是基于经验,编制视角是成人中心,使用目的是绩效评估,导致学生评教结果不能指向教师专业发展、教师不愿被评教、师生关系异化等后果。因此,良好的学生评教工具系统的建构原则应该是基于实证的理论建构、学生中心的编制视角、实用主义的使用目的。以此为指导尝试建构了三个模块共九个维度的学生评教工具系统。     

An Assessment of Statistical Process Control-Based Approaches for Charting Student Evaluation Scores

We compare three control charts for monitoring data from student evaluations of teaching (SET) with the goal of improving student satisfaction with teaching performance. The two charts that we propose are a modified p chart and a z‐score chart. We show that these charts overcome some of the shortcomings of the more traditional charts for analyzing SET data. A comparison of three charts (an individuals chart, the modified p chart, and the z‐score chart) reveals that the modified p chart is the best approach for analyzing SET data because it utilizes distributions that are appropriate for categorical data, and its interpretation is more straightforward. We conclude that administrators and faculty alike can benefit by using the modified p chart to monitor and improve teaching performance as measured by student evaluations.     

Assessing the validity and reliability of a quick scan for student's evaluation of teaching. Results from confirmatory factor analysis and G Theory

Using confirmatory factor analysis and G Theory analysis, this article explores the reliability and the validity of a short version of the SET37 questionnaire for students’ evaluation of teaching (SET). The results show that this instrument can be used as a valuable diagnostic instrument for gathering student feedback in internal practices and procedures aimed at both monitoring and improving the quality of instruction in higher education.