Faculty Perspectives on Course and Teacher Evaluations

Student ratings of instruction have been the subject of numerous studies with much of the research focusing on the validity and reliability of the ratings themselves. Comparatively little empirical investigation has been devoted to the perceptions of the individuals who are the subjects of the ratings, that is, the faculty. The current study explored faculty perspectives on the usefulness of student ratings for formative and summative purposes, and the actual use of student ratings for summative purposes. Contrary to what might have been deduced from the anecdotal literature, the results of this study do not portray a great deal of resistance to student ratings in general or to their use for formative and summative evaluation. It was also found that student ratings are actually being used for the latter purpose. The usefulness of the student feedback was viewed differentially by the faculty, with feedback on their interaction with students seen as most useful, followed by feedback on their grading practices, global ratings of instructor and course, and finally structural issues of the course.     

Interpreting and using student ratings data: Guidance for faculty serving as administrators and on evaluation committees

This article is about the accurate interpretation of student ratings data and the appropriate use of that data to evaluate faculty. Its aim is to make recommendations for use and interpretation based on more than 80 years of student ratings research. As more colleges and universities use student ratings data to guide personnel decisions, it is critical that administrators and faculty evaluators have access to research-based information about their use and interpretation.The article begins with an overview of common views and misconceptions about student ratings, followed by clarification of what student ratings are and are not. Next are two sections that provide advice for two audiences—administrators and faculty evaluators—to help them accurately, responsibly, and appropriately use and interpret student ratings data. A list of administrator questions is followed by a list of advice for faculty responsible for evaluating other faculty members’ records.     

Faculty Views of Student Evaluation of College Teaching

The literature abounds with psychometric studies of course evaluation measures and articles debating the merits of student ratings of instruction, but little research has focused on faculty perceptions of this procedure. In the present study faculty perceptions are explored at a teachers' college where evaluation is carried out annually on a sample of courses. The sample includes 101 instructors who completed the research questionnaire. Faculty attitudes reflected a broad range of responses towards validity of student ratings, and their usefulness for improving instruction. Although overall attitudes were mildly positive, few instructors reported changing instruction as a result of student ratings. Moreover, few supported sending evaluation results directly to college administrators or publishing them for student consumption.     

Effects of grading practices and time of rating on student ratings of faculty performance and student learning

The effects of grading practices (strict, lenient) and time of rating (after a lecture, after taking an examination, after receiving feedback on performance on the examination) on student ratings of faculty performance and student learning (i.e., performance on an examination) were assessed in two experimental studies. Results indicated that (1) student ratings were directly affected by grading practices and this effect was a function of the time at which ratings were cornpleted, and (2) student learning and students' ratings of faculty were positively correlated, and grading practices had no effect on student learning. Implications of these findings for the administrative use of student ratings are considered.     

Student Perspectives on Teaching and its Evaluation

The research on student ratings of instruction, while voluminous, has had minimal focus on the perceptions of the students who do the ratings. The current study explored student perspectives on course and teacher ratings as well as some issues related to teaching effectiveness and faculty roles. It was found that students are generally willing to do evaluations and to provide feedback, and have no particular fear of repercussions. However, they have little confidence that faculty or administrators pay attention to the results, and do not even consult the ratings themselves. The students view teaching and advising as the most important roles that should be played by faculty, yet project that faculty, while also viewing teaching as the most important, would rank research above the more student-interactive advising. Canonical correlations among various scales reveal a strong emphasis on such issues of the importance of faculty respect for student views.     

Investigating substantive and consequential validity of student ratings of instruction

This study aimed at exploring faculty and student perspectives on student evaluations, as well as identifying their perceptions of the usefulness and appropriateness of the ratings for evaluating teaching effectiveness. More specifically, the study aimed at identifying the consequences, both intended and unintended, of using the evaluations, in addition to better understanding the process students used in responding to evaluations and what use faculty members made of them. Two surveys were developed and placed on the website of the Office of Institutional Research and Assessment. Emails were sent to all students and faculty participating in evaluations soliciting their cooperation and requesting their input. Faculty and student perceptions were compared qualitatively and statistically. Results revealed that students and faculty believe in the effectiveness and usefulness of the system with the need to overcome some negative consequences and biases inherent in its application at the University. Recommendations for system improvement are provided.     

The Level of Decision-Making,Perceived Influence,and Perceived Satisfaction of Faculty and Their Impact on Student Retention in Community Colleges

ABSTRACT

This study expanded on previous models that looked primarily at student and support factors related to student retention and examined other stakeholder group functions not previously reviewed in relation to student retention. The research question assumed that greater faculty participation in campus decision-making and faculty satisfaction would have been associated with a higher campus student retention rate. The non-experimental retrospective design examined the unanalyzed 2007 community college (CC) Changing Academic Profession (CAP) survey in relation to campus retention rates. The CAP captured faculty perceptions of whether faculty or non-faculty entities were the primary influence on decisions regarding academic matters, as well as a rating of faculty satisfaction. Analysis indicated that CC faculty rated their participation in campus decision-making versus non-faculty entities similarly to the ratings of faculty who had completed the CAP at four-year institutions. When compared with the 2007 retention rate per campus, only faculty job satisfaction was statistically significantly positively associated with student retention. Recommendations for future research include continuing to look at retention as an outcome of a multi-factorial model involving all campus stakeholders, and more research with retention as the outcome.     

The impact of individual consultation on the teaching of younger versus older faculty

This study examines the effect of age of instructor on student ratings of teaching performance after individual consultation. Instructors self-presented over an 11-year period at a large Canadian university teaching service, where each received one of three interventions. End of term student ratings of the teaching of younger and older instructors are compared before consultation, immediately post consultation, and 1–3 years after year of consultation. Younger faculty obtained significantly improved ratings immediately after consultation, while older faculty achieved significant rating increases 1–3 years post consultation. Results from an earlier study on the impact of individual consultation on teacher ratings are reevaluated using this larger sample of faculty. Generally, results in this analysis parallel the original research. Consultation produced changes in student ratings both immediately after consultation as well as longitudinally, thus confirming the utility of intervention in producing enduring pedagogical improvements. Control analyses ensured that improvements were a result of the interventions and not an artefact of time.     

On the use of student ratings of faculty in determination of pay,promotion and tenure

Student ratings of faculty have traditionally been obtained in a manner designed to be useful to the individual faculty member or to other students. It is now sometimes proposed that the resulting data be used to determine faculty pay, promotion, and tenure. Recent articles and the reviews of past literature on ratings are analyzed to determine whether student ratings are usually associated with teaching effectiveness, whether they are sometimes biased by irrelevant factors, whether faculty can effectively revise their behaviors to improve their ratings, and whether improved ratings result in improved teaching effectiveness. The results tend to indicate that student ratings of faculty as they are currently collected cannot be trusted for considerations of pay, promotion, and tenure.     

Student Evaluation of Faculty: Effects of Purpose on Pattern

Much of the existing research addressing student ratings of faculty has focused on psychometric properties of the rating process or on physical characteristics of courses and individuals. Less explored is the cognitive processes of students, as evaluators, and the administrative purposes for which outcomes from student evaluations are used. To determine if students used different cognitive sets as a function of the explicit purpose for which ratings are obtained, a field experiment was conducted. Explicit purposes for course evaluations were manipulated to include a control condition, a formative condition (instructional improvement), and a summative condition (salary increase). These conditions served as the classification variable in a discriminant analysis and six different dimensions of classroom performance, as assessed by students, served as the discriminating variables. Results indicate that the purpose of assessment fails to influence the pattern of faculty ratings provided by students. From an administrative perspective, these findings have practical implications both for the collection of student evaluations and for the use of student ratings.     

Student ratings of college teaching: a comparison of faculty and their students

Based on the review of student ratings myths by Aleamoni (1987, 1999), a survey research design was used to analyse differences between college students' (n = 968) and faculty's (n = 34) perceptions. Generally, students held stronger beliefs in these myths, in that they believed faculty with excellent publication records were better qualified to evaluate teaching and that student ratings on single general items are accurate measures of teaching effectiveness. On the other hand, faculty believed that student ratings were invalid and unreliable. Further examination of student characteristics revealed that male students held stronger beliefs in these myths. Finally, students' beliefs in these myths were correlated with their actual ratings of nine dimensions of the Student Evaluation of Educational Quality. A discussion as well as suggestions for using student ratings is provided.     

In-Service Improvement of Teaching

We examine the validity of peer observation of classroom instruction for purposes of faculty evaluation. Using both a multi-section course sample and a sample of different courses across a university's School of Business and Economics we find that the results of annual classroom observations of faculty teaching are significantly and positively correlated with student learning outcome assessment measures. This finding supports the validity of classroom observation as an assessment of teaching effectiveness. The research also indicates that student ratings of teaching effectiveness (SETEs) were less effective at measuring student learning than annual classroom observations by peers.     

The localization,diagnostic, and monitoring functions of student ratings in a model for improving university teaching

A diversity of student questionnaires are used by colleges and universities to provide data on faculty teaching performance. Yet the purposes for collecting this data are frequently unclear, and at times superficial. Rarely is student rating data used as a tool to improve faculty teaching. A more relevant approach incorporates a variety of types of student ratings into a model for improving university teaching. One type of student rating data is used to identify broad instructional problem areas. Another type pinpoints probable causes and solutions for the instructional problems. Instructional improvement procedures are designed on the basis of this data. A third type of student rating data evaluates the instructional improvement procedures and indicates when modifications are needed. In addition to these three types of student ratings, and the generation of appropriate questionnaires, this paper presents an overview of the teaching improvement model and discusses its effectiveness.     

Student evaluation of instruction in higher education: exploring issues of validity and reliability

Many personnel committees at colleges and universities in the USA use student evaluation of faculty instruction to make decisions regarding tenure, promotion, merit pay or faculty professional development. This study examines the construct validity and internal consistency reliability of the student evaluation of instruction (SEI) used at a large mid‐western university in the USA for both administrative and instructional purposes. The sample consisted of 73,500 completed SEIs for undergraduate students who self‐reported as freshman, sophomore, junior or senior. Confirmatory factor analysis via structural equation modelling was used to explore the construct validity of the SEI instrument. The internal consistency of students' ratings was reported to provide reliability evidence. The results of this study showed that the model fits the data for the sample. The significance of this study as well as areas for further research are discussed.     

RECENTLY PUBLISHED DISSERTATIONS ON COMMUNITY AND JUNIOR COLLEGES

The use of student satisfaction measures remains an almost universal practice in higher education despite controversy regarding their usefulness and validity. Even with their prevalent use, and the importance placed on them for outcomes assessment and faculty evaluation, little research has examined personality factors that influence students while completing satisfaction measures. The role that emotional awareness plays in the evaluative judgments students make was the focus of the current study. Results indicate that community college student satisfaction ratings are influenced by the interaction of gender, emotional awareness, and awareness of physiological states.     

An Evaluation of Factors Regarding Students’ Assessment of Faculty in a Business School*

Student faculty ratings are used at most institutions of higher learning for three important reasons. First, the ratings provide direct feedback to the faculty, and this enables faculty to adjust their teaching styles. Second, the ratings provide the administration with information intended to assist in guiding and mentoring faculty toward more effective pedagogical performance in the classroom. Third, the ratings also provide the administration with information to be used in the reappointment, tenure, and promotion processes, as well as for assignment of salary range adjustments and teaching awards. To be of real value, however, all of this is predicated on the use of a valid and reliable faculty‐rating instrument along with a system designed to provide both the faculty and the administration with norming reports that allow for appropriate comparisons of ratings. This article reports such a study conducted within a large department of a business school and recommends that the process used be adapted by other business school departments and other academic units across the university and at other universities to ensure a more universally appropriate usage of students’ ratings.     

Examining the validity of office discipline referrals as an indicator of student behavior problems

Office discipline referral (ODR) data are increasingly used to monitor student behavior problems and the impact of interventions, but there has been limited research examining their validity. The current study examined the concordance of ODRs with teacher ratings of student behavior using data on 8,645 children in 335 classrooms at 21 elementary schools. The results of a variety of analyses (e.g., correlations, multivariate analysis of variance, receiver operating characteristics) suggested that ODRs are moderately valid and reliable. Multilevel analyses revealed that teacher ratings of disruptive behaviors were significantly associated with ODRs, even after controlling for other student‐, classroom‐, and school‐level factors. These findings suggest that ODRs are moderately valid indicators of student behavior problems and may be an efficient source of information for use in school‐based research and data‐based decision‐making. © 2011 Wiley Periodicals, Inc.     

Mass communication and journalism faculty’s perceptions of the effectiveness of email communication with college students: a nationwide study

Nearly 700 US journalism and mass communication faculty (all teaching personnel) reported their perceptions of student email use via a web‐based survey. This nationwide study focused on the content of email sent by faculty to students, email’s effectiveness, and email’s effect on student learning. Comparisons were made based on faculty gender, rank, age, and ethnicity. Findings suggest that despite statistical differences, when gender, rank, age, or ethnicity are considered, faculty are not in the habit of sending course materials like syllabi, project instructions, and lecture notes to students personally via email. Moreover, faculty tend to find favor with email communication and its effectiveness as a tool of teaching. The results of this survey coupled with previous research by the authors and other scholars suggest faculty ought to embrace the technology and develop positive ways to incorporate email, as well as other technology, into the educational process.     

A survey of selected administrative practices supporting student evaluation of instruction programs

A mail survey of 333 American universities was conducted to assess the current status of Student Evaluation of Instruction. Based on a 68% return, it was concluded that there has been an increase in the popularity of student ratings as a means of evaluating faculty performance, as well as an increase in the frequency with which evaluation results are used in decisions concerning faculty status. However, at most universities, research on the rating instruments does not seem to have kept pace with the decisions that are based on the rating instruments.