The impact of individual consultation on the teaching of younger versus older faculty

This study examines the effect of age of instructor on student ratings of teaching performance after individual consultation. Instructors self-presented over an 11-year period at a large Canadian university teaching service, where each received one of three interventions. End of term student ratings of the teaching of younger and older instructors are compared before consultation, immediately post consultation, and 1–3 years after year of consultation. Younger faculty obtained significantly improved ratings immediately after consultation, while older faculty achieved significant rating increases 1–3 years post consultation. Results from an earlier study on the impact of individual consultation on teacher ratings are reevaluated using this larger sample of faculty. Generally, results in this analysis parallel the original research. Consultation produced changes in student ratings both immediately after consultation as well as longitudinally, thus confirming the utility of intervention in producing enduring pedagogical improvements. Control analyses ensured that improvements were a result of the interventions and not an artefact of time.     

Effect of survey size on student ratings of teaching

Questionnaires filled in by students have become increasingly important due to the part they play in decisions about the tenure, promotion and merit pay of faculty. Since it is rare to get a complete return, it is important to determine what reliance to place on various completion rates. Using data from the faculty of engineering in one university the authors conclude that if class size is 30 or more, a 50% response rate gives an acceptable indication of rating. In a small class, less than 30, something like an 80% return is needed if the same degree of confidence is to be achieved.     

The impact of student feedback on teaching in higher education

Receiving feedback from students has become a normal part of life for university teachers worldwide. This puts pressure on them from several sides and may be an influential factor that leads them to tailor their teaching to students’ preferences. The aim of this study was to investigate teachers’ perceptions of student feedback and how it affects their teaching choices. A survey was sent out to all teachers at the School of Business, Economics and Law at the University of Gothenburg, Sweden. The study found that student feedback is perceived positively by university teachers, has a large impact on their teaching and helps improve courses. Student feedback pushes teaching in the direction of fewer lectures and more tutorials, seminars and case studies. Teachers receiving negative student feedback experience more negative feelings related to the feedback, and are also more likely to introduce unjustified changes to their teaching in order to please students. These teachers also tend to have less teaching experience. However, a very large majority of teachers have a high level of professional pride and integrity and do not make (as they perceive them) unjustified changes to their teaching.     

Kinds of Keys of student ratings of faculty teaching effectiveness

Three substudies of effects of different formats on student ratings of faculty teaching effectiveness were conducted. One substudy, Kinds of Keys, investigated three main kinds of keys: Agreement, Evaluation, and Needs Improvement. The second, NO TUP, investigated numbers of positive rating categories. The third, Item Wording Direction, investigated the same items worded positively, negatively, and neutrally, respectively. Practically important differences in level of ratings obtained in Kinds of Keys and practically and statistically significant differences obtained in NO TUP and Item Wording Direction. It was concluded that additional research is necessary to determine if apparent differences in teaching effectiveness are actually differences in teaching effectiveness or differences due to the methods of measurement.     

Reconceptualizing student ratings of teaching to support quality discourse on student learning: a systems perspective

Higher Education - This paper summarizes the discourse on student ratings of teaching in higher education. It reconceptualizes student ratings within a larger process of promoting quality in...     

How should we use student ratings to evaluate teaching?

The proper use of student ratings to evaluate teaching is the subject matter of this paper. In particular, two divergent views are contrasted: the multidimensional view of evaluation and the unidimensional. Marsh (Int. J. Ed. Res., 1987, v.11, pp. 253–388) is the most outspoken advocate of the multidimensional view of evaluation based on factor scores. Because we disagree over such an important use of ratings, I explain the position I have adopted, summarize Marsh's most recent position, and respond to his concerns.An earlier version of this paper was presented at the annual meeting of the American Educational Research Association, New Orleans, LA, April, 1988.     

The role of scales in student ratings

Questions used in course evaluations should mainly measure the quality of teaching, and students' answers to those questions should not be influenced by other factors. This paper investigates how seemingly neutral rating scales and multiple-choice questions might have an impact on the results of such evaluations. In several studies, it has been shown that the way in which a scale is constructed may strongly influence the answers it elicits in surveys and tests. Whether and to what extent common course-related ratings of students are also affected by the kinds of scales used is the main topic of this paper. Four studies examined the influence of scale polarity (unipolar vs. bipolar scales), the role of different ranges in time scales, and the impact of ordering choices in a certain way. Participants' ratings and answers were strongly influenced by all these manipulations. It is recommended to pay special attention to the role of scales when constructing questionnaires for course evaluations and when interpreting course evaluation reports.     

Using student ratings to measure quality of teaching in six European countries

This paper argues for the value of using student ratings to measure quality of teaching. An international study to test the validity of the dynamic model of educational effectiveness was conducted. At classroom level, the model consists of eight factors relating to teacher behaviour: orientation, structuring, questioning, teaching modelling, application, management of time, teacher role in making classroom a learning environment and assessment. In each participating country (i.e. Belgium/Flanders, Cyprus, Germany, Greece, Ireland and Slovenia), a sample of at least 50 primary schools was used and all grade 4 students (n?=?9967) were asked to complete a questionnaire concerning the eight factors of the dynamic model. Structural equation modelling techniques were used to test the construct validity of the questionnaire. Both across- and within-country analyses revealed that student ratings are reliable and valid for measuring the functioning of the teacher factors of the dynamic model. Implications for teacher education are drawn.     

The quality of student ratings of teacher behaviour

In a research project into the effectiveness of mathematics teaching in the first year of secondary education, external observers and students rated teachers' behaviour. The reliability and validity of both methods were established. The results show that teacher behaviour is assessed well when student ratings are aggregated at the classroom level. The quality of aggregated student ratings is as good as the quality of data from external observers. The predictive validity of aggregated student ratings is higher than the predictive validity of external observations when subject motivation is taken as a dependent variable.     

Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, Cohen, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.     

Effects of graduate teaching assistant attire on student learning,misbehaviors, and ratings of instruction

This study examined the influence of teaching assistant attire in the university classroom. Significant relationships were found between instructor attire and student cognitive learning, affective learning, and ratings of instruction. Analysis of variance indicated significant differences in these variables by levels of teaching assistant dress. Significant negative relationship was found between instructor attire and student likelihood of misbehavior. Analysis of variance revealed that student misbehaviors were less likely for teaching assistants with high professional dress.     

Interfaculty differences in classroom teaching behaviors and their relationship to student instructional ratings

This study tested for differences between instructors in three college faculties in the frequency with which they exhibited various classroom teaching behaviors, and in the correlation of these behaviors with perceived teaching effectiveness. Trained observers assessed the frequency of 95 classroom teaching behaviors shown by 124 teachers in the arts and humanities, the social science, and the natural science faculties. Teaching effectiveness was measured by formal end-of-term student ratings averaged over a three-year period. Behaviors reflecting interpersonal orientation occurred more frequently in arts and social science teachers than in natural science teachers, whereas behaviors reflecting task orientation occurred more frequently in natural and social science teachers than in arts teachers. However, the pattern of correlations between teaching behaviors and overall effectiveness ratings was quite similar in all three faculties. Results are discussed in terms of understanding, evaluating, and improving college teaching.     

Predicting student ratings: the relationship between actual student ratings and instructors’ predictions

This study examined the relationship between student ratings and instructors’ predictions of these ratings, taking into account other instructor, student, and course characteristics. Participants in the study were 198 instructors in the School of Education at a major teacher training college in Israel. Data corresponding to one randomly selected course per instructor were collected using student and instructor questionnaires and college records. Results indicate a systematic positive relationship between instructors’ predictions and actual student ratings with respect to overall ratings and the ratings of three dimensions of teaching. Results also demonstrate a systematic trend whereby low‐rated instructors tend to overestimate their student ratings, high‐rated instructors underestimated ratings, and moderately rated instructors gave accurate predictions. Results have implications for using predictions to motivate teaching improvement.     

The effects of organizational factors on student ratings and perceived instruction

The objective of this study is to explore a) which dimension of student ratings and which aspects of perceived instruction are affected by the two organizational factors, enrollment size and academic affiliation, and b) the nature of their effect. Two thousand five hundred students participating in 125 courses evaluated their instructors on Q-1 Evaluation of Instruction by Students. The 125 evaluated instructors responded to Q-2 Perception of own Instruction indicating the extent to which they employ various teaching planning and strategy attributes (TPS). Major findings suggest that academic affiliation has no effect on student ratings yet affects abstract aspects of perceived instruction. Instructors of the social sciences, unlike those of the humanities, manifest a vocational outlook in their instruction planning. Data indicate that enrollment size has an effect on the dimensions of student ratings and the perceived instruction, referring to concrete aspects of teaching. Students participating in small classes are more critical of instruction than those in larger classes. The nature of its effect on perceived instruction is mainly in terms of practical solutions of teaching methods and strategies and is hardly manifested in the planning phase of instruction. The relevance of these findings to university administrators is also discussed.     

A study of the validity of four types of student ratings of college teaching assessed on a criterion of student achievement gains

The purpose of the study was to investigate relationships between student ratings of college teaching using four types of student rating instruments and pre- vs. post-student achievement gains in 36 sections of an undergraduate analytic geometry and calculus course. Student rating instruments used varied according to type of items (high vs. low inference) and focus (students rating their own perceived growth vs. rating the instructor). Data were collected on 799 students (66% freshmen; 16% sophomore; and 15% juniors) at the University of Florida, and relationships were analyzed using the Pearson product-moment correlation technique. Significant relationships were not found between student ratings and student achievement.     

A description of effective college teaching in five disciplines as measured by student ratings

Student rating of teachers in five disciplines (science and math, education, social sciences, humanities, and business) were analyzed to determine which teacher attributes were important in predicting ratings of teaching effectiveness. Rating results from 1,439 courses taught at Southern Illinois University, Carbondale from 1973 to 1974 were used as data for this study. The results indicated that the instructor attributes rated as characteristic of effective instruction were highly consistent across disciplines, and the effective instructor was described as (1) knowing when students understood him, (2) increasing students' appreciation of the subject matter, (3) answering impromptu questions satisfactorily, (4) achieving the objectives of the course, and (5) giving several examples to explain complex topics.     

Exploring the impact of faculty reflection on weekly student evaluations of teaching

This exploratory study considered Larrivee’s assessment of teachers’ reflective practice levels by using a formative, weekly, online student evaluation of teaching (SET) tool through a virtual learning environment (VLE) as a means to encourage reflective practice. In‐depth interviews were conducted with six faculty members in three departments at a university college in the UK. The study found that: (a) faculty who experienced surface‐level reflection were more likely to have a reactive reflection style; and (b) faculty who experienced higher levels of reflection were more likely to have a proactive reflection style. Overall, the tool was found to be an efficient means of encouraging reflection by all participants and demonstrated that reflective practice could come about as a result of these weekly formative SETs. The study concludes with suggestions for academic development and future research on reflection that could be conducted using SETs via a VLE.     

The localization,diagnostic, and monitoring functions of student ratings in a model for improving university teaching

A diversity of student questionnaires are used by colleges and universities to provide data on faculty teaching performance. Yet the purposes for collecting this data are frequently unclear, and at times superficial. Rarely is student rating data used as a tool to improve faculty teaching. A more relevant approach incorporates a variety of types of student ratings into a model for improving university teaching. One type of student rating data is used to identify broad instructional problem areas. Another type pinpoints probable causes and solutions for the instructional problems. Instructional improvement procedures are designed on the basis of this data. A third type of student rating data evaluates the instructional improvement procedures and indicates when modifications are needed. In addition to these three types of student ratings, and the generation of appropriate questionnaires, this paper presents an overview of the teaching improvement model and discusses its effectiveness.     

Student ratings of teaching quality in primary school: Dimensions and prediction of student outcomes

The contribution examines theoretical foundations, factorial structure, and predictive power of student ratings of teaching quality. Three basic dimensions of teaching quality have previously been described: classroom management, cognitive activation, and supportive climate. However, student ratings, especially those provided by primary school students, have been criticised for being biased by factors such as teacher popularity. The present study examines ratings of teaching quality and science learning among third graders. Results of multilevel confirmatory factor analyses (N = 1556 students, 89 classes) indicate that the three-dimensional model of teaching quality can be replicated in ratings of third graders. In a longitudinal study (N = 1070 students, 54 classes), we found ratings of classroom management to predict student achievement, and ratings of cognitive activation and supportive climate to predict students' development of subject-related interest after teacher popularity is controlled for. The analyses show that student ratings can be useful measures of teaching quality in primary school.