Research productivity and teaching effectiveness

This study investigated the relationship between research productivity and teaching effectiveness to shed light on the long-debated question of whether performance in one area enhances performance in the other. The academic field and the stage of a faculty member's career were both considered in the analyses. Two samples — one of 2,973 and the other of 1,623 faculty members from a variety of institutions — were studied. In considering results of both analyses, teachers of social science courses were the only group for which there were consistent though modest relationships between the number of published articles and student ratings of instructor effectiveness. Thus spillover effects, or a general ability factor, or other reasons for a possible link between research and teaching performance are not totally supported. The relationship between performance in the two areas is either nonexistent or, where it appears, too modest to conclude that one necessarily enhances the other.     

Student Perspectives on Teaching and its Evaluation

The research on student ratings of instruction, while voluminous, has had minimal focus on the perceptions of the students who do the ratings. The current study explored student perspectives on course and teacher ratings as well as some issues related to teaching effectiveness and faculty roles. It was found that students are generally willing to do evaluations and to provide feedback, and have no particular fear of repercussions. However, they have little confidence that faculty or administrators pay attention to the results, and do not even consult the ratings themselves. The students view teaching and advising as the most important roles that should be played by faculty, yet project that faculty, while also viewing teaching as the most important, would rank research above the more student-interactive advising. Canonical correlations among various scales reveal a strong emphasis on such issues of the importance of faculty respect for student views.     

The localization,diagnostic, and monitoring functions of student ratings in a model for improving university teaching

A diversity of student questionnaires are used by colleges and universities to provide data on faculty teaching performance. Yet the purposes for collecting this data are frequently unclear, and at times superficial. Rarely is student rating data used as a tool to improve faculty teaching. A more relevant approach incorporates a variety of types of student ratings into a model for improving university teaching. One type of student rating data is used to identify broad instructional problem areas. Another type pinpoints probable causes and solutions for the instructional problems. Instructional improvement procedures are designed on the basis of this data. A third type of student rating data evaluates the instructional improvement procedures and indicates when modifications are needed. In addition to these three types of student ratings, and the generation of appropriate questionnaires, this paper presents an overview of the teaching improvement model and discusses its effectiveness.     

Does the relationship between student ratings of course easiness and course quality vary across schools? The role of school academic rankings

Student ratings of teaching effectiveness are widely used to make judgments of faculty teaching performance. Research, however, has found that such ratings may not be accurate indicators of teaching performance because they are contaminated by course easiness. Using student ratings of 9855 professors employed at 79 different colleges and universities, the author hypothesized and found that the relationship between perceived course easiness and perceived course quality was moderated by school academic rankings. More specifically, easiness ratings were more strongly correlated with quality ratings among low‐ranked schools than among high‐ranked schools. Furthermore, the easiness–quality relationship was slightly stronger among public schools than among private schools. The article concludes by discussing the practical implications of these findings.     

Temporal dynamics between faculty goals,burnout/engagement,and performance in teaching and research: A latent change score approach

Motivation plays a central role in faculty members’ professional lives—with achievement goals having been found to have important links with their burnout/engagement and performance. However, the few studies investigating these links were cross-sectional and considered only one of the two equally important work domains of faculty members. In the present research, we analyze the temporal relationships between achievement goals and burnout/engagement as well as performance and investigate domain specificity of goal pursuit by considering goals for teaching and for research. We conducted a longitudinal study (4 measurement points across two years) including 681 German faculty members. Multivariate Latent Change Score modeling attested that in both domains, mastery-approach goals were positively related to subsequent development of performance, while performance was also positively related to subsequent development of mastery goals, creating a double positive loop. Performance goals and work-avoidance goals were differentially associated with performance in both domains, indicating that the effects of goals can be bound to different contextual features. For overall burnout/engagement, our results implied that primarily research goals mattered for its development (with performance-avoidance and work-avoidance goals being risk factors), while high burnout levels were associated with subsequent reduction of adaptive mastery-approach goals in both domains. This highlights the relevance of achievement goals for burnout/engagement and performance of faculty and illuminates their complex temporal dynamics that can also meaningfully inform achievement goal research in other contexts.     

Faculty ratings as a measure of departmental and administrator quality

This study examined the usefulness of faculty ratings as a measure of departmental quality. Two hundred and thirty-three faculty members from six departments responded to two questionnaires relating to the department—the Department Evaluation Survey (DES)—and to the department head—the Administrator Evaluation Survey (AES). A factor analysis of the DES revealed three underlying dimensions: Satisfaction with the Department's Academic Environment, Satisfaction with the Department's Governance and Operating Procedures, and Satisfaction with Student Quality. The factor analysis of the AES identified only one underlying dimension. Both the DES and AES were found to be highly reliable and effective in discriminating among departments. Very high positive correlations were also found between selacted DES and AES items suggesting that the performance of the head plays a very important role in faculty perceptions of the departments, and vice versa. Faculty ratings would appear to be a useful initial criterion in evaluating a department or head.     

Identifying faculty types using peer ratings of teaching,research, and service

To further explain the relationship between teaching, research, and service, ninety full-time, tenure-track faculty were classified into types based on peer ratings in each of these three areas. The classification was determined using one method of cluster analysis and was validated in two ways: (1) by comparing initial results to those from a second method of cluster analysis and (2) through comparisons of faculty types on additional measures, for example, number of research publications, research grant dollars, and teaching awards. Results supported five discrete, faculty types: All Stars, Teachers and Good Citizens, Researchers, Teachers, and Uninvolved. This current mixture of faculty types is a result of past administrative decisions and will likely shape future decisions in the school.     

The measurement of administrative effectiveness of the academic department head

A total of 103 academic department heads in four universities rated a set of 15 administrative activities as to their importance. Faculty members in these departments (totalN=1,333) used the same set of activities to rate both the importance they should be given by the department head and the effectiveness with which the head performed each set during the previous 12 months. Tests of reliability revealed that faculty ratings of both importance and performance were made with reasonable internal consistency. Three tests of construct validity showed that each of the three types of ratings were made with at least minimal validity. A principal components analysis of faculty ratings of performance suggested that the department head has three major types of responsibility: personnel management; departmental planning and development; and building the department's reputation.     

The impact of individual consultation on the teaching of younger versus older faculty

This study examines the effect of age of instructor on student ratings of teaching performance after individual consultation. Instructors self-presented over an 11-year period at a large Canadian university teaching service, where each received one of three interventions. End of term student ratings of the teaching of younger and older instructors are compared before consultation, immediately post consultation, and 1–3 years after year of consultation. Younger faculty obtained significantly improved ratings immediately after consultation, while older faculty achieved significant rating increases 1–3 years post consultation. Results from an earlier study on the impact of individual consultation on teacher ratings are reevaluated using this larger sample of faculty. Generally, results in this analysis parallel the original research. Consultation produced changes in student ratings both immediately after consultation as well as longitudinally, thus confirming the utility of intervention in producing enduring pedagogical improvements. Control analyses ensured that improvements were a result of the interventions and not an artefact of time.     

Faculty ratings of course evaluation items

Instructors whose teaching was evaluated by students were given the opportunity to rate how applicable the evaluation items were to their classes. This study examined the kinds of items which instructors felt to be applicable or inapplicable, the relationships between the student ratings and the instructor applicability ratings, and the effect on an overall evaluation score of using the instructor applicability judgments as weights.Results generally support the consensus procedure of establishing rating forms; they suggest that the common criticism that faculty judgments of item applicability are influenced by anticipation of student ratings may be true for specific items and that while weighting composite evaluation scores by means of faculty applicability judgments does not affect those overall scores, the distributions of certain items may be altered.     

Concurrent and predictive validity of the WISC-R,McCarthy Scales,Woodcock-Johnson,and academic achievement

Twenty-six normal first-grade children (M = 7.0 years) were administered the WISC-R, the McCarthy Scales, and the Woodcock-Johnson Scales of Cognitive Ability. Two years later, their levels of academic achievement were determined by way of the WRAT and global teacher ratings of classroom performance. Pearson intercorrelations among the summary indices of the three intellectual measures were significant and uniformly high (rang. 77-.95). Correlations between first-grade ability scores and third-grade achievement measures also were significant (range .64-.90), suggesting strong predictive validity for each of the cognitive instruments. Though not significantly so, the correlations between the Woodcock-Johnson and achievement tended to be the highest. The results indicate that all three intellectual measures are appropriate for predicting later academic achievement in young school-aged children.     

An Evaluation of Factors Regarding Students’ Assessment of Faculty in a Business School*

Student faculty ratings are used at most institutions of higher learning for three important reasons. First, the ratings provide direct feedback to the faculty, and this enables faculty to adjust their teaching styles. Second, the ratings provide the administration with information intended to assist in guiding and mentoring faculty toward more effective pedagogical performance in the classroom. Third, the ratings also provide the administration with information to be used in the reappointment, tenure, and promotion processes, as well as for assignment of salary range adjustments and teaching awards. To be of real value, however, all of this is predicated on the use of a valid and reliable faculty‐rating instrument along with a system designed to provide both the faculty and the administration with norming reports that allow for appropriate comparisons of ratings. This article reports such a study conducted within a large department of a business school and recommends that the process used be adapted by other business school departments and other academic units across the university and at other universities to ensure a more universally appropriate usage of students’ ratings.     

Community College Faculty Motivation for Basic Research,Teaching Research,and Professional Development

Community college faculty members often find themselves divided between what they want to do and what they can do. Knowing what motivates faculty to engage in professional development and scholarly productive activities provides critical information for administrators. The present study explored the motivational characteristics of community college faculty and the contextual factors that support or thwart their engagement and productivity. It focused on three key professional activities: (a) basic or applied research, (b) classroom action/teaching research, and (c) faculty professional development. Findings indicate that community college faculty are motivated for all three activities primarily by intrinsic and value-related factors rather than by extrinsic or contextual factors. Further, faculty members present somewhat different motivational profiles for the three work activities. Their motivations also demonstrate responsiveness to workplace characteristics as faculty perceive and interpret them.

These findings provide implications to help college administrators make policy decisions to support the work of faculty and align with institutional mission and goals.     

Improving instruction of veteran physics professors

In spite of the abundance of publications describing university faculty development programs and activities, little is known about the effectiveness of such programs on specific disciplines or subject areas. The fact that differences have been identified in the dimensions on which students of different university departments rate their teachers suggests that instructors of different departments need different types of programs for teaching improvement. This article describes a study that has looked into methods for improving instruction of university physics full professors with many years of teaching experience. Two methods for this aim were examined for effectiveness: a workshop and individualized consultation, both augmenting students' midterm ratings of their instructors. Analysis of pre- and postworkshop questionnaires reveals impressive improvement on the majority of items, particularly those of specific teaching techniques discussed in the workshop, but not on the global ratings of the teacher. The special consultation procedure has been shown to bring about substantial increase on overall teaching performance. We conclude that veteran teachers are often unable to improve significantly their overall teaching performance when provided with midterm feedback from students' questionnaires or when participating in a workshop for teaching improvement. Improving their instruction requires substantial and continuous expert consultation as well as investing substantial time and efforts of their own.     

Professional socialization and contemporary career attitudes of three faculty generations

As one aspect of a study of individual and organizational conditions contributing to faculty vitality, career socialization experiences and current career attitudes of three faculty generations were compared for two groups of tenured university faculty members. The first group was a representative sample drawn from the fields of the humanities, biological sciences, physical sciences, and social sciences. The second was a selected sample of faculty from the same areas who had been identified by judges as highly active in teaching, research, and service. The analyses focus on differences in the professional socialization experiences and career attitudes of the two groups that appear to be indicative of career success. Within these groups the generational trends are also examined.Presented at the 23rd Annual Forum of the Association for Institutional Research, Toronto, May 1983.     

An assessment of the academic and social integration as perceived by the students in the University of Papua New Guinea

This study assesses the current and ideal levelsof academic and social integration within the University of PapuaNew Guinea as perceived by the students. In order to reducestudent attrition through developing an appropriate attritionpolicy and improving services, a comprehensive picture of thecurrent and ideal academic and social integration climate in theUniversity of Papua New Guinea was necessary. Previousattrition/retention research has not adequately described thevarying degrees of group differences in students' integrationprocess. Using the Tinto model and research instruments thatoperationalised the model, an institutional academic and socialintegration assessment instrument was developed for measuring theperceived current and ideal integration climate. Respondent'sperceived ideal ratings for 24 items representing five dimensionsof academic and social integration were higher than for thecurrent ratings. Significant differences were found between thecurrent and ideal perceptions of student groups in respect of a)students' informal contact with faculty; b) faculty concern forstudents' development and teaching and c) extra curricularactivities. Differences were found between the perceived ratingsof student groups on the basis of year of studies. Implicationsfor policy directives and discriminatory service delivery arealso discussed.     

Students' perceptions of teaching skills and overall effectiveness across instructional settings

This study examines the differences in students' perceptions of the amount of improvement in dimensions of teaching skills across levels of instruction, class sizes, and disciplines. The relationship between factor scores on the dimensions of teaching and students' overall ratings are also analyzed across instructional settings.Student ratings were collected in 2,816 classes in a Faculty of Commerce over three semesters. Using class means as a unit of analysis, the data were factor-analyzed. Factor scores were used as dependent variables in examining differences across course characteristics. Factor scores were further used to predict ratings of the overall quality of instruction. These predictions were subsequently compared across instructional settings.In the combined sample, differences in students' perceptions were found across levels of instruction for all factors, and across class size for two factors. When analyses were conducted on these variables within separate disciplines, the results varied. The relationship between factor scores on the dimensions of teaching and students' overall ratings varied across instructional settings.The results are discussed in relation to previous research on the dimensions of teaching and in relation to the concept of students' normative assumptions about teaching behavior. Some practical implications of the results are also described.     

THE IMPACT OF FORMATIVE AND SUMMATIVE EVALUATION OF TEACHING IN NORTH AMERICAN UNIVERSITIES

This paper reviews recent research evidence on formative (developmental) and summative (judgemental) evaluation of teaching in North American universities. The major conclusions of the review are as follows: (1) student ratings are the most common method of evaluating teaching in North American universities, and their popularity appears to be increasing; (2) student ratings have been found to provide reliable and valid information on limited aspects of teaching competence; (3) teaching evaluations make a difference in decisions on faculty salary, tenure and promotion; (4) evaluation of teaching has been shown to produce modest improvements in teaching effectiveness; (5) faculty satisfaction with teaching evaluation practices ranges from very high to very low, depending on institutional context; and (6) there is consensus that several specific improvements are needed in the current teaching evaluation system.     

On the use of student ratings of faculty in determination of pay,promotion and tenure

Student ratings of faculty have traditionally been obtained in a manner designed to be useful to the individual faculty member or to other students. It is now sometimes proposed that the resulting data be used to determine faculty pay, promotion, and tenure. Recent articles and the reviews of past literature on ratings are analyzed to determine whether student ratings are usually associated with teaching effectiveness, whether they are sometimes biased by irrelevant factors, whether faculty can effectively revise their behaviors to improve their ratings, and whether improved ratings result in improved teaching effectiveness. The results tend to indicate that student ratings of faculty as they are currently collected cannot be trusted for considerations of pay, promotion, and tenure.     

University instructors' ratings profiles: Stability over time,and disciplinary differences

This study investigated university instructors' teaching profiles (rank order of teaching behaviors) as they emerge from student ratings. The data consisted of student ratings of all faculty members in the physics and chemistry departments at Tel Aviv University, with no exception, who taught the same undergraduate course over two consecutive years. The ratings of these instructors were taken twice a semester (at mid- and end of semester), during the two academic years of the study, using a short rating instrument. A (4 × 5) × 2 ANOVA was conducted, with time of measurement and item serving as repeated measures, and department as a grouping factor. The same procedure was used for identifying differences between pairs of measurements for each item. These pairs compared the corresponding measures of the first year versus the second year, and separately those of mid-semester versus the end of semester. Findings indicate a very high degree of stability of instructors' profiles across time of measurement and activities to improve instructor performance on weak points; no clear-cut improvement from mid- to end-of-semester ratings except for teachers who undertake special improvement activities; and differences in teaching profiles for the two academic departments.