Research productivity and scholarly accomplishment of college teachers as related to their instructional effectiveness: A review and exploration

This analysis reviews the research that has been done on the connection between research productivity or scholarly accomplishment of faculty members and their teaching effectiveness (as assessed by their students). On average, there is a very small positive association between the two variables. To understand this relationship better, extant research was explored for factors that might mediate either positive or inverse associations between research productivity and teaching effectiveness and those that possibly could be common causes of them. Pedagogical practices and dispositions of faculty members, as well as certain course or class characteristics (size of class, electivity of course), were examined as potential mediating factors. Potential common causes investigated were academic rank and age of faculty members, their general ability, their personality characteristics, and the amount of time or effort they spend on research activities. The association between research productivity and teaching effectiveness was explored further by considering whether its size and direction varies by career stage of faculty members, their academic discipline, and the type of college or university in which they teach.     

Student evaluations of teaching: the impact of faculty procedures on response rates

In the last 10–15 years, many institutions of higher education have switched from paper-and-pencil methods to online methods of administering student evaluations of teaching (SETs). One consequence has been a significant reduction in the response rates to such instruments. The current study was conducted to identify whether offering in-class time to students to complete online SETs would increase response rates. A quasi-experiment (nonequivalent group design) was conducted in which one group of tenured faculty instructed students to bring electronic devices with internet capabilities on a specified day and offered in-class time to students to complete online SETs. A communication protocol for faculty members’ use was developed and implemented. A comparison group of tenured faculty who did not offer in-class time for SET completion was identified and the difference-in-differences method was used to compare the previous year’s response rates for the same instructor teaching the same course across the two groups. Response rates were substantially higher when faculty provided in-class time to students to complete SETs. These results indicate that high response rates can be obtained for online SETs submitted by students in face-to-face classes if faculty communicate the importance of SETs in both their words and actions.     

The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates

Course evaluations (often termed student evaluations of teaching or SETs) are pervasive in higher education. As SETs increasingly shift from pencil-and-paper to online, concerns grow over the lower response rates that typically accompany online SETs. This study of online SET response rates examined data from 678 faculty respondents and student response rates from an entire semester. The analysis focused on those tactics that faculty employ to raise response rates for their courses, and explored instructor and course characteristics as contributing factors. A comprehensive regression model was evaluated to determine the most effective tactics and characteristics. Using incentives had the most impact on response rates. Other effective tactics that increase response rates include reminding students to take the evaluation, explaining how the evaluations would be used to improve instruction, sending personal emails and posting reminders on Blackboard^®. Incentives are not widely used; however, findings suggest that non-point incentives work as well as point-based ones, as do simple-to-administer minimum class-wide response rate expectations (compared to individual completion).     

Research productivity and teaching effectiveness

This study investigated the relationship between research productivity and teaching effectiveness to shed light on the long-debated question of whether performance in one area enhances performance in the other. The academic field and the stage of a faculty member's career were both considered in the analyses. Two samples — one of 2,973 and the other of 1,623 faculty members from a variety of institutions — were studied. In considering results of both analyses, teachers of social science courses were the only group for which there were consistent though modest relationships between the number of published articles and student ratings of instructor effectiveness. Thus spillover effects, or a general ability factor, or other reasons for a possible link between research and teaching performance are not totally supported. The relationship between performance in the two areas is either nonexistent or, where it appears, too modest to conclude that one necessarily enhances the other.     

Part-time faculty status and student evaluation of teaching: An analysis of criminal justice classes

Although part-time (p/t) faculties constitute a growing proportion of college instructors, there is little work on their level of teaching effectiveness relative to full-time (f/t) faculty. Previous work on a key indicator of perceived teaching effectiveness, student evaluation of teaching (SET), and faculty status (p/t/ vs f/t) is marked by a series of shortcomings including lack of a systematic theoretical framework and lack of multivariate statistical analysis techniques to check for possible spuriousness. The present study corrects for these shortcomings. Data consist of SETs from 175 sections of criminal justice classes taught at a Midwestern urban university. Controls are introduced for variables drawn from the literature and include ascribed characteristics of the professor, grade distribution, and structural features of the course (e.g., level, size). The results of a multivariate regression analysis indicate that even after controlling for the other predictors of SETs, p/t faculty receive significantly higher student evaluation scores than f/t faculty. Further, faculty status was the most important predictor of SETs. The results present the first systematic evidence on faculty status and SETs.     

Under pressure: How faculty gender and contract uncertainty impact students’ grades

Although rising average grades appear to be common at post-secondary institutions in the U.S., there is still little work examining mechanisms driving this increase in grades. This paper uses data from a public research university to examine one mechanism in particular: instructor level incentives that are linked to gender and contract status. We hypothesize that instructors with more job uncertainty due to their rank will be most incentivized to award higher grades, as this may lead to better evaluations of teaching and an increase in retention probability. Our results indicate that students receive higher grades when their class is taught by a female instructor with more job uncertainty than if the class were taught by a tenured female faculty member. These higher grades appear to reflect more lenient grading rather than better preparation for follow-on courses. However, for students taking classes with male instructors, there is no significant difference across instructor rank in grades received. Our results have important implications for thinking about the role faculty contracts may play in affecting grading distributions.     

Class Level as a Possible Mediator of the Relationship between Grades and Student Ratings of Teaching

Although the research literature investigating the relationship between grade awarded to students and students’ evaluations of teaching performance is voluminous, very few studies have examined the grade‐rating relationship according to level of student. The present study examined correlations between mean instructor rating and mean class grade for all course evaluations (N = 625 classes) at Utah State University during an academic quarter. In lower‐division (courses 100–299) and upper‐division (courses 300–599) undergraduate classes, correlations between grades and ratings of faculty were of an expected direction and magnitude (0.29 and 0.28, respectively); however, the grade‐rating correlation for graduate classes (courses 600 +) was — 0.20. It is speculated that graduate students are both better students and more critical evaluators of instruction, but replication and extension with different samples are needed before this tentative explanation can be accepted with confidence.     

A longitudinal investigation of the impact of faculty reflective practices on students' evaluations of teaching

Though there have been many studies conducted that emphasise faculty reflection as a crucial feature of professional practice, there appears to have been little empirical evidence to support the proposition that reflective practice improves the quality of teaching. Previous research demonstrated that reflective practice could be encouraged by weekly formative student evaluations of teaching (SETs). This study investigated the impact of reported reflective practice using formative SETs on changes to summative SETs, typically conducted at the end of a teaching period. Data was collected in a rural UK‐based university‐college in 11 modules (n = six faculty members, n = 413 students) in Business, Countryside and Environment, Foundation Degree and Veterinary Nursing programmes over the period of 2 years of data collection. Findings show that on average, SET scores increased for all reflective practitioners year on year and increased more for those faculty members who demonstrated higher levels of reflection.     

Expectancy theory outcomes and student evaluations of teaching

As student evaluation of teaching (SET) instruments are increasingly administered online, research has found that the response rates have dropped significantly. Validity concerns have necessitated research that explores student motivation for completing SETs. This study uses Vroom's [(1964). Work and motivation (3rd ed.). New York, NY: John Wiley & Sons] expectancy theory to frame student focus group responses regarding their motivations for completing and not completing paper and online SETs. Results show that students consider the following outcomes when deciding whether to complete SETs: (a) course improvement, (b) appropriate instructor tenure and promotion, (c) accurate instructor ratings are available to students, (d) spending reasonable amount of time on SETs, (e) retaining anonymity, (f) avoiding social scrutiny, (g) earning points and releasing grades, and (h) being a good university citizen. Results show that the lower online response rate is largely due to students’ differing feelings of obligation in the 2 formats. Students also noted that in certain situations, students often answer SETs insincerely.     

An investigation of research self-efficacy beliefs and research productivity among faculty members at an emerging research university in the USA

The purposes of this study are to add to the existing knowledge base on research self-efficacy beliefs of faculty members and their influence on research productivity, and to inform higher education administrators about the relationship between research self-efficacy beliefs and research productivity. A theoretical framework of social cognitive theory underpins the study in which 109 faculty members completed the Research Self-Efficacy Inventory and Research Productivity Index. Quantitative analysis indicated that there is a significant but small correlation between research self-efficacy and productivity.     

A Critical Look at Ourselves: Do Male and Female Professors Respond the Same to Environment Characteristics?

We examined the relationship between gender and both job satisfaction and research productivity using data from 1,135 psychology faculty working in 229 academic departments. We found that gender differences in job satisfaction and research productivity were related to elements of the department (i.e., teaching orientation and structure). Overall, women reported lower levels of productivity than their male counterparts. Women also reported higher levels of job satisfaction in more teaching-oriented departments whereas men reported higher levels job satisfaction in more research-oriented departments. We suggest that these findings might be the result of gender differences in preferences with women preferring more socially-oriented positions and men prefer more “things/data”-oriented positions.     

Honesty on student evaluations of teaching: effectiveness,purpose, and timing matter!

Student evaluations of teaching (SETs) are an important point of assessment for faculty in curriculum development, tenure and promotion decisions, and merit raises. Faculty members utilise SETs to gain feedback on their classes and, hopefully, improve them. The question of the validity of student responses on SETs is a continuing debate in higher education. The current study uses data from two universities (n = 596) to determine whether and under what conditions students are honest on in-class and online SETs, while also assessing their knowledge and attitudes about SETs. Findings reveal that, while students report a high level of honesty on SETs, they are more likely to be honest when they believe that evaluations effectively measure the quality of the course, the results improve teaching and benefit students rather than the administration, and when they are given at the end of the term. Honesty on evaluations is not associated with socio-demographic characteristics.     

Management by results: Student evaluation of faculty teaching and the mis-measurement of performance

Using data on 4 years of courses at American University, regression results show that actual grades have a significant, positive effect on student evaluations of teaching (SETs), controlling for expected grade and fixed effects for both faculty and courses, and for possible endogeneity. Implications are that the SET is a faulty measure of teaching quality and grades a faulty signal of future job performance. Students, faculty, and provost appear to be engaged in an individually rational but socially destructive game of grade inflation centered on the link between SETs and grades. When performance is hard to measure, pay-for-performance, embodied by the link between SETs and faculty pay, may have unintended adverse consequences.     

Research productivity and perceived teaching effectiveness: A survey of economics faculty

The relationship between research productivity and teaching effectiveness has long been a controversial topic among university professors. This study utilizes a nationwide survey of economics faculty to examine the relationship between research productivity and classroom performance, using self-reported measures of research output and teaching evaluation scores, and to solicit the participants' opinions on this issue. Findings indicate a very weak relationship between research output and teaching effectiveness. However, individual and institutional characteristics seem to explain some differences in research output and teaching evaluation scores. Further, faculty opinions on the research-teaching relationship seem to be influenced by institutional and individual characteristics.     

How is language used to craft social presence in Facebook? A case study of an undergraduate writing course

This quantitative content analysis examines the way social presence was created through original posts and comments in a Facebook group for an undergraduate writing course. The author adapted a well-known coding template and examined how course members—one instructor, two undergraduate teaching assistants and twenty-two students--used language and image to generate social presence. Language is categorized by purpose, and each category is a social presence indicator. The results show which social presence indicators are used more frequently by course members and that social presence is higher when certain social presence indicators are utilized. Since existing research indicates that social presence is associated with increased course satisfaction and learning, these findings provide practical knowledge for faculty seeking to increase social presence in their classes. This is a case study with a small population, but these results indicate the need for additional research to further explore how language and image are used to generate social presence in Facebook.     

Scholarly Productivity and Impact of School Psychology Faculty in APA‐Accredited Programs

The primary objective of this study was to conduct a normative assessment of the research productivity and scholarly impact of tenured and tenure‐track faculty in school psychology programs accredited by the American Psychological Association (APA). Using the PsycINFO database, productivity and impact were examined for the field as a whole and by faculty rank and gender between 2005 and 2009. Results of our study reflected considerable variability in scholarly impact and productivity. For example, on average, school psychology faculty published slightly more than one refereed journal article per year, with productivity rates ranging from zero to eight articles per year. Similar variability in results was observed for scholarly impact. Results of this study also revealed no significant differences in productivity and impact by scholarly rank. Significant differences were observed for gender, however, with higher productivity and impact for men than women. A secondary objective of this study was to rank the most productive and impactful faculty by total authorship credit, number of publications, and number of citations, and to examine the relationships among these different rankings. Implications and limitations are discussed.     

Rate my attitude: research agendas and RateMyProfessor scores

The literature on student evaluations of teaching (SETs) generally presents two opposing camps: those who believe in the validity and usefulness of SETs, and those who do not. Some researchers have suggested that ‘SET deniers’ resist SETs because of their own poor SET results. To test this hypothesis, I analysed essays by 230 SET researchers (170 lead authors) and classified the researchers as having negative, neutral or positive attitudes towards SETs. I retrieved their RateMyProfessors.com (RMP) scores and, using logistic regression, found that lead authors with negative attitudes towards SETs were 14 times more likely to score below an estimated RMP average than lead authors with positive attitudes towards SETs. Co-authors and researchers with neutral attitudes, on the other hand, did not significantly differ from the RMP average. These results suggest that personal attitudes towards SETs may drive research findings.     

REFLECTIONS OF COMMUNITY-COLLEGE STUDENTS REGARDING MENTEE/INSTRUCTOR TEACHING AND LEARNING EFFECTIVENESS

This investigation used data collected from students who assessed their instructors' teaching and learning effectiveness. Instructors were community-college career and technical-education faculty enrolled in the Community College Induction Mentoring Program (CCIM), a jointly sponsored program between Iowa State University and the instructors' respective community colleges. These new faculty were involved in a structured mentor/mentee program in which the mentor was involved in a 1-day mentor training program. The mentor/mentee relationship was designed around annual goals with an action plan developed, executed, and assessed for each goal. One component of the mentor/mentee program involved documentation of an effectiveness plan, including students' perception of their mentored-instructors' teaching and, as a consequence, their own learning effectiveness. Students were asked to complete an evaluation instrument comparing their “reactions” and “learning” in classes taught by mentored instructors enrolled in the CCIM program relative to other nonmentored instructors. A total of 9 hypotheses provided the direction of the research. Student ratings were typically higher for new instructors who received peer mentoring. Mentees and mentors consistently reported a high level of satisfaction about their partnership. Supervisors voiced strong support for the program. This paper shares student survey results of mentee/instructor teaching and student learning effectiveness.