Teaching effectiveness as measured by student ratings and instructor self-evaluation

The present study is designed to evaluate teaching effectiveness based on effective teacher behaviors. Operationally, it is measured by assessing levels of agreement between perceptions of instructors and students on reported employment rates and rated ability for specific instructional behavior attributes. Individually-based and organizationally-based factors are incorporated in the study for a possible contribution to teaching effectiveness variance. Findings indicate a lack of agreement between perceptions on abstract instructional attributes but that agreement exists for attributes of a concrete nature. Analysis suggests that individually-based and organizationally-based factors do not contribute to agreement variance.     

Discrepancy between instructor and student evaluations of instruction: Effect on instructor

To find out whether a discrepancy between the instructor's and the student's evaluations of teaching influence teaching, 13 introductory and educational psychology instructors and their students were given a Student Opinion Questionnaire (SOQ) twice: on the fourth week of a fall term and eight weeks after feedback sessions with the instructors. The instructors received feedback on the direction and amount of initial discrepancy. The results showed that the unfavorable discrepant instructors (instructor rating better than students) changed more on skill, feedback, rapport, general teaching ability, and the overall value of the course than the favorably discrepant instructors (student ratings better than instructor). The unfavorably discrepant instructors improved their teaching significantly more than the favorably discrepant instructors.This article is based on portions of a dissertation submitted in partial fulfillment of the requirements for the Ph. D. degree at the University of Michigan. The author wishes to acknowledge the help given by Wilbert J. McKeachie and Alvin F. Zander and thank them for their suggestions. An earlier version of the paper was presented at the 81st Annual Convention of the American Psychological Association, Montreal, Que., August 1973.     

Relationship with instructor and other variables influencing student evaluations of instruction

This study examined some of the factors which influence college students’ evaluations of their instructors. Data were collected from 557 students enrolled in a basic speech communication course. Overall, a student's relationship with the instructor was the best predictor of the student's evaluation of the instructor. This study lends partial support to the idea that students evaluate male and female instructors differently. The variables examined in this study were better predictors of student evaluation of female than male instructors.     

Administrative and structural changes in student evaluations of teaching and their effects on overall instructor scores

Using nine years of student evaluation of teaching (SET) data from a large US research university, we examine whether changes to the SET instrument have a substantial impact on overall instructor scores. Our study exploits four distinct natural experiments that arose when the SET instrument was changed. To maximise power, we compare the same course/instructor before and after each of the four changes occurred. We find that switching from in-class, paper course evaluations to online evaluations generates an average change of ?0.14 points on a five-point scale, or 0.25 standard deviations (SDs) in the overall instructor ratings. Changing labelling of the scale and the wording of the overall instructor question generates another decrease in the average rating: ?0.15 of a point (0.27 SDs). In contrast, extending the evaluation period to include the final examination and offering an incentive (early grade release) for completing the evaluations do not have a statistically significant effect on the overall instructor rating. The cumulative impact of these individual changes is ?0.29 points (0.52 SDs). This large decrease shows that SET scores are not comparable over time when instruments change. Therefore, administrators should measure and account for such changes when using historical benchmarks for evaluative purposes (e.g. appointments and compensation).     

Predicting student self-concept,anxiety, and responsibility from self-evaluation and self-praise

Nineteen elementary teachers were observed for the frequency with which they modeled self-praise for their students and taught their students to praise and evaluate themselves. Measures of student self-concept, anxiety, and achievement responsibility were taken at the end of the year. It was found that teacher modeling of self-praise correlated negatively with boys' selfconcept and positively with girls'. Teacher encouragement of students to praise other students correlated positively with boys' anxiety and negatively with girls'. Teacher modeling of self-praise and teacher encouragement of students to praise other students were the best predictors of self-concept.     

Teacher student interaction: A case study

This study is concerned with possible differences in interaction patterns between teachers and certain groups of students: high and low achievers, and boys and girls. Detailed and comprehensive records of all verbal teacher-student interactions, both public and private, were obtained through videotaping a grade 6 class in three different settings: mathematics, language and science. While there was much consistency in the way the teacher monitored in this study interacted with the different groups, some important differences occurred in the quality and quantity of his interactions with students perceived as best and weakest, and with boys and girls in mathematics classes.     

Parental evaluations of psychoeducational reports: A case study

The purpose of this case study report was to determine what parents want and the degree to which they value psychoeducational reports prepared by school psychologists. The sample (N = 44) consisted of parents whose children were seen as out-patients of the Learning Disorder Clinic within the Department of Pediatrics at the UCLA Center for the Health Sciences. All subjects completed a questionnaire designed to determine the parents' expectation and satisfaction concerning their child's psychological evaluation. Results showed positive impressions regarding evaluations, their usefulness, and their content.     

A case study comparing the influence of LED and fluorescent lighting on early childhood student engagement in a classroom setting

Studies suggest that the learning environment can act as a third teacher in contributing towards student academic success. A variable within the physical learning environment that has received little attention in the literature is the interior lighting. Because of budget constraints and age, most American public school classrooms have fluorescent lighting fixtures installed. A new lighting technology, Light Emitting Diodes (LEDs), is becoming popular in commercial facilities across the United States because of energy efficiency. Previous research has demonstrated that lighting influences adult worker productivity and mood in a workplace. However, because children process stimuli faster, it is unknown whether LED lighting would have the same influence in a learning environment. Researchers hypothesise that students display more engaged behaviours in classrooms lit with LEDs as compared with fluorescent lighting fixtures. The current study tested this hypothesis by observing child engagement behaviours in a pre-K classroom under LED lighting and fluorescent lighting fixtures to compare differences. Students displayed more engaged behaviours under the LED lighting condition.

     

Students' response to traditional and computer-assisted formative feedback: A comparative case study

The national movement towards progress files, incorporating personal development planning and reflective learning, is supported by lecturers providing effective feedback to their students. Recent technological advances mean that higher education tutors are no longer obliged to return comments in the ‘traditional’ manner, by annotating students' work with red pen. This paper considers some of the options currently available for returning computer‐assisted feedback, including the Electronic Feedback freeware. This MS Office application enables tutors to readily synthesise and email feedback reports to students. To further ascertain the value of this software, 169 1st‐year Pharmaceutical Science and Pharmacy students completed a questionnaire to gauge their reaction to formative feedback on an extended laboratory report. This included 110 responses from students graded by three tutors who marked work using either handwritten annotations or the Electronic Feedback program. Principle component analysis (PCA) of the Likert scale responses indicates that the identity of the marker did not significantly affect the response of students. However, the type of feedback was a factor that influenced the students' responses, with electronic feedback being rated superior. A Mann‐Whitney analysis of the satisfaction ratings (generated by PCA) indicates that four features of the assignment and feedback were considered significantly improved when the software was used to create feedback, namely; markscheme clarity, feedback legibility, information on deficient aspects, and identification of those parts of the work where the student did well. Modern academics face a number of challenges if they wish to return meaningful and timely feedback to students, among them large class sizes and infrequent face‐to‐face contact. It is pleasing to note, therefore, that assessors reported taking less time to mark when using the software. It is concluded that electronic formative feedback can be returned more quickly and may be used to synthesise relevant feedback that is both fair and balanced.     

Acquiescence,instructor’s gender bias and validity of student evaluation of teaching

Abstract

The validity of student evaluation of teaching (SET) scores depends on minimum effect of extraneous response processes or biases. A bias may increase or decrease scores and change the relationship with other variables. In contrast, SET literature defines bias as an irrelevant variable correlated with SET scores, and among many, a relevant biasing factor in literature is the instructor’s gender. The study examines the extent to which acquiescence, the tendency to endorse the highest response option across items and bias in the first sense affects students’ responses to a SET rating scale. The study also explores how acquiescence affects the difference in teaching quality (TQ) by instructor’s gender, a bias in the latter sense. SET data collected at a faculty of education in Ontario, Canada were analysed using the Rasch rating scale model. Findings provide empirical support for acquiescence affecting students’ responses. Latent regression analyses show how acquiescence reduces the difference in TQ by instructor’s gender. Findings encourage greater attention to the response process quality as a way to better defend the utility of SET and prevent potentially misleading conclusions from the analysis of SET data.     

Developer,teacher, student and employer evaluations of competence-based assessment quality

This study examines how different stakeholders experience the quality of a nationally developed assessment framework for summative, competence-based assessment (CBA) in AVET, which aims to reflect theoretical characteristics of high quality CBAs. The quality of two summative CBAs, based on this national framework, is evaluated along an extensive, validated set of quality criteria for CBA evaluation and through involving key stakeholders (i.e., students, teachers, developers, and employers). By triangulating quantitative and qualitative evaluations and argumentations of key stakeholders, this study gives insight into the processes and characteristics that determine CBA quality in VET educational practice in relation to theoretical notions of high quality CBAs. Results support many theoretical characteristics and refine them for reaching quality in actual assessment practice. Strikingly, developers and teachers are more critical about the assessment quality than students and employers. The discussion reflects on the theoretical CBA characteristics in the light of the empirical findings and deduces practical implications for the national assessment framework as well as other summative CBAs in VET.     

Assessing student workload in Problem Based Learning: Relationships among teaching method, student workload and achievement. A case study in Natural Sciences

This study examines student workload after a change in teaching style from lecture to Problem Based Learning and Cooperative Learning, and its relationship with student outcomes. Results show that the change clearly overloads students if it is not adequately planned and monitored. Marks, drop-outs and attendance were markedly better with the new instruction method, but also with a higher workload. The main conclusion is that calculating student workload in terms of hours is very important, but especially when implementing a change in teaching style.     

Attractiveness,easiness and other issues: student evaluations of professors on Ratemyprofessors.com

Felton et al. (2004 Felton, J., Mitchell, J. and Stinson, M. 2004. Web‐based student evaluation of professors: the relations between perceived quality, easiness and sexiness. Assessment and Evaluation in Higher Education, 29(1): 91–108. [Taylor & Francis Online] [Google Scholar]) reported that web‐based student evaluations of teaching (SET) demonstrated a student preference for course easiness and instructor sexiness. This study explores these same relationships with a larger and improved database. Results indicate even stronger relationships than previously reported. In addition, this study demonstrates significant cultural differences by institution and discipline in the relationships between Quality, Easiness, and ‘Hotness’ in web‐based SET.     

A generational examination of instructional Facebook use and the effects on perceived instructor immediacy,credibility and student affective learning

This study examined Millennial student perceptions of use of social networking, specifically Facebook, by instructors. Two independent variables were examined: instructor age (Baby Boomer or Millennial) and use of Facebook (utilising a course group site through the service versus not using the service at all). Results revealed that Baby Boomer instructors who used a class Facebook group were rated more highly on mediated immediacy, credibility and affective learning than Baby Boomer or Millennial instructors who did not use Facebook. The role of expectancy violations theory in Millennials’ perceptions of their instructors’ communication is discussed and recommendations are made for future research.     

Empowering student learning through Tablet PCs: A case study

Empowering students to learn through ICT is seen as a way to address the growing awareness and demand for preparing students to effectively participate in the emerging global knowledge economy. It is believed that, in order to maintain competitiveness in a global economy, traditional classroom practices must be re-conceptualized in a way that enables students to engage themselves in knowledge building, to become more self-directed, and to assume greater autonomy and social responsibility over their own learning. In this article, we report a case study on the use of Tablet PCs to support teaching and learning in a primary school in Hong Kong, and provide insights into how schools can harness and capitalize on the opportunities offered by such emerging technologies.     

Comparing student,instructor, classroom and institutional data to evaluate a seven-year department-wide science education initiative

We compared seven unrelated data-sets to evaluate a major education improvement initiative. Perceptions of students in 54 course sections were surveyed regarding the helpfulness of 39 specific teaching or learning strategies, and relative workloads and enthusiasm were compared to their other courses. Classes were observed using an established protocol, instructors completed a teaching practices inventory, and their experience with evidence-based pedagogies was established. A graduation exit survey provided longitudinal indications of changes prior to the study, and institutional student ratings of instruction were obtained. This study sought to determine whether improvements were consistently revealed by these data, how perceptions depended upon class size, course improvement model and instructor experience, and whether student ratings captured consistent perceptions about effectiveness. Overall, results compared favourably. Student perceptions and observed effectiveness depended mainly upon class size and improvement strategy. Students found experiences more effective in courses taught by experienced instructors and in classes observed to be more active. Relative workloads were not correlated with any measure of effectiveness while relative enthusiasm was higher in courses perceived to be more effective. Student ratings were consistent with other data-sets, although they did not provide information specific enough for informing further improvements.     

A comparison of student evaluations of teaching between online and face-to-face courses

The literature contains indications of a bias in student evaluations of teaching (SET) against online instruction compared to face-to-face instruction. The present case study consists of content analysis of anonymous student responses to open-ended SET questions submitted by 534 students enrolled in 82 class sections taught by 41 instructors, one online and one face-to-face class section for each instructor. There was no significant difference in the proportion of appraisal text segments by delivery method, suggesting no delivery method bias existed. However, there were significant differences in the proportion of text segments for topical themes and topical categories by delivery method. Implications of the findings for research and practice are presented.     

The student,the text,and the classroom context: A case study of teacher response

This article presents a case study of classroom-based research into teacher response. It examines the teacher-researcher's responding practices in the full context of a first-year college writing class, through an investigation of his written comments to one student, over time. The article outlines the principles and goals of the course, defines the role of teacher response within this setting, and critically examines the relation between response theory and this teacher's responding practice. The case study shows how the teacher's responses were shaped according to the sequence of assignments and classroom instruction, the teacher's style, and the work and needs of individual students. Effective teacher response, the article looks to demonstrate, is a matter of putting into practice certain accepted principles of response as well as a matter of adapting comments to the larger classroom context. The way a teacher presents comments on the pages of the student's text is just as important as locating response amid the particular circumstances of the class. The two activities are mutually dependent and reinforcing. By examining one teacher's response and reflection, the study suggests how other teachers might go about reflecting on their ways of responding in light of their own assignments, instruction, and goals and usefully integrate contemporary response theory in their classroom practice.