An application of importance-performance analysis to students’ evaluation of teaching

Educational Assessment, Evaluation and Accountability - Students’ feedback is usually gathered in institutions of higher education to evaluate the teaching quality from the students’...     

A new approach to evaluation of university teaching considering heterogeneity of students’ preferences

Students’ evaluations of teaching are increasingly used by universities to evaluate teaching performance. However, these evaluations are controversial mainly due to fact that students value various aspects of excellent teaching differently. Therefore, in this paper we propose a new approach to students’ evaluations of university teaching based on data from conjoint analysis. Conjoint analysis is a multivariate technique used to analyze the structure of individuals’ preference. In particular, our approach accounts for different importance students attach to various aspects of teaching. Moreover, it accounts explicitly for heterogeneity arising from students’ preferences, and incorporates it to form comprehensive teaching evaluation score. We have conducted survey and confirmed applicability and efficiency of the proposed approach.     

Towards teaching analytics: a contextual model for analysis of students’ evaluation of teaching through text mining and machine learning classification

Education and Information Technologies - Recent trends in educational technology have led to emergence of methods such as teaching analytics (TA) in understanding and management of the...     

An idealistic view of teaching: teacher students’ personal practical theories

Personal practical theories (PPTs) are built based on underlying beliefs, experiences, values and conceptions regarding ‘good teaching’. Having a vision of teaching and valuing the work may have a strong effect on one’s sense of identity, resilience and commitment. This study focused on Finnish teacher students’ (N = 84) PPTs constructed during the middle or at the end of their studies, on average during their fourth year of a five-year MA programme. All of the PPTs, written as statements (N = 647), were combined, and a unified conception of the theories was formed using inductive content analysis. The three main categories found were the foundations of good teaching, the teacher and the teacher’s daily work. The analysis of the statements shows that the notions of good teaching are very idealistic: they emphasise abstract ideologies and high standards for the teacher, but also everyday matters of teaching, although also here the ideology of the pupil’s individuality is at the centre. The paper concludes by arguing that it is good to have high aims and values for teaching, although in reality some of them might be difficult to achieve.     

Students’ evaluation of academic courses: An exploratory analysis to an Italian case study

Students’ evaluations of teaching is a common practice in higher education institutions, with the main purpose of improving course quality and effectiveness. In this paper we would like to contribute to the existing literature on course and teaching evaluation by providing an empirical analysis based on questionnaires collected by an Italian private institution, namely the Libera Università Maria Ss. Assunta (LUMSA), for several degrees in Social Sciences. In order to identify the main factors affecting students’ satisfaction, we use not only teaching indicators and degree-specific characteristics, but also control for student-specific characteristics. Our analysis is based on a Multiple Correspondence Analysis for categorical variables, which represents a very useful method to study the multidimensional relationship among qualitative variables, along with a hierarchical clustering, in order to better summarize the results. Our findings reveal that student satisfaction relates to teaching and course organization. Moreover, we find some evidence that students typically evaluate their course on the basis of their experience rather than their personal interests.     

An analysis of teachers’ self-reported competencies for teaching metacognition

For successful reading experiences in native and/or foreign/second language, individuals need to benefit from not only cognitive strategies but also metacognitive strategies. Although research found reading comprehension and performance increase following metacognitive trainings, such findings may not transfer into mainstream classrooms as easily for several reasons. This study, therefore, aimed to disseminate the phenomenon of teaching metacognition with an emphasis on teacher’s instrumental role during classroom learning. More specifically, it investigated language instructors’ metacognition and their self-reported competencies for teaching metacognition. It also examined whether and how self-reported competencies changed following a professional development (PD) module of teaching for metacognition. Utilising Metacognitive Awareness Inventory (MAI) and think aloud protocols for instructional planning, this study found most participants were either highly metacognitive or metacognitive individuals. It was also found that most participants were initially not knowledgeable about and/or competent in teaching metacognition. Following PD, highly metacognitive teachers developed authentic lesson plans manifesting metacognition instruction while metacognitive teachers adopted similar instructional designs presented during the PD. Besides, half of the participants appreciated teaching metacognitively following the PD. Finally, under the light of these findings, future research and policy adjustments were proposed.     

Does evaluation of teaching lead to improvement of teaching?

Abstract

Given the widespread use of student evaluation of teaching in North American colleges and universities, it is reasonable to ask whether student evaluation has accomplished one of its major intended outcomes, namely improvement of instructional quality. A review of research evidence from three independent sources (faculty surveys, field experiments and longitudinal comparisons) suggests that student evaluation does in fact contribute significantly to improvement of certain aspects of university teaching, particularly if evaluation is supplemented by expert consultation. Furthermore, there is no clear evidence that student evaluation has led to undesirable instructional side‐effects, such as grade inflation and entrenchment of traditional teaching methods.     

The number of feedbacks needed for reliable evaluation. A multilevel analysis of the reliability,stability and generalisability of students’ evaluation of teaching

A multilevel analysis approach was used to analyse students’ evaluation of teaching (SET). The low value of inter-rater reliability stresses that any solid conclusions on teaching cannot be made on the basis of single feedbacks. To assess a teacher’s general teaching effectiveness, one needs to evaluate four randomly chosen course implementations. Two implementations are needed when one course is evaluated, and if one implementation is evaluated, up to 15 feedbacks are needed. The stability of students’ ratings is very high, which reflects students’ stable rating criteria. There is an obvious rating paradox: from the student’s point of view, each rating is very precise, stable and justifiable, but from the teacher’s point of view a single feedback reflects the quality of teaching to just a moderate extent. Cross-hierarchical analysis reveals that there are large discrepancies between the uses of rating scales; some students are systematically more lenient in their rating whereas others are systematically more severe. The study also reveals that some courses are generally rated more favourably and that some courses are more suitable for certain teachers. Managers can thus improve the quality of teaching by finding the most suitable courses for each teacher.     

Instructor characteristics and students’ evaluation of teaching effectiveness: evidence from an Italian engineering school

This study examines the relationship between students’ evaluation of teaching effectiveness and the instructor's characteristics, based upon an original data set from an Italian university. Age and seniority (academic rank) are both found to affect negatively teaching evaluation, although the effect of rank depends heavily on the discipline. Profession-oriented disciplines are more poorly evaluated and exhibit stronger effects for rank; in the same disciplines, many courses are taught by non-faculty members, who are evaluated poorly. Gender is also found to be relevant, as it affects students’ evaluation of female instructors, although this result might be biased by the gender composition of classes. Past research outputs, as measured by an instructor's publication track record, has a positive impact on students’ evaluation of teaching effectiveness, although the rate of impact decreases with the amount of publications.     

Teachers’ potential to promote students’ deeper learning in whole-class teaching: An observation study in Norwegian classrooms

Journal of Educational Change - Teachers worldwide are challenged to adjust their teaching to meet students’ needs for deeper learning. The lack of mutual understanding among researchers,...     

Course evaluation matters: improving students’ learning experiences with a peer-assisted teaching programme

In the rapidly changing global higher education sector, greater attention is being paid to the quality of university teaching. However, academics have traditionally not received formal teacher training. The peer-assisted teaching programme reported on in this paper provides a structured yet flexible approach for peers to assist each other in reinvigorating and refining their teaching practice. Academics participated in this national, multi-institutional trial for varied reasons: the majority voluntarily, others to increase low student evaluation of course scores and some as part of a graduate certificate teaching qualification. Here we report on how academics used the scheme, and the teaching areas they focused on. Student evaluation of course scores increased in the majority of courses, suggesting the changes made had a positive effect on students’ learning experiences. The experiences of the multi-institutional trial reported here may benefit others considering such a scheme to reinvigorate and refine teaching practice and improve course evaluation scores.     

Reconsideration of “teaching evaluation by students” vision in china’s higher institution

In china,student’s evaluation of teaching(SET) is very appreciated by administrators,like a superstar.But I assert SET would cause strained relations between teacher and student.In order to achieve good teaching result,the administrators should care about the feelings of faculty,not always overemphasize the satisfaction of student.Only in this way can achieve a win-win situation on the teaching.     

The relationship between trainees’ evaluation of teaching and trainee performance among Norwegian executive students

The purpose of this study was to explore the relationship between students’ reactions to teaching and their consequent performance as measured by course grades. Reactions to teaching were reported by 189 adult part‐time students enrolled in postgraduate studies over two consecutive academic semesters across six different study programs in a large Norwegian training institution. These students’ reactions to teaching were then assessed in relation to their consequent performance, in terms of their individual final course exam grades. The results showed that students’ reactions to teaching were unrelated to their consequent course performance. Implications for practice and directions for future research are discussed.     

Correction to: Saudi science teachers’ perceptions of the cultural factors influencing elementary students’ science learning

Cultural Studies of Science Education - In the Original Publication of the article, second author’s name was misspelt.     

An exploratory study of changes to university EFL teachers’ teaching practices

The brief report describes an exploratory investigation into changes to three university EFL teachers’ teaching practices in a research-intensive university framed by managerialist technologies to improve its visibility and global ranking. The findings revealed that substantial changes have been brought about to their teaching practices in terms of falling into the routine of ‘technification of teaching’, producing a spectacle ‘what is there simply to be seen and judged’ or striving to seek out new spaces for ‘authentic’ English teaching. The study concluded with implications for facilitating the managerial reforms that are congruent with the cognitive and contextual realities of different generations of Chinese university EFL teachers.     

An interdisciplinary analysis of microteaching evaluation forms: how peer feedback forms shape what constitutes “good teaching”

Microteaching, a standard method for developing teaching skills, places high importance on peer feedback, which is guided by post-session feedback forms. This paper focuses on how feedback forms can shape what becomes understood as important to teaching. A sample of 10 microteaching evaluation forms drawn from North American postsecondary education institutions were examined using both qualitative and quantitative methods. Through a mixed-methods, interdisciplinary approach combining quantitative form classification based on distinct teaching elements with qualitative analysis drawing on Foucauldian and post-structural feminisms, key challenges are identified in the way that peer feedback forms may shape perceptions of what constitutes “good teaching”. We interpret that the close attention paid to the management of the body and the disproportionate focus on presentation and style may foreclose other modes of teaching beyond a conventional lecture-based class. This leads to a discussion on the ways in which the evaluation forms can be enhanced to provide a more effective educational tool.     

Correction to: Primary school students’ perceived social support in relation to study engagement

European Journal of Psychology of Education -     

Teachers’ perceptions of and responses to student evaluation of teaching: purposes and uses in clinical education

Student evaluation of teaching (SET) only becomes an effective tool for improving teaching and learning when the relevant stakeholders seriously consider and plan appropriate actions according to student feedback. It is common practice in medical education to provide clinical teachers with student feedback. However, there is limited evidence about how teachers in higher education, and medical education in particular, systematically apply student feedback to improve the quality of their teaching practice. The focus of this case study was to examine clinical teachers’ perceptions of and responses to SET with respect to its purposes and uses for enhancing their teaching. An explanatory sequential mixed methods approach was employed to collect both quantitative and qualitative data from the clinical coaches. These clinical coaches perceived the main purpose of student evaluation as quality assurance, and were moderately receptive to student feedback. Four key factors enabling or inhibiting their responses were revealed: institutional requirements, operational practices, personal biases and provision of support. Future research should further explore the interrelationships among the above factors as the core mechanism in influencing clinical teachers’ perceptions of and responses to student evaluation.     

Engineering students’ experiences of interactive teaching in calculus

This study reports on an educational development initiative where peer instruction was used instead of traditional lectures in a calculus course for first-year engineering students. The aim of the study was to explore students’ experiences of this method. Data were collected by means of an open-ended questionnaire on two occasions: early and late in the course. The data were analyzed with an inductive content analysis. The findings comprise three qualitatively different ways to experience the interactive teaching method in calculus: (1) enthusiasm, (2) nuanced skepticism and (3) aversion. The categories differed regarding emotional reactions to the teaching, experiences of learning, conceptions of teaching and learning, and experiences of meaningfulness. Many students expressed enthusiasm for learning with peer instruction and noted that the method gave both teachers and students feedback on what students have difficulties with. These students perceived that they were responsible for their own learning. Other students experienced that peer instruction had some advantages and disadvantages, and preferred a mix between traditional lectures and peer instruction sessions. They seemed to believe that teachers and students share responsibility for learning. Some students expressed an aversion for peer instruction and the method seemed to challenge their beliefs of how teaching and learning is best conducted. Our study illustrates that educational development initiatives, even though based on research on student learning, do not benefit all students. One of the major obstacles seems to be that students’ underlying beliefs regarding teaching and learning may be counterproductive to the ideas behind the educational initiative. We suggest that beliefs regarding teaching and learning need to be addressed when introducing new teaching and learning methods.