Correction to: An application of importance-performance analysis to students’ evaluation of teaching

Educational Assessment, Evaluation and Accountability -     

A new approach to evaluation of university teaching considering heterogeneity of students’ preferences

Students’ evaluations of teaching are increasingly used by universities to evaluate teaching performance. However, these evaluations are controversial mainly due to fact that students value various aspects of excellent teaching differently. Therefore, in this paper we propose a new approach to students’ evaluations of university teaching based on data from conjoint analysis. Conjoint analysis is a multivariate technique used to analyze the structure of individuals’ preference. In particular, our approach accounts for different importance students attach to various aspects of teaching. Moreover, it accounts explicitly for heterogeneity arising from students’ preferences, and incorporates it to form comprehensive teaching evaluation score. We have conducted survey and confirmed applicability and efficiency of the proposed approach.     

An analysis of teachers’ self-reported competencies for teaching metacognition

For successful reading experiences in native and/or foreign/second language, individuals need to benefit from not only cognitive strategies but also metacognitive strategies. Although research found reading comprehension and performance increase following metacognitive trainings, such findings may not transfer into mainstream classrooms as easily for several reasons. This study, therefore, aimed to disseminate the phenomenon of teaching metacognition with an emphasis on teacher’s instrumental role during classroom learning. More specifically, it investigated language instructors’ metacognition and their self-reported competencies for teaching metacognition. It also examined whether and how self-reported competencies changed following a professional development (PD) module of teaching for metacognition. Utilising Metacognitive Awareness Inventory (MAI) and think aloud protocols for instructional planning, this study found most participants were either highly metacognitive or metacognitive individuals. It was also found that most participants were initially not knowledgeable about and/or competent in teaching metacognition. Following PD, highly metacognitive teachers developed authentic lesson plans manifesting metacognition instruction while metacognitive teachers adopted similar instructional designs presented during the PD. Besides, half of the participants appreciated teaching metacognitively following the PD. Finally, under the light of these findings, future research and policy adjustments were proposed.     

Towards teaching analytics: a contextual model for analysis of students’ evaluation of teaching through text mining and machine learning classification

Education and Information Technologies - Recent trends in educational technology have led to emergence of methods such as teaching analytics (TA) in understanding and management of the...     

Does evaluation of teaching lead to improvement of teaching?

Abstract

Given the widespread use of student evaluation of teaching in North American colleges and universities, it is reasonable to ask whether student evaluation has accomplished one of its major intended outcomes, namely improvement of instructional quality. A review of research evidence from three independent sources (faculty surveys, field experiments and longitudinal comparisons) suggests that student evaluation does in fact contribute significantly to improvement of certain aspects of university teaching, particularly if evaluation is supplemented by expert consultation. Furthermore, there is no clear evidence that student evaluation has led to undesirable instructional side‐effects, such as grade inflation and entrenchment of traditional teaching methods.     

The number of feedbacks needed for reliable evaluation. A multilevel analysis of the reliability,stability and generalisability of students’ evaluation of teaching

A multilevel analysis approach was used to analyse students’ evaluation of teaching (SET). The low value of inter-rater reliability stresses that any solid conclusions on teaching cannot be made on the basis of single feedbacks. To assess a teacher’s general teaching effectiveness, one needs to evaluate four randomly chosen course implementations. Two implementations are needed when one course is evaluated, and if one implementation is evaluated, up to 15 feedbacks are needed. The stability of students’ ratings is very high, which reflects students’ stable rating criteria. There is an obvious rating paradox: from the student’s point of view, each rating is very precise, stable and justifiable, but from the teacher’s point of view a single feedback reflects the quality of teaching to just a moderate extent. Cross-hierarchical analysis reveals that there are large discrepancies between the uses of rating scales; some students are systematically more lenient in their rating whereas others are systematically more severe. The study also reveals that some courses are generally rated more favourably and that some courses are more suitable for certain teachers. Managers can thus improve the quality of teaching by finding the most suitable courses for each teacher.     

An idealistic view of teaching: teacher students’ personal practical theories

Personal practical theories (PPTs) are built based on underlying beliefs, experiences, values and conceptions regarding ‘good teaching’. Having a vision of teaching and valuing the work may have a strong effect on one’s sense of identity, resilience and commitment. This study focused on Finnish teacher students’ (N = 84) PPTs constructed during the middle or at the end of their studies, on average during their fourth year of a five-year MA programme. All of the PPTs, written as statements (N = 647), were combined, and a unified conception of the theories was formed using inductive content analysis. The three main categories found were the foundations of good teaching, the teacher and the teacher’s daily work. The analysis of the statements shows that the notions of good teaching are very idealistic: they emphasise abstract ideologies and high standards for the teacher, but also everyday matters of teaching, although also here the ideology of the pupil’s individuality is at the centre. The paper concludes by arguing that it is good to have high aims and values for teaching, although in reality some of them might be difficult to achieve.     

Students’ evaluation of academic courses: An exploratory analysis to an Italian case study

Students’ evaluations of teaching is a common practice in higher education institutions, with the main purpose of improving course quality and effectiveness. In this paper we would like to contribute to the existing literature on course and teaching evaluation by providing an empirical analysis based on questionnaires collected by an Italian private institution, namely the Libera Università Maria Ss. Assunta (LUMSA), for several degrees in Social Sciences. In order to identify the main factors affecting students’ satisfaction, we use not only teaching indicators and degree-specific characteristics, but also control for student-specific characteristics. Our analysis is based on a Multiple Correspondence Analysis for categorical variables, which represents a very useful method to study the multidimensional relationship among qualitative variables, along with a hierarchical clustering, in order to better summarize the results. Our findings reveal that student satisfaction relates to teaching and course organization. Moreover, we find some evidence that students typically evaluate their course on the basis of their experience rather than their personal interests.     

Reconsideration of “teaching evaluation by students” vision in china’s higher institution

In china,student’s evaluation of teaching(SET) is very appreciated by administrators,like a superstar.But I assert SET would cause strained relations between teacher and student.In order to achieve good teaching result,the administrators should care about the feelings of faculty,not always overemphasize the satisfaction of student.Only in this way can achieve a win-win situation on the teaching.     

The relationship between trainees’ evaluation of teaching and trainee performance among Norwegian executive students

The purpose of this study was to explore the relationship between students’ reactions to teaching and their consequent performance as measured by course grades. Reactions to teaching were reported by 189 adult part‐time students enrolled in postgraduate studies over two consecutive academic semesters across six different study programs in a large Norwegian training institution. These students’ reactions to teaching were then assessed in relation to their consequent performance, in terms of their individual final course exam grades. The results showed that students’ reactions to teaching were unrelated to their consequent course performance. Implications for practice and directions for future research are discussed.     

An exploratory study of changes to university EFL teachers’ teaching practices

The brief report describes an exploratory investigation into changes to three university EFL teachers’ teaching practices in a research-intensive university framed by managerialist technologies to improve its visibility and global ranking. The findings revealed that substantial changes have been brought about to their teaching practices in terms of falling into the routine of ‘technification of teaching’, producing a spectacle ‘what is there simply to be seen and judged’ or striving to seek out new spaces for ‘authentic’ English teaching. The study concluded with implications for facilitating the managerial reforms that are congruent with the cognitive and contextual realities of different generations of Chinese university EFL teachers.     

Engineering students’ experiences of interactive teaching in calculus

This study reports on an educational development initiative where peer instruction was used instead of traditional lectures in a calculus course for first-year engineering students. The aim of the study was to explore students’ experiences of this method. Data were collected by means of an open-ended questionnaire on two occasions: early and late in the course. The data were analyzed with an inductive content analysis. The findings comprise three qualitatively different ways to experience the interactive teaching method in calculus: (1) enthusiasm, (2) nuanced skepticism and (3) aversion. The categories differed regarding emotional reactions to the teaching, experiences of learning, conceptions of teaching and learning, and experiences of meaningfulness. Many students expressed enthusiasm for learning with peer instruction and noted that the method gave both teachers and students feedback on what students have difficulties with. These students perceived that they were responsible for their own learning. Other students experienced that peer instruction had some advantages and disadvantages, and preferred a mix between traditional lectures and peer instruction sessions. They seemed to believe that teachers and students share responsibility for learning. Some students expressed an aversion for peer instruction and the method seemed to challenge their beliefs of how teaching and learning is best conducted. Our study illustrates that educational development initiatives, even though based on research on student learning, do not benefit all students. One of the major obstacles seems to be that students’ underlying beliefs regarding teaching and learning may be counterproductive to the ideas behind the educational initiative. We suggest that beliefs regarding teaching and learning need to be addressed when introducing new teaching and learning methods.     

Instructor characteristics and students’ evaluation of teaching effectiveness: evidence from an Italian engineering school

This study examines the relationship between students’ evaluation of teaching effectiveness and the instructor's characteristics, based upon an original data set from an Italian university. Age and seniority (academic rank) are both found to affect negatively teaching evaluation, although the effect of rank depends heavily on the discipline. Profession-oriented disciplines are more poorly evaluated and exhibit stronger effects for rank; in the same disciplines, many courses are taught by non-faculty members, who are evaluated poorly. Gender is also found to be relevant, as it affects students’ evaluation of female instructors, although this result might be biased by the gender composition of classes. Past research outputs, as measured by an instructor's publication track record, has a positive impact on students’ evaluation of teaching effectiveness, although the rate of impact decreases with the amount of publications.     

Development and application of a scale to measure students’ STEM continuing motivation

ABSTRACT

The aim of this study was to develop a scale to measure students’ STEM continuing motivation (CM). In accordance with the conceptualisation of CM by Maehr (1976. Continuing motivation: An analysis of a seldom considered educational outcome. Review of Educational Research, 46(3), 443–462. doi:10.3102/00346543046003443), we define STEM CM as students’ tendencies to actively engage in STEM-related activities. The STEM Continuing Motivation Scale was developed from a literature review and student cognitive interviews, and was validated on a sample of 465 Chinese 7th and 8th graders. The results demonstrated that the developed scale had high reliability (Cronbach’s α?=?0.91) and good validity. Analysis of variance (ANOVA) showed that boys had significantly higher STEM CM than girls and there were significant differences in engineering CM among students from different schools. The implications of the findings of this study are also discussed.     

Grammar for writing? An investigation of the effects of contextualised grammar teaching on students’ writing

The role of grammar instruction in the teaching of writing is contested in most Anglophone countries, with several robust meta-analyses finding no evidence of any beneficial effect. However, existing research is limited in that it only considers isolated grammar instruction and offers no theorisation of an instructional relationship between grammar and writing. This study, drawing on a theorised understanding of grammar as a meaning-making resource for writing development, set out to investigate the impact of contextualised grammar instruction on students’ writing performance. The study adopted a mixed-methods approach, with a randomised controlled trial and a complementary qualitative study. The statistical analyses indicate a positive effect on writing performance for the intervention group (e = 0.21; p < 0.001); but the study also indicates that the intervention impact differentially on different sub-groups, benefiting able writers more than weaker writers. The study is significant in being the first to supply rigorous, theorised evidence for the potential benefits of teaching grammar to support development in writing.     

Teachers’ potential to promote students’ deeper learning in whole-class teaching: An observation study in Norwegian classrooms

Journal of Educational Change - Teachers worldwide are challenged to adjust their teaching to meet students’ needs for deeper learning. The lack of mutual understanding among researchers,...     

Teachers’ ability to assess students for teaching and supporting learning

Over the last decade, the pressure has increased for teachers to facilitate the best possible learning for all children. States collect much information to ensure that schools are accountable to all students. But is this information helping to improve provision for a diverse student population in inclusive settings? It is well established that teachers’ attitudes and beliefs influence students’ learning. Research evidence also shows that teachers seem to face particular difficulties in trying to understand students who have a disability or are performing poorly. Teacher education could contribute more substantially towards helping teachers to adequately understand those student characteristics most relevant to learning. This article presents a model to help understand the complex competency required to assess students adequately. It is suggested that introducing teachers to a functional language for describing disability could help teachers to focus on enabling learning and development rather than labelling.     

An exploration of Turkish teachers’ attributions to barriers faced within learner-centred teaching

This study explored Turkish teachers’ attributional reasoning regarding the barriers they face when they attempted to undertake learner-centred teaching, which is a core requirement of educational reform-based initiations in Turkey. To achieve this aim, a naturalistic inquiry was conducted in order to capture the clarifications of the barriers and related attributional reasoning of 31 teachers. The teachers defined 12 teaching barriers for the initial trials of learner-centred teaching and made several attributions to the faced barriers. The study showed that the teachers made pervasively externalised attributions when explicating the reasons for the barriers they faced. They were the usual suspects of making fundamental attribution error due to the excessive externality of their attributions. The teachers’ barrier clarifications and accompanied attributional reasoning typologies in terms of learner-centred teaching have confirmed that the teachers in this study are only in the early stages in regard to being competent implementers of reform-based initiatives in Turkey.