Transformational classroom leadership: a novel approach to evaluating classroom performance

In higher education, student evaluation of teaching is widely used as a measure of an academic’s teaching performance despite considerable disagreement as to its value. This paper begins by examining the merit of teaching evaluations with reference to the factors influencing the accuracy of the teaching evaluation process. One of the central assumptions on which student evaluation of teaching is based is that there is a relationship between student achievement and student rating of teachers. However, the findings of the majority of studies do not support this assumption. The absence of a strong link between student achievement and teaching evaluations suggests that there is scope for examining other approaches to measuring effective classroom dynamics. This paper presents such an approach based on the notion of transformational classroom leadership.     

The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates

Course evaluations (often termed student evaluations of teaching or SETs) are pervasive in higher education. As SETs increasingly shift from pencil-and-paper to online, concerns grow over the lower response rates that typically accompany online SETs. This study of online SET response rates examined data from 678 faculty respondents and student response rates from an entire semester. The analysis focused on those tactics that faculty employ to raise response rates for their courses, and explored instructor and course characteristics as contributing factors. A comprehensive regression model was evaluated to determine the most effective tactics and characteristics. Using incentives had the most impact on response rates. Other effective tactics that increase response rates include reminding students to take the evaluation, explaining how the evaluations would be used to improve instruction, sending personal emails and posting reminders on Blackboard^®. Incentives are not widely used; however, findings suggest that non-point incentives work as well as point-based ones, as do simple-to-administer minimum class-wide response rate expectations (compared to individual completion).     

What motivates higher education educators to innovate? Exploring competence,autonomy, and relatedness – and connections with wellbeing

ABSTRACT

Background: High quality tertiary teaching is important for maximising the impact of tertiary education for students, employers and society, as well as for institutional reputation and accountability. Varied interpretations exist regarding what counts as ‘innovative’ tertiary teaching.

Purpose: This study sought to explore the deeper drivers of motivations of tertiary educators to initiate and implement teaching innovations.

Method: Drawing from the data from semi-structured interviews from a wider study into innovative teaching in a tertiary education institution in Aotearoa New Zealand, we analysed the rationale of 13 expert tertiary educators for teaching in novel ways. The framework for analysis was based on Self-Determination Theory: data were analysed in relation to three basic psychological needs known to impact on motivation and wellbeing – competence, autonomy and relatedness.

Findings: Our analysis suggests that tertiary educator motivation to innovate in their teaching is related to feelings of all three needs. Findings suggest that innovative pedagogical change was linked to educators’ need for feelings of competence in relation to subject area content, pedagogy and developing student competence. Enhancing autonomy for the educators themselves, or for their students, was also an important factor motivating innovative practice. Developing relatedness between students, between themselves and students, and between students and society were reported as motivations for innovative teaching. Some teaching innovations were motivated by the educator’s desire for greater personal or student wellbeing.

Conclusions Educators’ motivations for being innovative in their tertiary pedagogical practice encompassed content, pedagogical, personal and social dimensions. None reported that their innovation was motivated by institutional policy or expectations. The reported challenges to innovative teaching practice, including institutional structures and processes, lack of access to specific technologies, and the time needed to develop and implement changed practice, indicate that educators must be strongly committed to making change in their practice to sustain the effort required.     

Tertiary teachers and student evaluations: never the twain shall meet?

Internationally, centralised systems of student evaluation have become normative practice in higher education institutions, providing data for monitoring teaching quality and for teacher professional development. While extensive research has been done on student evaluations, there is less research-based evidence about teachers’ perceptions of and engagement with student evaluations, the focus of the research reported in this paper. An interpretive approach framed the study in which data were gathered through questionnaire and interview responses from teaching staff at three New Zealand tertiary institutions. Results highlighted the general acceptance of the notion of student evaluations, recurring ideas about the limitations of evaluations and significant gaps in the way academics engage with student evaluation feedback. Recommendations for enhancing teacher engagement with student evaluation are made to optimise the potential for student evaluations to inform teaching development and to improve students’ learning experiences.     

Functioning and participation problems of students with ASD in higher education: which reasonable accommodations are effective?

Abstract

Students with autism spectrum disorder (ASD) experience various functioning and participation problems in higher education, which may cause difficulties such as drop out or low grade point averages. However, it remains unclear how often and during which teaching and evaluation methods the functioning and participation problems occur and which reasonable accommodations are effective in dealing with them. These gaps in the literature are addressed in this survey-based study. In total, 43 students with ASD, 30 student counsellors and 43 students without a disability of institutions of higher education in Flanders (Belgium) participated in the study. The results show that students with ASD most frequently experience problems with verbal and non-verbal communication, are oversensitive to change and have difficulty distinguishing the gist of the syllabus from the details. Furthermore, it is shown that, on average, these problems arise mostly during classical teaching and evaluation methods. Finally, the perceived effectiveness of reasonable accommodations is dependent on the functioning and participation problem experienced by the student with ASD in higher education. In conclusion, both personal and environmental characteristics should be taken into account when selecting and implementing reasonable accommodations for these students.     

A Comparison of Students' Evaluations of a Peer-delivered Sex Education Programme and Teacher-led Provision

This article reports data from the process evaluation of a randomised controlled trial of peer-led sex education in English secondary schools. Data from 52 focus group discussions in 19 schools and selected items from a questionnaire survey completed by 7770 students in 1998/99 are used to compare student views on teacher- and peer-led sex education. Questionnaire data show that a significantly greater proportion of students taught by peer educators than teachers felt that sex education was enjoyable, engaging and useful to them. Data also show that a slightly greater proportion of students taught by teachers felt the classroom was well controlled. Data from an analysis of the focus group discussions are used to explore and illustrate factors which students associated with the acceptability and satisfaction of the peer-led intervention. The importance of peer educator empathy and similarity with students is highlighted and data show how these characteristics are associated with peer educators' choice of teaching methods, the content of lessons and the sexual values and attitudes they espouse. The influence of contextual factors like student prior expectations and experiences of relationships with teachers, school environment and norms are considered along with the potential influence of the research process on responses.     

The Withdrawal of Non-traditional Students: Developing an explanatory model

In the past decade there has been a dramatic growth in UK student numbers, with students from non-traditional and under-represented groups being encouraged to participate in higher education. However, levels of withdrawal in those higher education institutions that have a greater proportion of non-traditional students have also been increasing. Higher education institutions have introduced various strategies in an effort to cope with this problem, but unfortunately current models offer little in the way of explaining the causes of (and little justification for the strategies used to reduce) withdrawal. It is proposed that in attending to the explanations of withdrawal, consideration must be given to discovering the underlying characteristics of the teaching and learning environment and the manner in which a student's perceptions and expectations of that environment may impact on their decision to withdraw. An ethnographical study using grounded theory is used to capture these underlying characteristics. This study provides an explanation of the teaching and learning environment as it relates to the student's beliefs, the actions between the student and staff and the intentions of the institution. The results of this initial study are presented. These results represent a preliminary 'grounded' model of the teaching and learning environment of the Technology Faculty at Southampton Institute.     

An Investigation of the Joint Influence of Faculty Control and Student Locus of Reinforcement on Instructional Evaluation

Abstract

This research was based on the theoretical belief that student evaluations of faculty teaching are related to the person-environment congruence between students’ locus of reinforcement and the degree of control exercised by faculty instructional styles. Fifty-seven college students were classified as internally or externally reinforced based on Rotter’s locus of reinforcement theory. Students evaluated high and low faculty control instructional styles by rating daily seven classroom evaluative questions on a five-point differential scale. It was hypothesized that the evaluation of instruction by students experiencing congruent control styles (internal students/low control and external students/high control) would be more positive than incongruent combinations (external students low control and internal students/high control). ANOVA results yielded a significant 937’value p <.01) for interaction effects which supported the research hypothesis. Implications for student evaluation of faculty teaching are discussed.     

Teacher expectations in a university setting: the perspectives of teachers

ABSTRACT

This study was designed to investigate the bases of teacher expectations in higher education. The first author interviewed 20 university teachers from an English-as-a-foreign-language course, exploring their expectations for the first-year undergraduates in their classes. The grounded theory method was adopted to analyse the data that had been collected. The results showed that for this sample of 20 teachers, student characteristics were important contributing factors to their expectations in the teachers’ university settings. The factors the teachers considered important included students’ (a) prior academic achievement, (b) motivation, (c) study skills, and (d) academic discipline. Also, teacher characteristics were found to be another major source of these university teachers’ expectations, including teachers’ (a) past teaching and learning experience and (b) teaching self-efficacy. The findings suggested that the bases of teacher expectations in higher education may differ from those at the elementary or secondary school level.     

New Directions in the Evaluation of the Effectiveness of Educational Technology

Summary

Drawing from work by Shadish, Cook, and Leviton (1991) on social program evaluation, the authors discuss recent changes in evaluation theory and practices, and they connect these changes to technology and student learning. Concluding with a list of recommendations for evaluating the effectiveness of technology in teaching and learning, the authors challenge the purposes of education and prevalent goals for evaluation. After questioning how technology can impact student learning, they call for new and expanded definitions of student learning outcomes. Recommendations include redefining technology as a process rather than as a product, conducting implementation evaluations prior to outcome evaluations; reducing the reliance on standardized test scores as the primary outcome measure; and adopting multifaceted evaluation approaches (including case studies).     

Liberal Arts Core Curriculum Quality Assurance Based on Teaching Evaluation by Students: A Case Study of Fudan University

Abstract

After more than 20 years of exploring liberal arts education in Chinese universities, its curriculum development has been institutionalized. Efforts to build localized curricula that fit the realities at each institution have shifted from focusing on quantity and structure to emphasizing curriculum quality assurance based on teaching evaluation by students. An empirical analysis using Fudan University’s teaching evaluation data shows that some liberal arts core courses offered by technical and engineering departments are too specialized, while some offered by social sciences and humanities and languages departments lack rigor. Student evaluations of such courses therefore tend to be low. Improving the quality of liberal arts courses can begin by examining both teaching evaluations by students and course grading, starting with the less rigorous courses for non-majors. Improving the quality of liberal arts courses can begin with the courses that are too specialized or lacking rigor based on both teaching evaluations by students and grade distribution.     

Enhancing the pedagogical value of multimedia products in higher education

Abstract

In higher education settings two aims of multimedia development and use are: to enhance student learning by changing the nature of the curriculum experience, and to improve the teaching environment. This paper proceeds from the premise that the involvement of academic developers in the production of multimedia projects has the potential to enhance the pedagogical value of the final product. The evaluation of multimedia projects is considered in relation to five issues: conceptual need, students, medium, integration, and relation to existing products. Also presented are associated questions which can be used to explore each issue, and corresponding techniques which can be used to answer those questions. Suggestions are made for ways in which academic developers might create opportunities to work on multimedia development, and specific journals are suggested to assist developers in becoming familiar with the field.     

The Nominal Group Technique: extending the evaluation of students' teaching and learning experiences

Obtaining and evaluating student opinion about their teaching and learning experiences can, if taken seriously, be a complex process. There are not many approaches available for student evaluations, and there has in our view, been an over dependence on course evaluation questionnaires. The Nominal Group Technique (NGT) is a less widely used approach, which this paper argues provides a potential way of extending the evaluation of students' experiences of courses. The paper specifically considers the use of the NGT in the context of an evaluation of the first three cohorts of students on a new degree course. The strengths and weaknesses of the approach as they have emerged in this evaluation are considered. Attention is also paid to evidence concerning the use of the NGT in other contexts. The paper concludes that the NGT has much to offer within the portfolio of student evaluation methods currently being used in higher education. Its particular strengths are assessed in relation to certain possible shortcomings, and its usefulness is considered in relation to various possible contexts. The arguments are illustrated by data taken from our current evaluation study.     

‘Content’ versus ‘style’: Acquiescence in student evaluation of teaching?

Structural equation modelling is used to measure the existence of a response style (in particular, acquiescence) behind three balanced Likert scales measuring different concepts in a questionnaire for student evaluation of teaching in higher education. Exploration with one sample (n = 1125) and confirmation in a second sample (n = 710) from a different population (different students, different courses) shows the existence of a common factor behind these scales. However, the weak correlation with the variable ‘sum of agreements’ does not support the idea of acquiescence in student evaluation of teaching. Instead, the common factor could be interpreted as a halo factor influencing different scales in the questionnaire. The implications of this exploratory study for future research on acquiescence in student evaluations of teaching are discussed.     

Functioning and participation problems of students with ADHD in higher education: which reasonable accommodations are effective?

Abstract

Students with ADHD struggle in higher education as a result of various functioning and participation problems. However, there are remaining gaps in the literature. First, it remains unclear how often and during which teaching and evaluation methods problems arise. Second, we do not yet know which reasonable accommodations are most effective to deal with the functioning. And third, we do not know which accommodations are most effective to address participation problems of students with ADHD in higher education. This study addresses these three gaps in literature. In total, 86 students with ADHD, 42 student counsellors and 86 students without a disability participated in a survey-based study. The results show that students with ADHD most frequently experience problems with sustaining and focusing attention and it is demonstrated that most problems arise during classical teaching or evaluation methods. Finally, the perception of the effectiveness of reasonable accommodations is strongly dependent on which problems students experience in higher education. These findings suggest that it is important to consider both personal and environmental characteristics when selecting and implementing reasonable accommodations.     

Action Learning Circles: action learning in theory and practice

ABSTRACT

This paper is a commentary on how action learning circles can be effectively used (to offer experiential and emancipatory learning for students. The focus is on the preparation of students for health care education, and its value in the context of professional client‐based teaching and learning. The concept of action learning is described, and its role in promoting learning from practice explored using student and teacher evaluation evidence. An outline of the use of action learning circles in practice and the subsequent learning is analysed using evaluation data to inform our discussion, which gives the perspective of both learner and teacher.     

Evaluate to improve: useful approaches to student evaluation

Many teachers in higher education use feedback from students to evaluate their teaching, but only some use these evaluations to improve their teaching. One important factor that makes the difference is the teacher’s approach to their evaluations. In this article, we identify some useful approaches for improving teaching. We conducted focus groups with award-winning university teachers who use student evaluations to improve their teaching, and we identified how they approach their evaluation data. We found that these teachers take a reflective approach, aiming for constant improvement, and see their evaluation data as formative feedback, useful for improving learning outcomes for their students. We summarise this as the improvement approach, and we offer it for other teachers to emulate. We argue that if teachers take this reflective, formative, student-centred approach, they can also use student evaluations to improve their teaching, and this approach should be fostered by institutions to encourage more teachers to use student evaluations to improve their teaching.     

The Assessment of Higher Education Quality from the Perspective of Students through a Case Study Analysis

The assessment of quality in higher education from the perspective of students has three dimensions: students’ assessment of teaching, students’ satisfaction and students’ learning engagement. These differ in conceptions of quality, evaluation methods, evaluation content, evaluation purposes, traits and priorities. The authors conducted three rounds of empirical investigations to study higher education quality assessment from students’ perspective and concluded that students play multiple roles in higher education evaluation and assessment, all of which can be improved by strengthening students’ objectivity and participation, evaluating the added value of a college education oriented to student development and taking the students’ perspective as an important way to contribute to higher education quality enhancement, assurance and control, and make proper use of higher education evaluations and assessments.     

Disciplinary,Gender and Course Year Influences on Student Perceptions of Teaching: Explorations and implications

The prevalent use of student ratings in teaching evaluations, particularly the reliability of such data, has been debated for many years. Reports in the literature indicate that there are many factors influencing student perceptions of teaching. Three of these factors were investigated at the University of Western Australia, namely the broad discipline group, course/unit year level and student gender. Data collected over 3 years were analysed. The outcomes of this study confirmed results reported by other workers in the field that there are differences in ratings of students in different discipline groups and at different year levels. It also provided a possible explanation for the mixed results reported in studies of student gender in relation to student ratings.     

How reliable are students’ evaluations of teaching quality? A variance components approach

The inter-rater reliability of university students’ evaluations of teaching quality was examined with cross-classified multilevel models. Students (N = 480) evaluated lectures and seminars over three years with a standardised evaluation questionnaire, yielding 4224 data points. The total variance of these student evaluations was separated into the variance components of courses, teachers, students and the student/teacher interaction. The substantial variance components of teachers and courses suggest reliability. However, a similar proportion of variance was due to students, and the interaction of students and teachers was the strongest source of variance. Students’ individual perceptions of teaching and the fit of these perceptions with the particular teacher greatly influence their evaluations. This casts some doubt on the validity of student evaluations as indicators of teaching quality and suggests that aggregated evaluation scores should be used with caution.