Evaluating the effect of a lecturer’s language background on a student rating of teaching form

This study, using student ratings of lecturers, examines the perceived effect of the lecturer’s ability to communicate effectively. The relationship between the standard question—’The lecturer was able to communicate ideas and information clearly’—and the global rating question—’Overall, the lecturer is an effective teacher’—was investigated in 7072 undergraduate standard teaching surveys from one university, using the lecturer’s language background as a factor. The results show that overall student ratings of English as a second language (ESL) lecturers are, on average, 0.4 points lower on a five‐point scale than student ratings of native English speaking lecturers. There is a strong interaction between this average difference and the lecturer’s faculty, with little difference in arts (humanities and social sciences) through to 0.6 points difference in science. The study also found that, of the four categorical questions used in the university’s standard teaching survey, the ‘communication’ question had the highest correlation with the ‘overall’ question. The correlation (R?=?0.96) suggests that the standard teaching survey is overly influenced by the students’ perception of this one aspect of teaching—reflecting a transmission model. The rating difference between ESL and native English speaking lecturers is briefly explored. In addition, the paper briefly considers the implications of the above findings for teacher development and for student expectations against a background of a growing ESL student population.     

The effects of mandatory and optional use on students' ratings of a computer-based learning package

Continuing developments in educational technology and increasing undergraduate numbers in the UK have facilitated the use of computer‐based learning packages. One of the effects of the increasing use of these packages is the shift in teaching of material from being primarily lecturer‐led to student‐driven. This has a number of implications concerning using the package, since it is quite likely that students will be working in isolation, and the use of the package may be mandatory or optional. In this study we report on the usefulness ratings of a software package for undergraduate use within the context of whether or not students choose to use it. Level 1 students and lecturers in Economics were surveyed at three UK universities. Findings indicate that students whose use was mandatory rated the learning package as more useful than those whose use was optional. Reasons for non‐use centred on lack of instruction and on student apathy. These results have important implications for the use of computer‐based learning packages and for understanding user attitudes.     

Factor analysis of the pervasive developmental disorders rating scale with teacher ratings of students with autistic disorder

The Pervasive Developmental Disorders Rating Scale (PDDRS; Eaves, 2003) is a rating scale that is used in the screening process for pervasive developmental disorders (PDD). The PDDRS contains three scales: Arousal, Affect, and Cognition. In this study, the construct validity of the PDDRS was examined with teacher ratings from a sample of 168 children with autistic disorder through exploratory and confirmatory factor analysis. In the exploratory factor analysis, a three‐factor structure was examined. Confirmatory factor analysis was employed to evaluate the model‐fit of the data for the hypothesized second‐order factor structure. The results supported the averred factor structure of the PDDRS with teacher ratings for students with autistic disorder. Limitations of the data and areas for future research are discussed. © 2005 Wiley Periodicals, Inc. Psychol Schs 42: 207–216, 2005.     

Compulsory Higher Education Teacher Training in Sweden: Development of a national standards framework based on the Scholarship of Teaching and Learning

Today visible proofs of excellence in teaching and learning are increasingly important aspects of institutional branding in higher education (HE). Teaching competence is brought forward as a central aspect of the quality of programmes. Still, the induction of new university teachers is managed in many different ways. Approaches may vary according to how teaching competence is perceived; as growing from practice only, requiring formal courses or, for instance, outlined in the Scholarship of Teaching and Learning (SoTL) movement. In Sweden, the HE Ordinance from 2002 states that to get permanent positions, lecturers should have completed Compulsory HE Teacher Training (CHETT). The size and organisation of the courses were not regulated in the Ordinance and institutional practices varied. In a three‐year project intended learning outcomes for CHETT was suggested. These outcomes are based on SoTL and linked to an estimated work‐load of 10 weeks. Based on a national survey in 2006, institutional responses to the proposals are analysed.     

Perceptions of future and cur rent teachers on the organization of elementary schools: a dissonance approach to the investigation of job satisfaction

The study compares the satisfaction ratings of future and current elementary school teachers with respect to the main aspects of school organization and administration. Data were collected from 66 in-service teachers and 79 preservice teachers in Cyprus. The application of factor analysis on the data resulted in the identification of the following main dimensions of school organization: headmaster's role, school organization, school climate, teacher incentives/work conditions, inspector's role and teachers' role. The comparison of the satisfaction ratings of future and current teachers on the above factors produced the following findings: first, the ratings of the two groups differed significantly for the first four factors. Secondly, future teachers reported lower satisfaction ratings than their in-service counterparts in the headmaster's role, school organization and school climate factors and higher ratings on the teacher incentives/work conditions factor.These discrepancies are considered to constitute causes of concern for educational policy-makers. The high expectations of future teachers on the teacher incentives/work conditions factor are viewed as a probable subsequent cause of cognitive dissonance and job dissatisfaction. On the other hand, the low expectations of future teachers with respect to three factors (headmaster's role, school organization, school climate) are likely to reduce their initial enthusiasm for their job. The implications of the findings for teacher training programmes are discussed, and suggestions are made on ways of reducing discrepancies between the expectations of future teachers and the realities of the workplace.     

Recommendations for the use of e‐tools for improvements around assignment marking quality

This article reports on selected aspects of a larger study on the use of electronic tools in the context of the management and marking of assignments. The study comprised a literature review, interviews and a review of e‐tools. The article briefly summarises the findings from the literature on what comprises quality in assignment marking. The focus then shifts towards the analysis of the interviews on e‐tool use for assignments that were conducted with 90 lecturers across five higher education institutions in New Zealand, selected purposively for engagement in this area. The article shows how these lecturers are using e‐tools to support the management and marking of assignments. Special emphasis is given on grounding the e‐tool use in educational theories on assignment marking. An important issue with high quality assignment marking is the time required. The article shows how e‐tools are used to increase efficiency, with the time freed‐up being invested into quality improvements, again supported by e‐tools.     

Doctoral Studies and Public Policies Affecting the Labour Market for Ph.D.s in Norway and Sweden

If the number of doctorates were only a function of the number of professors, Sweden should be expected to excel Norway by a factor of 1.5‐2. This figure is actually 8‐10 times as high for those faculties in which Sweden requires a Ph.D. for high‐school lecturers. Within Sweden, there is also a systematic covariation over time between the examination of doctors and the number of new positions filled as lecturers. Thus, the merit‐value of doctorates for Swedish higher education has been of great importance for the development of Swedish research in certain fields.     

Students' response to traditional and computer-assisted formative feedback: A comparative case study

The national movement towards progress files, incorporating personal development planning and reflective learning, is supported by lecturers providing effective feedback to their students. Recent technological advances mean that higher education tutors are no longer obliged to return comments in the ‘traditional’ manner, by annotating students' work with red pen. This paper considers some of the options currently available for returning computer‐assisted feedback, including the Electronic Feedback freeware. This MS Office application enables tutors to readily synthesise and email feedback reports to students. To further ascertain the value of this software, 169 1st‐year Pharmaceutical Science and Pharmacy students completed a questionnaire to gauge their reaction to formative feedback on an extended laboratory report. This included 110 responses from students graded by three tutors who marked work using either handwritten annotations or the Electronic Feedback program. Principle component analysis (PCA) of the Likert scale responses indicates that the identity of the marker did not significantly affect the response of students. However, the type of feedback was a factor that influenced the students' responses, with electronic feedback being rated superior. A Mann‐Whitney analysis of the satisfaction ratings (generated by PCA) indicates that four features of the assignment and feedback were considered significantly improved when the software was used to create feedback, namely; markscheme clarity, feedback legibility, information on deficient aspects, and identification of those parts of the work where the student did well. Modern academics face a number of challenges if they wish to return meaningful and timely feedback to students, among them large class sizes and infrequent face‐to‐face contact. It is pleasing to note, therefore, that assessors reported taking less time to mark when using the software. It is concluded that electronic formative feedback can be returned more quickly and may be used to synthesise relevant feedback that is both fair and balanced.     

Social constructivist teaching methods in Australian universities – reported uptake and perceived learning effects: a survey of lecturers

Pragmatic social constructivist teaching methods require students to construct knowledge by engaging collaboratively with realistic problems, cases or projects. It is hypothesised that they are more effective than traditional didactic teaching methods in developing undergraduate students’: (1) theoretical knowledge; (2) profession‐specific skills; and (3) knowledge creation capacity. Results of a survey show the second and third learning effects to be salient among Australian university lecturers, but not the first. Lecturers report that these teaching methods have been adopted more widely in human service‐related faculties and design‐related faculties than in business‐related faculties, possibly owing to the lesser emphasis placed by business lecturers on developing students’ profession‐specific skills and knowledge creation capacity. A corresponding survey of business practitioners revealed a surprising gap between the value that business practitioners place on new graduates’ knowledge creation capacity and the rather limited emphasis that business lecturers place upon developing that capacity in their undergraduate students.     

Improving pre‐service teacher learning through peer teaching: process,people and product1

This research investigates the perceptions of first‐year Bachelor of Teaching students (primary and secondary) and Diploma of Education students (secondary) about their peer teaching experience in a postgraduate subject called Curriculum and Assessment. Peer teaching is a learner‐centred approach to teaching and learning that is intended to provide significant benefits for learners' knowledge, skills and metacognition. However, concerns have been raised over the quality of the learning and teaching and the risks associated with such a pedagogy. In the present study, student responses to questionnaires and semi‐structured interviews were analysed, using a mixed methods approach, with respect to three broad and somewhat interconnected categories: process, people and product. These responses suggested a wide range of reactions to peer teaching, but overall students feel they benefited from the experience. The findings of this study should be of interest to lecturers and students in pre‐service teacher education courses, especially. Knowledge about peer teaching, learning and assessment would be especially valuable for both education lecturers and beginning teachers seeking to design and manage learner‐centred pedagogy in their own primary, secondary and tertiary classrooms. However, the results of this research would have far‐reaching appeal for all teaching and learning contexts.     

The susceptibility of student evaluation of lecturers to situational variables

To investigate the effect of a number of situational factors on student evaluations of lecturers, a teacher rating form (TRF) was administered to 326 first-year students taking an Introduction to Economics course. A comparison was made of the responses of the following subgroups among the students: full-time versus part-time students; students asked to sign their responses versus those whose responses were left anonymous; students told that their responses were just for feedback to the lecturer versus those told that their responses would be used for promotion, tenure, and salary decisions about the lecturer; students told that the TRFs were sponsored by the student association versus those told that the TRFs were sponsored by the staff association. The factors of sponsor identity (staff or student association) and use of ratings (feedback or decisional) had significant effects on TRF scores; but the effects of the factors of anonymity and full- versus part-time were insignificant. These results are taken as suggestive evidence that conditions of administration can modify student responses to TRFs and that they should be as uniform as possible if TRFs are to be used to compare lecturers.     

The relative importance of teaching evaluation criteria from the points of view of students and faculty

Abstract

The student evaluation of teaching (SET) tool is widely used to measure student satisfaction in institutions of higher education. A SET typically includes several criteria, which are assigned equal weights. The motivation for this research is to examine student and lecturer perceptions and the behaviour of the students (i.e. ratings given by them to lecturers) of various criteria on a SET. To this end, an analytic hierarchy process methodology was used to capture the importance (weights) of SET criteria from the points of view of students and lecturers; the students' actual ratings on the SET were then analysed. Results revealed statistically significant differences in the weights of the SET criteria; those weights differ for students and lecturers. However, analysis of 1436 SET forms of the same population revealed that, although students typically rate instructors very similarly on all criteria, they rate instructors higher on the criteria that are more important to them. The practical implications of this research is the reduction of the number of criteria on the SETs used for personnel decisions, while identifying for instructors and administrators those criteria that are perceived by students to be more important.     

ESTABLISHING THE VALIDITY OF A SET OF SUMMATIVE TEACHING PERFORMANCE SCALES

A policy at a New Zealand University to assess lecturers’ teaching performance as an administrative requirement provided for information to be obtained from the chairperson, colleagues, students and the individual being evaluated.

In designing the summative scales to obtain student feedback, four conditions were seen as important for achieving a system acceptable to both lecturers and students. These conditions—length of scale, time to complete the scale, face validity of the scale and the acceptability of the procedures used to distribute the scales, together with the context within which teaching takes place at the Institution, provided constraints to the design of the summative scales.

Following 98 lecturing and 43 course evaluations, the scales were subjected to a principal factor analysis. This analysis showed that the items most strongly related to overall teaching effectiveness, as assessed by students, included the lecturer's ability to communicate ideas and information and to stimulate interest in the subject.

By pooling student ratings from different teaching situations for any one lecture, a Teaching Performance Profile was prepared. This profile increased the reliability and validity of the information obtained. At the same time it provided a summary suitable for combining with the evaluative . information from the other three sources in making a judgement as to the individual's level of teaching competency.     

SELF AND STUDENT EVALUATION OF ACADEMIC STAFF

At the University of Queensland a questionnaire to students for their evaluation of tutors’ and lecturers’ classroom presentation and classroom management has been in (increasing) use since 1982. The introduction of the evaluation questionnaire and the management of the evaluation system are based on the research lieterature on change. Student evaluations per se do not induce change. However, self evaluations focus staffs attention on their own perception as teachers, and possible discrepancies between self and student evaluation may then motivate staff to change.

Therefore self evaluation has been used in the evaluation of teaching schemesto facilitate change. Overall, there was no relationship between student ratings and staff self ratings on the question, “All things considered, how would you rate this staff member's overall effectiveness as a university teacher?” Both highly and poorly rated lecturers showed large discrepancies between their self perception and student perception. This emphasises the importance of using more than one source of evaluative information for decision makeing. An interview study found that nearly all those evaluated had implemented changes and felt positive about evaluation.     

An illustration to assist in comparing and remembering several multiplicity adjustment methods

There are many well‐known or new methods to adjust statistical tests for multiplicity. This article provides an illustration helping lecturers or consultants to remember the differences of three important multiplicity adjustment methods and to explain them to non‐statisticians.     

The influence of gender composition in a field of study on students' drop‐out of higher education

Combining Tinto's classical model of student drop‐out with Kanter's assessment of minorities, this article examines the influence of gender composition in a field of study on drop‐out from higher education. Our empirical analysis is based on a sample of students who left German higher education in 2014. Our results confirm previous findings that women in gender‐atypical subjects show a higher drop‐out risk than their male fellow students. We assess several mechanisms which could contribute to explain this effect. Contrary to our expectations, social integration, in the sense of contact with lecturers, seems to be a protective factor for women and men in gender‐atypical subjects. For women in gender‐atypical fields of study, contact with peers is an additional protective factor against drop‐out. The most important mechanism to explain higher education drop‐out is women's more negative self‐assessment of their suitability for male‐dominated subjects.     

Understanding similarities and differences between parents' and teachers' construal of children's behaviour

This study compares and contrasts teachers' and parents' construal of children's behaviour, drawn from the general population rather than clinical extremes. Questionnaire data were obtained from parents and teachers on the same 3–7 year old children via nurseries and mainstream schools. Item reduction followed standard psychometric approaches. Varimax‐rotated principal components analysis was undertaken on dichotomised teacher and parent responses, giving a four‐factor solution as most appropriate both for teachers and parents (aggression; anxiety; context‐directed behaviour; social maturity/communicativeness). The low correlation found between parents and teachers on corresponding factors suggests strong divergence of perspectives and/or situation‐dependence of behaviours. Reliability was high enough to not provide the explanation – i.e., serial correlations over three months within each respondent type were relatively strong. However, slight changes also took place over time: teacher ratings of anxiety differentiated out into anxiety proper and social maturity, while ratings of aggression and non‐context‐directed (‘inappropriate’) behaviour integrated towards a single factor which may reflect “threat to classroom order”. Implications for parent–teacher communication are discussed.     

Assessing Early Language Development in Children with Vision Disability and Motor Disability

This article describes a method for identifying test items as disability neutral for children with vision and motor disabilities. Graduate students rated 130 items of the Preschool Language Scale and obtained inter‐rater correlation coefficients of 0.58 for ratings of items as disability neutral for children with vision disability, and 0.77 for ratings of items as disability neutral for children with motor disability. These ratings were used to create three item sets considered disability neutral for children with vision disability, motor disability, or both disabilities. Two methods for scoring the item sets were identified: scoring each set as a partially administered developmental test, or computing standard scores based upon pro‐rated raw score totals. The pro‐rated raw score method generated standard scores that were significantly inflated and therefore less useful for the assessment purposes than the ratio quotient method. This research provides a test accommodation technique for assessing children with multiple disabilities.     

The impact of individual consultation on student ratings of teaching

Abstract

The present study addressed the impact of individual consultation on teaching improvement as measured by changes in student ratings. Subjects included 91 professors who presented naturally for individual consultation services over a seven‐year period at the teaching centre of a Canadian university. Interventions by the consultant fell into three categories: 1) Feedback‐Consultation, 2) Feedback‐Consultation‐Class Observation, and 3) Feedback‐Consultation‐Class Observation and Student Consultation. End of term student ratings for the course that was the subject of the consultation were compared with student ratings for the same course taught between one and three years prior to the consultation service, and for the same course taught between one to three years following consultation. The results showed that, overall, consultation was effective in improving the quality of the consultees’ teaching, as evidenced by an increase in mean student ratings of instruction. This effect persisted post consultation. Not all intervention groups, however, showed the same pattern of results. Change was evident immediately after the intervention except in the case of brief consultation, although follow‐up data showed improved teaching for the latter group. Control data provided evidence that the change in student ratings post consultation could reasonably be attributed to consultation effects.