Comparison of online and classroom-based student evaluations of instruction

In recent years, colleges have been moving from traditional, classroom‐based student evaluations of instruction to online evaluations. Because of the importance of these evaluations in decisions regarding retention, promotion and tenure, instructors are justifiably concerned about how this trend might affect their ratings. We recruited faculty members who were teaching two or more sections of the same course in a single semester and assigned at least one section to receive online evaluations and the other section(s) to receive classroom evaluations. We hypothesised that the online evaluations would yield a lower response rate than the classroom administration. We also predicted that there would be no significant differences in the overall ratings, the number of written comments, and the valence (positive/neutral/negative) of students’ comments. A total of 32 instructors participated in the study over two semesters, providing evaluation data from 2057 students. As expected, online evaluations had a significantly lower response rate than classroom evaluations. Additionally, there were no differences in the mean ratings, the percentage of students who provided written comments or the proportion of comments in the three valence categories. Thus, even with the lower response rate for online evaluations, the two administration formats seemed to produce comparable data.     

Student performance in computing education: an empirical analysis of online learning in programming education environments

The benefits of using online exercises have been analysed in terms of distance learning, automatic assessment and self-regulated learning. In this study, we have not found a direct proportional relationship between student performance in the course exercises that use online technologies and the exam grades. We see that the average submission rate to these online exercises is not positively correlated with the exercise points. Yet, our results confirm that doing exercises along supports student learning and skill accumulation equipping them with the knowledge of programming. While the student performance in programming courses is affected by factors such as prior background in programming, cognitive skills and the quality of teaching, completing the course exercises via learning-by-doing is an indispensable part of teaching. Based on the student feedback from the course survey, the students are highly satisfied with using online technologies as part of learning.     

高等学校学生评价教学方式比较研究

通过对高等学校学生评价教师教学的两种方式(纸笔评价和网基评价)的比较研究发现,纸笔评价和网基评价有各自的特点,网基评价能给评价组织者在评价数据的收集和处理方面带来极大的便利,并逐步被越来越多的高校所采用,但要注意控制评价过程中的过程误差对评价结果的影响.     

Updating RIGs: including the systematic influence of online study on student evaluation of teaching

Based on student evaluation of teaching (SET) ratings from 1,432 units of study over a period of a year, representing 74,490 individual sets of ratings, and including a significant number of units offered in wholly online mode, we confirm the significant influence of class size, year level, and discipline area on at least some SET ratings. We also find online mode of offer to significantly influence at least some SET ratings. We reveal both the statistical significance and effect sizes of these influences, and find that the magnitudes of the effect sizes of all factors are small, but potentially cumulative. We also show that the influence of online mode of offer is of the same magnitude as the other 3 factors. These results support and extend the rating interpretation guides (RIGs) model proposed by Neumann and colleagues, and we present a general method for the development of a RIGs system.     

The Development and Validation of the Student Evaluation of Online Teaching Effectiveness

Abstract

Sponsored by a Preparing Tomorrow's Teachers to Use Technology(PT3) grant, we participated in various activities designed to help us learn to infuse technology in our teacher education courses. The purpose of this paper is to describe the specific impact of the PT3 project activities on our change process-including the formation, evolution, and efforts of our curriculum design team-and to share the activities and products that we developed through participation in the grant activities.     

关于加强网上评教结果科学性的几点建议

介绍了学生网上评教与传统纸质问卷调查相比的优点,指出了网上评教存在的问题。最后对完善网上评教体制、建立起科学的网上评教机制提出了几点建议,以此保证网上评教结果的可信性、科学性,同时保护教师改善教学的积极性,最终使网上评教起到提高教学质量的积极作用。     

Student evaluation of teaching and matters of reliability

The student evaluation of teaching process is generally thought to produce reliable results. The consistency is found within class and instructor averages, while a considerable amount of inconsistency exists with individual student responses. This paper reviews these issues along with a detailed examination of common measures of reliability that are utilised with the instruments. While inter-item consistency of the evaluations has been shown to be high, the agreement between students was shown to be no better than what would be expected by chance, indicating that students do not agree on what they are being asked to evaluate. The reliability measures generated by the student evaluations of teaching are an insufficient foundation for establishing validity. Further, the pattern of reliability indicates that the instruments are generally providing information about students, not instructors.     

The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates

Course evaluations (often termed student evaluations of teaching or SETs) are pervasive in higher education. As SETs increasingly shift from pencil-and-paper to online, concerns grow over the lower response rates that typically accompany online SETs. This study of online SET response rates examined data from 678 faculty respondents and student response rates from an entire semester. The analysis focused on those tactics that faculty employ to raise response rates for their courses, and explored instructor and course characteristics as contributing factors. A comprehensive regression model was evaluated to determine the most effective tactics and characteristics. Using incentives had the most impact on response rates. Other effective tactics that increase response rates include reminding students to take the evaluation, explaining how the evaluations would be used to improve instruction, sending personal emails and posting reminders on Blackboard^®. Incentives are not widely used; however, findings suggest that non-point incentives work as well as point-based ones, as do simple-to-administer minimum class-wide response rate expectations (compared to individual completion).     

Applying the scholarship of teaching and learning: student perceptions,behaviours and success online and face-to-face

This study compares student perceptions, learning behaviours and success in online and face-to-face versions of a Principles of Microeconomics course. It follows a Scholarship of Teaching and Learning (SoTL) approach by using a cycle of empirical analysis, reflection and action to improve the learning experience for students. The online course design involves 58 interactive narrated online modules, interactive online quizzes and biweekly online meetings with the instructor via video and voice-over-IP technology. Findings indicate that schedule flexibility motivates students to choose the online course format. Students in both learning environments felt they had high-quality communication with the instructor, while online students indicated limited peer-to-peer communication. Online students report studying more at home than face-to-face students, but not enough to compensate for the time face-to-face students spend in class. Reflecting on the findings, the authors suggest actions to improve the online course experience.     

Student evaluation of instruction in higher education: exploring issues of validity and reliability

Many personnel committees at colleges and universities in the USA use student evaluation of faculty instruction to make decisions regarding tenure, promotion, merit pay or faculty professional development. This study examines the construct validity and internal consistency reliability of the student evaluation of instruction (SEI) used at a large mid‐western university in the USA for both administrative and instructional purposes. The sample consisted of 73,500 completed SEIs for undergraduate students who self‐reported as freshman, sophomore, junior or senior. Confirmatory factor analysis via structural equation modelling was used to explore the construct validity of the SEI instrument. The internal consistency of students' ratings was reported to provide reliability evidence. The results of this study showed that the model fits the data for the sample. The significance of this study as well as areas for further research are discussed.     

Refugees and online education: student perspectives on need and support in the context of (online) higher education

ABSTRACT

Online education and in particular Massive Open Online Courses (MOOCs) are often regarded as a way to solve global educational challenges. In this article, we highlight the students’ uptake of such a ‘digital solution’. Presenting initial findings from a research project in Germany, we situate our investigation in the specific context of digital educational offers for refugees, using Kiron Open Higher Education as an example. Kiron has designed an innovative academic model, with MOOCs at its core, to ease refugees’ access into higher education. Drawing from student data of 1375 Kiron students we look at students’ actual usage of the offer and the accompanying support services as well as the difficulties refugee students face while navigating online higher education. Results show, amongst others, rather low completion rates in the online courses and point to a much more nuanced picture of how students make use of the offer – putting online education as an easy, straightforward formula to the integration of disadvantaged students into question.     

Generalizability of Multidimensional Student Ratings of University Instruction Across Courses and Teachers

Course quality is multifaceted, being determined by instructor, students, and external conditions. Consequently, any attempt at measurement should reflect this diversity, so that stable evaluations can be made that reflect both personal (instructor) and situational (student and external conditions) variables. This study extends previous research by examining the stability of both dimensions across different courses, student populations, and universities. In addition, the sample (N = 692 courses) was drawn from 6 traditional and technical German universities that have a different ethos of student interaction with academic staff than those in many other Western countries. Using the Heidelberg Inventory, it was found that instructor variables were reliable across courses given by the same instructor, but student scales or background variables were less consistent across courses in which the content was identical. It was concluded that the instrument was both reliable and valid for student evaluations of both teaching performance and course quality within a European context.     

Electronic in-class course evaluations and future directions

This study sought to determine whether an evaluator conducting electronic course evaluations in person in a classroom increased response rates. At a large public university in the United States, course evaluations were being conducted electronically, attaining a response rate of <50% over several semesters. At the end of the Spring 2015 at this university, an evaluator entered several classrooms having a total enrollment of 273 students and guided students through the process of completing course evaluations using their mobile devices. A response rate of 78.4% was achieved. The process described in this study provides evidence that response rates can be increased in similar circumstances. In addition, several ways the technology described could be used in the future for course evaluation are discussed.     

Student ratings of college teaching: a comparison of faculty and their students

Based on the review of student ratings myths by Aleamoni (1987, 1999), a survey research design was used to analyse differences between college students' (n = 968) and faculty's (n = 34) perceptions. Generally, students held stronger beliefs in these myths, in that they believed faculty with excellent publication records were better qualified to evaluate teaching and that student ratings on single general items are accurate measures of teaching effectiveness. On the other hand, faculty believed that student ratings were invalid and unreliable. Further examination of student characteristics revealed that male students held stronger beliefs in these myths. Finally, students' beliefs in these myths were correlated with their actual ratings of nine dimensions of the Student Evaluation of Educational Quality. A discussion as well as suggestions for using student ratings is provided.     

Four narratives about online international students: a critical literature review

In the current higher education context, offering online programmes is seen as an effective means to recruit more international students. However, supporting online international students studying at a distance is not a simple task for both universities and tutors. The problem mainly stems from a lack of theoretical understanding of online international students and their learning experiences. The present article, therefore, aims to address the gap, by systematically, yet critically reviewing relevant academic narratives about online international students. Our review reveals four types of narratives presented in the literature, describing online international students in particular ways: unspecified others with a rapid increase in their numbers, specific others with deficits, specific others as pedagogical resources and active participants in international learning communities. We discuss the merits and the drawbacks of each type of narratives for online educators seeking pedagogical suggestions about supporting online international students.     

提高高职院校学生评教信度与效度的思考

学生评教是全面、科学评价教师教学、促进教师教学质量提高的重要手段,搞好学生评教工作,必须努力提高学生评教的信度与效度。提高高职院校学生评教的信度与效度可从建立和完善学生评教制度、做好学生评教的组织过程、合理利用学生评教结果三方面进行。     

Correlation between grade point averages and student evaluation of teaching scores: taking a closer look

One of the most contentious potential sources of bias is whether instructors who give higher grades receive higher ratings from students. We examined the grade point averages (GPAs) and student ratings across 2073 general education religion courses at a large private university. A moderate correlation was found between GPAs and student evaluations of teaching (SETs); however, this global correlation did not hold true for individual teachers and courses. In fact, there was a large variance in the correlations between GPAs and SETs, including some teachers with a negative correlation and a large variance between courses.     

The relationship between student engagement with online content and achievement in a blended learning anatomy course

Blended learning has become increasingly common in higher education. Recent findings suggest that blended learning achieves better student outcomes than traditional face‐to‐face teaching in gross anatomy courses. While face‐to‐face content is perceived as important to learning there is less evidence for the significance of online content in improving student outcomes. Students enrolled in a second‐year anatomy course from the physiotherapy (PT), exercise physiology (EP), and exercise science (ES) programs across two campuses were included (n = 500). A structural equation model was used to evaluate the relationship of prior student ability (represented by grade in prerequisite anatomy course) and final course grade and whether the relationship was mediated by program, campus or engagement with the online elements of the learning management system (LMS; proportion of documents and video segments viewed and number of interactions with discussion forums). PT students obtained higher grades and were more likely to engage with online course materials than EP and ES students. Prerequisite grade made a direct contribution to course final grade (P < 0.001) but was also mediated by engagement with LMS videos and discussion forums (P < 0.001). Student learning outcomes in a blended anatomy course can be predicted the by level of engagement with online content. Anat Sci Educ 11: 471–477. © 2017 American Association of Anatomists.     

Student satisfaction and online teaching

This paper presents an analysis of the response of student satisfaction scores to online delivery for courses at a large Australian university. Taking data from 2653 courses delivered since 2011 we employ a difference-in-differences estimator approach to evaluate the impact of a transition from face-to-face learning to online learning of courses on student satisfaction. We estimate that, on a five-point scale, conversion to online learning lowers course satisfaction by about 0.2 points and instructor satisfaction by about 0.15 points. These correspond to shifts relative to the underlying distributions of about 25–30% of a standard deviation. Some implications of the (slight) relative unpopularity of online learning are discussed.     

Exploring graduate and undergraduate course evaluations administered on paper and online: a case study

The purpose of this study was to analyse the students’ evaluations of the course and instructor for all statistics courses offered during fall semester 2009 at a large university in the southern United States. Data were collected and analysed for course evaluations administered both online and on paper to students in both undergraduate and graduate courses. Unlike most previous studies on this subject, class section rather than student was treated as the unit of analysis. It was of specific interest to verify prior research findings that evaluation surveys administered online would not result in lower course and instructor ratings and lower response rates. The results showed that there is not sufficient evidence within the collected data to conclude that either course and instructor ratings or response rates are lower for evaluations administered online (online evaluations) than they are for evaluations administered on paper (paper evaluations). Of secondary interest was whether class ratings would be associated with students’ attendance and a comparison of variability among answers for undergraduate vs. graduate students. It was observed that class and teacher ratings were not related to students’ attendance and individual students did not tend to give the same answer for every question on their survey.