The effect of incentives and other instructor-driven strategies to increase online student evaluation response rates

Course evaluations (often termed student evaluations of teaching or SETs) are pervasive in higher education. As SETs increasingly shift from pencil-and-paper to online, concerns grow over the lower response rates that typically accompany online SETs. This study of online SET response rates examined data from 678 faculty respondents and student response rates from an entire semester. The analysis focused on those tactics that faculty employ to raise response rates for their courses, and explored instructor and course characteristics as contributing factors. A comprehensive regression model was evaluated to determine the most effective tactics and characteristics. Using incentives had the most impact on response rates. Other effective tactics that increase response rates include reminding students to take the evaluation, explaining how the evaluations would be used to improve instruction, sending personal emails and posting reminders on Blackboard^®. Incentives are not widely used; however, findings suggest that non-point incentives work as well as point-based ones, as do simple-to-administer minimum class-wide response rate expectations (compared to individual completion).     

Understanding the dynamics of motivation in socially shared learning

The aim of this study was to gain an understanding of the dynamics of motivation in socially shared learning from both individual and group perspectives. Higher education students’ motivation was analysed in the context of collaborative learning tasks, applying quantitative and qualitative methods. The research questions were: (1) what kind of motivations and task-specific goals do individual students have and (2) how do they combine their motivations as a group in socially shared learning? Three- to five-member groups of university students (n = 99) were observed as they worked with three different collaborative learning tasks in an educational psychology class. The experiment included differently organised conditions for collaboration, ranging from face-to-face situations to virtual collaboration situations which were perceived as motivationally and emotionally challenging. Self-reports and video-tapings were collected. The results show that the students who studied in face-to-face settings reported significantly more learning goals and fewer performance goals during the collaborative tasks than the students in the virtual group. Therefore, the collaborative process of volunteer face-to-face groups was analysed qualitatively and an attempt was made, by observing their shared motivation, to uncover the reasons why they achieved their learning goals.     

Expectancy theory outcomes and student evaluations of teaching

As student evaluation of teaching (SET) instruments are increasingly administered online, research has found that the response rates have dropped significantly. Validity concerns have necessitated research that explores student motivation for completing SETs. This study uses Vroom's [(1964). Work and motivation (3rd ed.). New York, NY: John Wiley & Sons] expectancy theory to frame student focus group responses regarding their motivations for completing and not completing paper and online SETs. Results show that students consider the following outcomes when deciding whether to complete SETs: (a) course improvement, (b) appropriate instructor tenure and promotion, (c) accurate instructor ratings are available to students, (d) spending reasonable amount of time on SETs, (e) retaining anonymity, (f) avoiding social scrutiny, (g) earning points and releasing grades, and (h) being a good university citizen. Results show that the lower online response rate is largely due to students’ differing feelings of obligation in the 2 formats. Students also noted that in certain situations, students often answer SETs insincerely.     

The Effects of Individual and Group Monetary Incentives on High Performance

This study examined whether high performers performed better (a) under individual and group incentive pay than under hourly pay and (b) under individual incentive pay than under group incentive pay. Participants were 11 college students. An ABCDC within‐subject design was used in which A was hourly pay with individual feedback, B was individual incentives with individual feedback, C was individual incentives with individual and group feedback, and D was group incentives with individual and group feedback. Participants performed better under both individual and group incentive pay than under hourly pay (p<.01). They did not significantly decrease performance when switched from individual to group incentives, but increased their performance when switched back to individual incentives (p<.01). Individual incentive pay was rated as the most preferred and most satisfying of the three pay systems, and group incentive pay as the least preferred, least satisfying and most stressful. These data suggest that high performers perform better under individual than group incentive pay after exposure to both and are highly likely to prefer individual incentive pay.     

Exploring graduate and undergraduate course evaluations administered on paper and online: a case study

The purpose of this study was to analyse the students’ evaluations of the course and instructor for all statistics courses offered during fall semester 2009 at a large university in the southern United States. Data were collected and analysed for course evaluations administered both online and on paper to students in both undergraduate and graduate courses. Unlike most previous studies on this subject, class section rather than student was treated as the unit of analysis. It was of specific interest to verify prior research findings that evaluation surveys administered online would not result in lower course and instructor ratings and lower response rates. The results showed that there is not sufficient evidence within the collected data to conclude that either course and instructor ratings or response rates are lower for evaluations administered online (online evaluations) than they are for evaluations administered on paper (paper evaluations). Of secondary interest was whether class ratings would be associated with students’ attendance and a comparison of variability among answers for undergraduate vs. graduate students. It was observed that class and teacher ratings were not related to students’ attendance and individual students did not tend to give the same answer for every question on their survey.     

Differentiations in motivation and need satisfaction based on course modality: a self-determination theory perspective

Online education has grown exponentially over the past two decades, in large part due to its promise of flexibility and connectivity for students. However, this approach to pedagogy has remained relatively unexamined in regard to issues of motivation and intellectual thriving. Using self-determination theory as a foundation, we assessed the degree to which course modality (namely online vs. face-to-face) led to psychological need satisfaction and quality motivation. Our survey of 240 (n = 240) college students confirmed previous research in which higher quality motivation predicted the satisfaction of autonomy, competence and relatedness, which in turn predicted course and instructor approval. However, in a series of matched-pairs t-tests, students reported lower levels of quality motivation, autonomy support, competence and relatedness in online courses than they did for face-to-face courses.     

A pedagogical alliance for trust,wellbeing and the identification of errors for learning and formative assessment

Formative assessments and feedback are vital to enhancing learning outcomes but require that learners feel at ease identifying their errors, and receiving feedback from a trusted source – teachers. An experimental test of a new theoretical framework was conducted to cultivate a pedagogical alliance to enhance students’ (a) trust in the teacher, (b) well-being in the learning environment and (c) identification of confusion and errors for the purpose of learning, assessment and feedback. A sample of 101 undergraduate students was randomly assigned to either an intervention (n = 51) or control (n = 50) condition in Elementary Statistics. Results indicated that a pedagogical alliance could be created to enhance student trust in the instructor, leading students to report greater well-being and a higher number of potential areas of confusion in their understanding of new content material relative to a control group. These results have implications for formative feedback, assessments, and by extension learning outcomes.     

Mixed-methods exploration of students' motivation in using augmented reality in neuroanatomy education with prosected specimens

The use of augmented reality (AR) in teaching and studying neuroanatomy has been well researched. Previous research showed that AR-based learning of neuroanatomy has both alleviated cognitive load and was attractive to young learners. However, how the attractiveness of AR effects student motivation has not been discovered. Therefore, the motivational effects of AR were investigated in this research by the use of quantitative and qualitative methods. Motivation elicited by the GreyMapp-AR, an AR application, was investigated in medical and biomedical sciences students (n = 222; mean age: 19.7 ± 1.4 years) using the instructional measure of motivation survey (IMMS). Additional components (i.e., attention, relevance, confidence, and satisfaction) were also evaluated with motivation as measured by IMMS. Additionally, 19 students underwent audio-recorded individual interviews which were transcribed for qualitative analysis. Males regarded the relevance of AR significantly higher than females (P < 0.024). Appreciation of the GreyMapp-AR program was found to be significantly higher in students studying biomedical sciences as compared to students studying medicine (P < 0.011). Other components and scores did not show significant differences between student groups. Students expressed that AR was beneficial in increasing their motivation to study subcortical structures, and that AR could be helpful and motivating for preparing an anatomy examination. This study suggests that students are motivated to study neuroanatomy by the use of AR, although the components that make up their individual motivation can differ significantly between groups of students.     

Instructional practices in reformed undergraduate STEM learning environments: a study of instructor and student behaviors in biology courses

ABSTRACT

Theoretically, organisational culture, instructor training, and learning space design influence how faculty teach STEM courses. Previous studies have used classroom observation protocols to characterise the range of teaching practices in mostly teacher-centered, traditional STEM classrooms. In this study, we examined the classroom behaviour of 13 STEM faculty teaching biology courses in a reformed undergraduate STEM learning environment. Our findings indicate that instructors teaching in this reformed environment guided student learning (58.4?±?1.9%) almost three times more than they presented information (20.0?±?2.2%). Students worked individually or in groups and talked to the whole class (57.1?±?1.8%) 1.5 times more than they received information (35.5?±?1.9%). We found significant positive correlation between ‘instructor presenting’ and ‘students receiving’ information (r?=?0.743, p?=?1.4?×?10^?4) and ‘instructor guiding’ and ‘student working and talking’ in class (r?=?0.605, p?=?7.2?×?10^?5), suggesting that instructors can change their own classroom behaviours and expect concurrent change in their students’ behaviours. Finally, sequencing teaching practices in high active-engagement classrooms showed instructors move and guide student group work and lead whole class discussions before lecturing to students, which could lead to deeper learning of conceptual knowledge. We discuss insights from these findings that have implications for acculturating evidence-based teaching practices in STEM departments.     

Efficacy of Supplemental Phonics-Based Instruction for Low-Skilled First Graders: How Language Minority Status and Pretest Characteristics Moderate Treatment Response

We examined the efficacy of 20 weeks of individual supplemental phonics-based instruction for language minority (LM) and non-LM first graders. Students were designated LM if the primary home language was not English (otherwise non-LM). Those performing in the bottom half of their classroom LM/non-LM group in letter knowledge and phonological awareness were randomly assigned to treatment and control conditions. Treatment included alphabetics, decoding, and oral reading practice. Results showed that treatment students (n?=?93) outperformed controls (n?=?94) on 5 of the 6 posttests; however, LM students exhibited lower treatment response on passage reading fluency. Pretest word reading did not moderate treatment response, and LM students with greater baseline vocabulary showed greater treatment response on posttest word reading and spelling.     

ADHD Coaching With College Students: Exploring the Processes Involved in Motivation and Goal Completion

College students with attention-deficit/hyperactivity disorder (ADHD) often experience increased academic difficulties, which can negatively impact graduation rates, employment, self-esteem, and mental health. ADHD coaching assists students with ADHD to reduce such difficulties. The present study evaluated the processes involved in ADHD coaching by examining undergraduate and graduate level students (n = 23) who attended individualized ADHD coaching sessions for 8 weeks. Specifically, the study examined the types of goals established, the use of weekly objectives (tasks to be completed), barriers to task completion, the use of incentives and consequences to increase motivation, and the role of task enjoyment and therapeutic benefit on task completion. The most common goals involved time management and academic performance. Results showed a significant effect for use of incentives/consequences on weekly task completion. Therapist-coach ratings were predictive of task completion, while client ratings were not. The primary barriers to task completion were lack of motivation and poor time management.     

Individual and Triadic and Group: Supervisee and Supervisor Perceptions of Each Modality

In this consensual qualitative research study, the authors explored supervisors' (n= 11) and their supervisees' (n= 31) perceptions of individual, triadic, and group supervision sessions during practicum. Data from supervisor individual interviews and supervisee focus‐group interviews revealed several themes regarding the advantages and disadvantages of each supervision modality. Findings suggest the relative place of each modality in terms of goals and impact.     

Multimedia Effect in Problem Solving: A Meta-Analysis

Pictures are commonly used to represent problems. However, it is unclear how the addition of pictures affects students’ problem-solving performance. The multimedia effect in problem solving describes the phenomenon whereby an individual’s problem-solving performance is enhanced when equivalent pictures are added to illustrate or replace part of the problem text. Using meta-analytic techniques, this study sought to determine the overall size of the multimedia effect in problem solving and the possible boundary conditions (k = 51; N = 38,987; Range n = 10 – 31,842; Median n = 63). The results showed a significant small-to-medium multimedia effect size on response accuracy (Hedges’s g = 0.32) and a significant medium-to-large multimedia effect size on students’ response certainty (Hedges’s g = 0.74), but no significant multimedia effect on response time. The results for the effects of decorative pictures were not sufficient for a reliable interpretation. Representational (Hedges’s g = 0.24) and organizational (Hedges’s g = 0.52) pictures had a significant and positive impact on response accuracy, but informational or multiple pictures across studies did not have a significant aggregate effect on an individual’s response accuracy. These findings suggest that the multimedia effect in problem solving is diverse and limited by multiple boundary conditions. Further primary studies are needed to further investigate the multimedia effect in problem solving.

     

It's more about a lesson than a domain: Lesson-specific autonomy support,motivation, and engagement in math and a second language

Expanding research on the relative impact of different autonomy-supportive strategies employed by teachers across domains, the present study investigated the variation in 4 lesson-specific autonomy-supportive strategies (providing choices, rationales, accepting frustration, and stimulating interests) and 6 aspects of students' motivation and engagement in 2 domains with a repeated measurement design. For 3 weeks, 202 Dutch students from 8 eighth grade classes and 1 ninth-grade class and 12 teachers completed lesson-specific measures at the end of Math and German lessons. Students' perceptions of teachers' autonomy support and their motivation and engagement varied considerably across lessons within a domain (variance at the within-student level ranged from 19% to 51%). In random intercept-random slope models, we found that all autonomy-supportive strategies showed meaningful associations with aspects of students’ motivation and engagement. We did not find substantial domain-dependency in the associations between autonomy support and the outcomes.     

Investigating the effects of gamification-enhanced flipped learning on undergraduate students’ behavioral and cognitive engagement

ABSTRACT

In recent years, flipped learning has attracted much attention around the world. This instructional approach is appealing because it can free up class time for knowledge application activities with help from the instructor and peers. However, its implementation can be fraught with challenges. Student disengagement in out-of-class activities, for example, is one of the major challenges of flipped learning. The purpose of this study is to examine whether gamification can enhance student engagement in a flipped course. A comparison study was conducted, involving two classes of undergraduate students in an Information Management course. The results indicated that students in the gamification-enhanced flipped learning group (n?=?48) were more likely to complete the pre-class and post-class activities on time than those in the non-gamified flipped learning group (n?=?48). Students in the gamification-enhanced flipped learning group also produced higher quality artifacts than the non-gamified flipped learning group in the pre-class thinking activities. Moreover, students in the gamification-enhanced flipped learning group scored significantly higher in the post-course test than did their non-gamified counterparts.     

Maintaining motivation and regulating emotion: measuring individual differences in academic volitional strategies

This study examined self-regulatory strategies used by students to maintain motivation on academic goal tasks. Of particular interest were strategies for managing potentially disruptive emotional states. The Academic Volitional Strategy Inventory (AVSI) was developed to investigate this aspect of student self-regulatory behavior. Three separate student samples (n = 378; n = 463; n = 246) from a large southwestern university contributed to the refinement of the AVSI, and supported theoretical arguments for the existence of volitional self-regulatory action by students in maintaining academic task motivation. Factor analyses performed on items for each administration of the AVSI revealed a three-factor structure as providing a conceptually clear division of items. Item groupings consisted of strategies reflecting (1) self-efficacy enhancement, (2) stress reduction, and (3) negative-based incentives. Reliability estimates revealed strong internal consistency and stability. Further analyses currently in progress to support the validity of the scale are also discussed. However, the preliminary results presented in this paper suggest that the AVSI is a promising research instrument, and one that captures an aspect of self-regulatory action not adequately examined by existing instruments.     

Marketing students at work: organisation citizenship behaviour training as a tool to minimise survey non-response

A continuing decline in an institution’s response rates for student evaluations of teaching (SET) raised faculty concerns about non-response bias in summary statistics. In response, this institution’s SET stakeholders partnered with a Marketing Methods class section to create strategies for increasing response rates. The project was also an exercise in organisational citizenship behaviour (OCB) training because students in that class section received intensive training on how SET feedback is valued by instructors and its role in improving their academic organisation. Within the context of OCB theory, this article finds student exposure to OCB training increases SET response rates because knowing how SET benefits their organisation increases unit-level response propensity for member surveys intended to improve their institution. In the year of the OCB training, SET response rates increased by 26%, though the increases did not persist into later academic years. The response rate increases are realised across all demographic groups with disproportionate increases among low response rate groups, including low performing students, men and ethnic minorities.     

Student evaluations of teaching: the impact of faculty procedures on response rates

In the last 10–15 years, many institutions of higher education have switched from paper-and-pencil methods to online methods of administering student evaluations of teaching (SETs). One consequence has been a significant reduction in the response rates to such instruments. The current study was conducted to identify whether offering in-class time to students to complete online SETs would increase response rates. A quasi-experiment (nonequivalent group design) was conducted in which one group of tenured faculty instructed students to bring electronic devices with internet capabilities on a specified day and offered in-class time to students to complete online SETs. A communication protocol for faculty members’ use was developed and implemented. A comparison group of tenured faculty who did not offer in-class time for SET completion was identified and the difference-in-differences method was used to compare the previous year’s response rates for the same instructor teaching the same course across the two groups. Response rates were substantially higher when faculty provided in-class time to students to complete SETs. These results indicate that high response rates can be obtained for online SETs submitted by students in face-to-face classes if faculty communicate the importance of SETs in both their words and actions.     

Individual differences and course attendance: why do students skip class?

Individual differences are theorised as being key predictors of students’ lecture attendance. Understanding students’ (lack of) motivation for studying is an essential first step in supporting these individual differences and potentially increasing class/lecture attendance. The current study explored the role of students’ deficits in motivations (ability/effort/task value) for studying and their predictive relationship with course attendance, while controlling for prior competence and academic self-concept. Second-year students (n = 219; Female = 40) undertaking three semester-long courses in English as a foreign language participated in the year-long study. The structural model tested included prior achievement and self-concept (Time-1; 15 weeks prior), the motivational deficits for studying in the course (Time-2; ability, effort and task value), and class attendance across the two semesters of study. Self-concept negatively predicted all motivational deficits (moderate-large effects), and positively predicted attendance. Two of the motivational deficits (ability and effort) also had strongly contrasting effects (positive and negative respectively) on attendance. The implications and practical suggestions for intervention are discussed.