The effects of affective pedagogical agent in multimedia learning environments: A meta-analysis

Affective pedagogical agent (PA) is an image of a character embedded in multimedia lessons with the ability to influence learners' affective experiences and learning performance. Prior studies on the effects of affective PA have shown inconsistent findings. In this study, we conducted four separate meta-analyses to address whether adding an affective PA to multimedia lessons can increase learners' retention performance, transfer performance, positive emotions, and intrinsic motivation, and to explore several moderators that may have contributed to the inconsistencies of previous studies. The research framework mainly includes introducing the concept of affective PA, reviewing research on the impact of affective PA on learning performance, emotions, and motivation, analyzing the moderators that may affect the effects of affective PA, performing a meta-analysis, and discussing the results based on the findings of the meta-analysis. We found 36 articles met the inclusion criteria. The results of the meta-analysis indicated that affective PA could increase learners’ positive emotions (k = 25, g = 0.26), improve intrinsic motivation (k = 26, g = 0.26), and facilitate learning performance (retention: k = 35, g = 0.26; transfer: k = 45, g = 0.34). Furthermore, moderator analysis found that affective PA characteristics (i.e., appearance, the number of emotional cues, and body movement) and learning materials characteristics (i.e., subject domain, pacing of presentation) moderated the effects of affective PA. We discussed these findings from different theoretical perspectives. In general, affective PA could help students be happier and more motivated to learn in multimedia learning environments.     

Mixed feelings: Profiles of emotions among elementary mathematics students and how they function within a control-value framework

Mixed feelings happen in and outside of the classroom; yet prior research has focused on discrete emotions, essentially ignoring the interaction between emotions. We extend prior person-centered studies of achievement emotions by placing emotions within the Control-Value Theory framework to examine how patterns of emotions mediate the relation between motivation and achievement. We found four profiles of emotion in both fourth (n = 5228) and fifth graders (n = 5299)—two positive profiles, a negative profile, and a mixed emotions profile where frustrated and challenged were the primary emotions. All profiles mediated the relationship between math expectancy and achievement. However, only three of the four emotion profiles mediated the relation between math value and achievement.     

Measuring emotions in students’ learning and performance: The Achievement Emotions Questionnaire (AEQ)

Aside from test anxiety scales, measurement instruments assessing students’ achievement emotions are largely lacking. This article reports on the construction, reliability, internal validity, and external validity of the Achievement Emotions Questionnaire (AEQ) which is designed to assess various achievement emotions experienced by students in academic settings. The instrument contains 24 scales measuring enjoyment, hope, pride, relief, anger, anxiety, shame, hopelessness, and boredom during class, while studying, and when taking tests and exams. Scale construction used a rational-empirical strategy based on Pekrun’s (2006) control-value theory of achievement emotions and prior exploratory research. The instrument was tested in a study using a sample of university students (N = 389). Findings indicate that the scales are reliable, internally valid as demonstrated by confirmatory factor analysis, and externally valid in terms of relationships with students’ control-value appraisals, learning, and academic performance. The results provide further support for the control-value theory and help to elucidate the structure and role of emotions in educational settings. Directions for future research and implications for educational practice are discussed.     

Expectancy-value appraisals and achievement emotions in an online learning environment: Within- and between-person relationships

Motivation and emotion are of critical importance for students' academic learning and achievement. Drawing on Eccles and Wigfield's situated expectancy-value theory and Pekrun's control-value theory, we examined to what extent specific expectancy-value appraisals related to studentsʼ achievement emotions. We collected intensive state data of N = 95 university students over one semester in an online learning environment. Students' appraisals were analyzed on different aggregation levels in a hierarchical design, which accounts for variability within learning situations and between students. Our results corroborated theoretical assumptions that expectancy-value appraisals are positively associated with positive emotions and negatively with negative emotions. However, we found that students experienced positive emotions in learning situations of high intrinsic and utility value, but not in situations of high attainment value. Examining appraisal combinations and discrete emotions, we found that particularly studentsʼ perceived costs moderated the relationship between expectancy and frustration and boredom on the situation level.     

Effect of problem-based learning in pharmacology education: A meta-analysis

A consensus is emerging for meta-analyses regarding a range of selected topics in pharmacology education. This review provides a critical overview of problem-based learning (PBL) in pharmacology education. Based on several inclusion criteria, databases were searched, and 37 controlled trial studies were identified and extracted. The meta-analysis found that PBL had a positive effect on gaining higher theoretical scores (SMD = 6.77, 95% CI [5.23, 8.31], p < 0.00,001) assessed through examinations. The results of questionnaires for students’ feedback showed that PBL was superior to conventional teaching methods in improving students’ outcomes of self-study, learning interest, team spirit, problem solving, analyzing, scope of knowledge, communication, and expression. These results suggest that PBL in pharmacology education is considered superior to traditional lecture-based learning.     

Facilitating flexible learning by replacing classroom time with an online learning environment: A systematic review of blended learning in higher education

Higher education institutions are trying to provide more flexibility and individualization, which is mainly realized through the use of new technologies and implemented in online or blended learning designs. This systematic review aims to investigate the impact of replacing classroom time with an online learning environment. The meta-analysis (k = 21 effect sizes) applied strict inclusion criteria concerning research design, measurement of learning outcomes and implementation of blended learning. The estimated effect size (Hedge's g) was positive, although not significantly different from zero and the confidence interval [-0.13, 0.25], suggesting that overall differences between blended and conventional classroom learning are small, and, at best, very small negative or moderate positive effects are plausible. This means that despite a reduction in classroom time between 30 and 79 per cent, equivalent learning outcomes were found. Consequently, blended learning with reduced classroom time is not systematically more or less effective than conventional classroom learning.     

The effect of feedback on academic achievement in technology-rich learning environments (TREs): A meta-analytic review

This study aimed to investigate the effect of feedback on students' academic achievement in a technology-rich environment through a systematic and quantitative synthesis of the studies conducted over several decades. We focused on three issues: (a) the effectiveness of feedback in enhancing learning performance; (b) possible factors (feedback characteristics and study features) associated with different studies that could have resulted in the inconsistent findings across the studies; and (c) how different types of feedback differed in their effect in enhancing academic achievement. Based on 182 effect sizes extracted from 61 studies, we found that, compared with no feedback condition, feedback had at least a medium effect (g = 0.44, 95%CI [0.324, 0.555]) in enhancing academic achievement, and the effect of explanation feedback was the strongest compared to other types of feedback. The study further revealed that the feedback in blended learning was more effective than that in online learning. Possible explanations and implications of these findings, as well as limitations and future research directions were discussed.     

The myth of contextual interference learning benefit in sports practice: A systematic review and meta-analysis

The contextual interference (CI) model hypothesizes random practice (high CI) leads to inferior acquisition, but superior retention and transfer capabilities compared to blocked practice (low CI). These phenomena are well established in laboratory settings. However, the transfer to applied settings, particularly in sports practice is still under discussion. The aim of this systematic review and meta-analysis is to evaluate the possible generalisability of the CI phenomenon in sports-based contexts with regard to performance outcomes. Up to April 16th, 2022, five electronic databases (PubMed, Web of Science, Scopus, Taylor & Francis, and SciELO) were used to search for relevant studies investigating differences between blocked and random schedules at post-acquisition, retention, and/or transfer tests. Using the PICOS criteria, a total of 933 records were screened. Outcomes evaluating the acquisition, retention, and/or transfer performances in sports skills in healthy participants were included. The quality of the selected studies was scored using the PEDro scale. Thirty-seven studies were selected, of which twenty-seven were of good quality and the remaining ten were rated as fair quality. Out of 205 overall pooled outcomes, only 43 performance outcomes (21%) agreed with the CI phenomenon after the acquisition (18 out of 103), retention (19 out of 84), and/or transfer (6 out of 36) phases. No statistically significant overall difference between blocked and random practice was detected at post-acquisition (effect size (ES) = 0.1, p = 0.154), retention (ES = −0.159, p = 0.141) or transfer testing (ES = −0.243, p = 0.071). The subgroup analysis showed inferior acquisition and superior retention following random practice only in individuals aged 20–24 years (ES = 0.282, p = 0.030 during the acquisition, and ES = −0.405, p = 0.011 during retention), with no difference at transfer testing for this specific age group. No similar significant effects have been concurrently identified during both acquisition and retention phases in any of the remaining subgroups (e.g., examined based on experience level, sports, skills, and testing protocol categories). The present results suggest CI effects can only be confirmed under very limited conditions, which seriously challenges the extension of CI effects to the sport context in general. Problematic conclusions for children's learning are discussed, as well as future research strategies to better understand counterintuitive learning approaches.     

Examining research on the impact of distance and online learning: A second-order meta-analysis study

Distance learning has evolved over many generations into its newest form of what we commonly label as online learning. In this second-order meta-analysis, we analyze 19 first-order meta-analyses to examine the impact of distance learning and the special case of online learning on students’ cognitive, affective and behavioral outcomes. We examine to what extent distance learning generation level, and instructional setting moderate the influences of distance learning on cognitive, affective and behavioral outcomes. This second-order meta-analyses also analyzes the first-order meta-analyses for methodological quality and robustness. The findings revealed a statistically significant overall average effect size (g = 0.156, p < .001 with a 95% confidence interval of 0.087–0.224) of distance learning impacting cognitive, affective and behavioral outcomes in comparison to face-to-face learning. Meta-analyses on higher education had a statistically significant larger effect size than K-12 education. Limitations, implications, and directions for future research are discussed.     

The cohesion of small groups in technology-mediated learning environments: A systematic literature review

The development of group cohesion for small groups in face-to-face educational learning settings has been widely studied for decades. The calibre of group cohesion within these contexts is typically associated with increases in performance, resulting in a productive collaboration that helps drive knowledge building and innovation within a learning environment. However, understanding and measuring cohesion in technology-mediated learning experiences still requires additional research. This paper reports on a systematic literature review of small groups in technology-mediated learning settings to explore the definitions of cohesion and the methodologies used for the measurements that have been identified arising in this area. Furthermore, the review provides a characterisation of the factors that impact collaborative learning in computer-supported environments, highlighting two important aspects: i) there is a significant gap in measuring cohesion using unobtrusive methods, such as relying on log or trace data and ii) there is a lack of longitudinal approaches to understand the emergence of cohesion. Finally, we discuss implications for future research in studying group cohesion.     

Evaluating student engagement and experiential learning in global classrooms: A qualitative case study

The global classroom is an emerging technology-based pedagogy used internationally by educational institutions. To evaluate a global classroom, we conducted a qualitative study using written reflections and semi-structured interviews of global classroom participants, based on two theoretical frameworks: Kearsley and Shneiderman’s engagement theory and Kolb’s experiential learning theory. We analyzed student reflections and transcribed interviews, using the software package, NVivo, with two objectives: (1) to evaluate if global classroom is engaging and experiential to students and (2) to elucidate how student engagement is fostered in the global classroom through experiential learning. Results illustrated a complex relationship between student engagement and experiential learning. During the experiential learning cycle, engagement theory (relate-create-donate) principles contributed to student engagement at varying levels and for different purposes. Based on the results of this study, we created a framework that demonstrates the interactivity of engagement theory and experiential learning theory to describe how student engagement featured in experiential learning during this global classroom, with strategies to maximize student engagement in experiential learning.     

The effects of robot-assisted language learning: A meta-analysis

Given the importance of learner–interlocutor interactions, social robots have received considerable attention in the field of language learning, as this technology can physically accompany learners, is equipped with common social behaviors, and can effectively communicate with learners for language acquisition. In line with the continuous efforts to examine the effects of using social robots in language learning, this study is a meta-analysis aiming to integrate the findings of previous experimental studies on robot-assisted language learning (RALL), focusing on language learning achievement. To this end, we selected 12 independent samples (N = 522) from the initial pool of 16 empirical studies—including 23 independent samples (N = 1147)—and computed effect sizes along with their standard errors. The results indicated that RALL had a medium-sized average effect (d = 0.59, SE = 0.09, 95% CI: 0.41–0.76) compared to non-RALL conditions, such as traditional learning conditions without technology (e.g., teacher-only) or other technology-mediated conditions (e.g., virtual agents and tablet PCs). We further identified several moderator variables (i.e., age group, target language, language domain, robots’ role, interaction type, type of non-RALL condition), and provided an in-depth discussion on how these variables were related to the effectiveness of using robots for language learning.     

Effectiveness of grade retention: A systematic review and meta-analysis

Research on the effectiveness of grade retention has a long history, yet, has seen an upsurge during the last decade. In this study, we review 84 recent, methodologically sound studies estimating effects of retention in grades K-12 on repeaters' and nonrepeaters' development, in a variety of countries across the world, disentangling grade and age comparison results. Based on vote counting analysis and three-level metaregression analysis we find grade retention to have an average zero effect, indicating that repeaters and non-repeaters seem to show a similar development, on average. At the same time, we find grade retention effects to differ according to some specific effect and study characteristics. More specifically, grade retention seems less effective in countries applying a mixture of grade retention and tracking to tackle student heterogeneity, and when repeaters are compared with non-repeaters of the same age. Conversely, grade retention seems more effective in countries using strategies such as ability grouping, setting, and streaming to deal with student heterogeneity. Positive effects also seem to arise when studying students' psychosocial functioning, when investigating short-run effects, when comparing repeaters with their younger non-retained grade-mates, and when evaluating effects via a regression discontinuity method.     

Interventions for academically underachieving students: A systematic review and meta-analysis

Despite decades of research on interventions for academically underachieving students, no clear answers have emerged. Synthesizing across existing intervention efforts can help in understanding not only the overall effectiveness for these interventions, but also the factors that may moderate such effectiveness. In the current study, we conducted a systematic review and meta-analysis to examine the effectiveness of interventions for academically underachieving students, exploring effects on achievement and psychosocial outcomes. Overall, findings from 53 studies revealed that interventions are moderately effective in improving achievement and psychosocial outcomes. Moderator analyses revealed that intervention effectiveness varied by grade level. Implications for research and practice are discussed, particularly the need for rigorous evaluations of well-designed interventions that consider the fit between students’ unique reasons for underachievement and the makeup of the intervention.     

A sequence of learning processes in an intelligent tutoring system from topic-related appraisals to learning gains

Although intelligent tutoring systems (ITSs) are increasingly used, it is unclear which psychological processes precede students' learning gains. Using a pre- and posttest design, the present study examined a sequence of psychological processes informed by control value theory. We investigated (a) whether secondary school students' topic-related cognitive appraisals (value and control) affected their task-related affective (enjoyment and boredom) and cognitive (engagement and performance) outcomes while using the ITS and (b) whether task-related outcomes affected learning. Path analyses showed that students’ topic-related interest, but not perceived utility, personal importance or self-efficacy, was associated with task-related enjoyment. In turn, enjoyment showed reciprocal effects on and of engagement and ongoing task performance, which predicted final performance and, ultimately, learning gains. The influence of boredom, in contrast, was minimal along this sequence. More generally, the findings highlight the difficulty of establishing a clear pattern of sequential causation derived from control value theory for the current ITS context, with evidence demonstrating the systematic influence of confounders accounting for the predicted relations among components. Despite these limitations, we identified key psychological processes involving the contribution of affective and cognitive processes to learning in the ITS context.     

Signaling text-picture relations in multimedia learning: A comprehensive meta-analysis

The signaling principle recommends emphasizing relevant aspects of a multimedia message by means of signals (e.g., color coding). We determined the effectiveness of signals that highlight correspondences between text and pictures as well as possible boundary conditions by means of a meta-analysis. To this end, 58 potentially relevant articles were identified in a comprehensive search. After rating the studies based on inclusion criteria and correcting for biases, 27 studies were included in the meta-analysis yielding 45 pairwise comparisons with N = 2464 participants. Domain-specific prior knowledge, pacing of the materials, pictorial format, mapping requirements, and distinctiveness of signals were coded as moderators. For transfer and comprehension performance a positive small-to-medium effect size (r = .17, 95% CI [0.11, 0.22]) favoring signaled multimedia material was found, which was moderated by prior knowledge. The findings support the effectiveness of the signaling principle in particular for learners with low prior knowledge.     

Emotions in medical education: Examining the validity of the Medical Emotion Scale (MES) across authentic medical learning environments

To advance emotion research in education, there is a need to develop practical and context-relevant measures of emotion and to test the applicability of emotion theories using these measures. In two studies, we examined validity evidence of a self-report scale (the Medical Emotion Scale, MES) designed to measure the unique range of emotions activated within medical education. In Study 1, we administered the MES and conducted interviews with medical trainees (N = 15). Content analysis of interviews demonstrated that the MES captured an appropriate range of emotions and that there was alignment between scale responses and interview responses. In Study 2, we measured medical trainees' (N = 60) emotions using the MES for three learning environments. Results from principal components analysis revealed a structure of emotions according to valence (negative, positive) and novelty. The findings have implications for the measurement of emotions within technology-rich learning environments and beyond.     

The role of relationships and emotions in student perceptions of learning and assessment

The paper addresses the way in which participants in a qualitative study drew upon accounts of relationships and emotions in sharing their perceptions of assessment. By first exploring ideas about emotions and relationships in learning and assessment through the literature and subsequently discussing an interpretation of participant narratives, the author suggests that emotions and relationships surrounding past learning and assessment contexts can influence current perceptions of assessment and learning in powerful ways. The study also reveals how students value opportunities to express their beliefs, feelings and emotions during the assessment process. They also expect teachers to balance objectivity in assessment with empathy for those parts of themselves shared in the process. The conclusion is drawn that there are important implications for learning in the emotional response of students to assessment and in the nature of teaching and learning relationships that are worthy of further investigation.     

Self-regulated learning partially mediates the effect of self-regulated learning interventions on achievement in higher education: A meta-analysis

It is often assumed that interventions aimed at supporting students’ self-regulated learning (SRL) are effective for improving achievement because these interventions support SRL activity. In this study, meta-analytic structural equation modeling (MASEM) was used to test whether SRL activity indeed mediates the effect of SRL interventions on achievement in higher education. Contrary to popular belief, the results only provide evidence for partial mediation. Furthermore, three separate meta-analyses were performed to investigate the role of possible moderators of the relations between: (1) SRL interventions and achievement, (2) SRL interventions and SRL activity, and (3) SRL activity and achievement. Although SRL interventions were effective in improving SRL activity and achievement, most of the study, measurement, and intervention moderators did not explain significant variance of the investigated effect sizes. Other factors, such as task motivation and time on task, potentially influence the effectiveness of SRL interventions. Practical, theoretical and methodological implications are provided.     

Learning design and learning analytics in mobile and ubiquitous learning: A systematic review

Mobile and Ubiquitous Learning (m/u-learning) are finding an increasing adoption in education. They are often distinguished by hybrid learning environments that encompass elements of formal and informal learning, in activities that happen in distributed settings (indoors and outdoors), across physical and virtual spaces. Despite their purported benefits, these environments imply additional complexity in the design, monitoring and evaluation of learning activities. The research literature on learning design (LD) and learning analytics (LA) has started to deal with these issues. This paper presents a systematic literature review of LD and LA, in m/u-learning. Apart from providing an overview of the current research in the field, this review elicits elements of common ground between both communities, as shown by the similar learning contexts and complementary research contributions, and based on the research gaps, proposes to: address m/u-learning beyond higher education settings, reinforce the connection between physical and virtual learning spaces, and more systematically align LD and LA processes.