Detecting bias in meta-analyses of distance education research: big pictures we can rely on

This article has two interrelated purposes. The first is to explain how various forms of bias, if introduced during any stage of a meta-analysis, can provide the consumer with a misimpression of the state of a research literature. Five of the most important bias-producing aspects of a meta-analysis are presented and discussed. Second, armed with this information, we examine 15 meta-analyses of the literatures of distance education (DE), online learning (OL), and blended learning (BL), conducted from 2000 to 2014, with the intention of assessing potential sources of bias in each. All of these meta-analyses address the question: “How do students taking courses through DE, OL, and BL compare to students engaged in pure classroom instruction in terms of learning achievement outcomes?” We argue that questions asked by primary researchers must change to reflect issues that will drive improvements in designing and implementing DE, OL, and BL courses.     

Effects of flipping the classroom on learning outcomes and satisfaction: A meta-analysis

In a flipped classroom, students study instructional material before class and apply this material during class. To provide a statistical synthesis of current research on effects of flipped classrooms, we conducted meta-analyses that included 114 studies which compared flipped and non-flipped classrooms in secondary and postsecondary education. We found a small positive effect on learning outcomes, but no effect was found on student satisfaction regarding the learning environment. In addition, we found considerable heterogeneity between studies. Moderator analyses showed that students in flipped classrooms achieve higher learning outcomes when the face-to-face class time was not reduced compared to non-flipped classrooms, or when quizzes were added in the flipped classrooms. We conclude that a flipping the classroom (FTC) approach is a promising pedagogical approach when appropriately designed. Our results provide insights into effective instructional FTC design characteristics that support an evidence-informed application of FTC.     

Toward a set of design principles for mathematics flipped classrooms: A synthesis of research in mathematics education

This paper analyzed the journal publications of mathematics flipped classroom studies in K-12 and higher education contexts. We focused specifically on a set of flipped classroom studies in which pre-class instructional videos were provided prior to face-to-face class meetings. We examined the following four major issues: (a) the types of out-of-class and in-class instructional activities used, (b) the effect of flipped learning on student achievement, (c) the participant perceptions of flipped classroom benefits, and (d) the main challenges of flipped classroom implementations. A meta-analysis of 21 comparison studies showed an overall significant effect in favor of the flipped classroom over the traditional classroom for mathematics education (Hedges' g = 0.298, 95% CI [0.16, 0.44]), with no evidence of publication bias. A broader research synthesis of 61 studies revealed that the flipped classroom approach benefited student learning in three main aspects: increasing in-class time for task/practice, integrating new knowledge with existing beliefs, and real-time feedback. The two most frequently reported flipped classroom challenges were students' unfamiliarity with flipped learning and significant start-up effort on the part of instructors. We hence propose a set of design principles to help foster the transition to the flipped classroom and improve the out-of-class and in-class learning designs. This set of design principles can also provide a more focused agenda for future research to examine the effect of the flipped classroom approach on student learning and motivation.     

Flipped classroom instruction for inclusive learning

The flipped classroom is a teaching methodology that has gained recognition in primary, secondary and higher education settings. The flipped classroom inverts traditional teaching methods, delivering lecture instruction outside class, and devoting class time to problem solving, with the teacher's role becoming that of a learning coach and facilitator. This methodology provides an avenue for more hands‐on and student‐driven learning during class time. The benefits of a flipped classroom include increased student motivation, differentiating instruction, self‐pacing lessons and mastery learning, increased collaboration and instant feedback for formative assessment. Although limited research has been conducted regarding students with learning difficulties in inclusive settings using a flipped learning model, initial research indicates flipping a classroom can be beneficial. Information is provided on inclusive practices from numerous countries that are beneficial to students with learning difficulties, as well as strategies and resources for individuals who may want to implement a flipped classroom.     

The “flipped classroom” approach: Stimulating positive learning attitudes and improving mastery of histology among medical students

Traditional medical education methodologies have been dramatically impacted by the introduction of new teaching approaches over the past few decades. In particular, the “flipped classroom” format has drawn a great deal of attention. However, evidence regarding the effectiveness of the flipped model remains limited due to a lack of outcome‐based studies. In the present study, a pilot histology curriculum of the organ systems was implemented among 24 Traditional Chinese Medicine (TCM) students in a flipped classroom format at Jinan University. As a control, another 87 TCM students followed a conventional histology curriculum. The academic performance of the two groups was compared. In addition, a questionnaire was administered to the flipped classroom group. The test scores for the flipped classroom participants were found to be significantly higher compared to non‐participants in the control group. These results suggest that students may benefit from using the flipped classroom format. Follow‐up questionnaires also revealed that most of the flipped classroom participants undertook relatively more earnest preparations before class and were actively involved in classroom learning activities. The teachers were also found to have more class time for leading discussions and delivering quizzes rather than repeating rote didactics. Consequently, the increased teaching and learning activities contributed to a better performance among the flipped classroom group. This pilot study suggests that a flipped classroom approach can be used to improve histology education among medical students. However, future studies employing randomization, larger numbers of students, and more precise tracking methods are needed before definitive conclusions can be drawn. Anat Sci Educ 10: 317–327. © 2016 American Association of Anatomists.     

从认知负荷视角探究翻转课堂＊--兼及翻转课堂的典型模式分析

翻转课堂是一种将课内与课外的学习活动颠倒的教学模式,其实质是颠倒了学习内容传递与内化的顺序。自翻转课堂产生以来,很多学校或教育机构采用此模式,并取得了较优的成效。与此同时,可汗学院的出现也成为了翻转课堂得以迅速发展的催化剂。目前,国内众多学校都在采用翻转课堂模式,期望翻转课堂成为教学改革的一个转折点。从心理学的角度看,翻转课堂教学模式本质上是降低了认知负荷,使学习者的认知资源得到更合理的分配,从而获得更好的学习效果。另外,通过分析多位学者提出的不同翻转课堂模式,以期能为翻转课堂的应用研究提供帮助。     

Implementing the flipped classroom: an exploration of study behaviour and student performance

The flipped classroom is becoming more popular as a means to support student learning in higher education by requiring students to prepare before lectures and actively engaging students during lectures. While some research has been conducted into student performance in the flipped classroom, students’ study behaviour throughout a flipped course has not been investigated. This study explored students’ study behaviour throughout a flipped and a regular course by means of bi-weekly diaries. Furthermore, student references to their learning regulation were explored in course evaluations. Results from the diaries showed that students’ study behaviour in the flipped course did not appear to be very different from that of students in a regular course. Furthermore, study behaviour did not appear strongly related to student performance in both the flipped and the regular course. Exploration of student references to their learning regulation in the course evaluations showed that some students experienced the flipped course design as intended to support their learning process. Other students, however, demonstrated resistance to changing their study behaviour even though changing study behaviour is expected in order to benefit from the flipped classroom. Further research on the relationship between students’ learning regulation and actual study behaviour and course results is necessary to understand when and why implementing the flipped classroom is successful. Recommendations that may help more effective flipped classroom implementation include considering the prior history between students and instructor(s), the broader curriculum context, and frequent expectation communication especially with large numbers of students and non-mandatory lecture attendance.     

A meta-analysis on the relationship between school characteristics and student outcomes in science and maths – evidence from large-scale studies

ABSTRACT

In the present meta-analysis, we examine how secondary school characteristics – such as schools’ academic press, school climate, material resources, personnel resources, classroom climate, instructional practices, out-of-school activities, and socioeconomic status (SES) composition – provide opportunities for students to engage in science and maths, and how these matter with regard to students’ cognitive and motivational-affective outcomes. The meta-analysis includes 71 (international) articles from large-scale studies with a total of 3,960,281 students, 260,390 schools, and 285 effect sizes that were transformed to correlation coefficients. Multilevel meta-analyses were performed. Results identified a number of school variables that can be regarded as relevant for making a difference in student outcomes and at the same time be influenced by education. These refer to school variables such schools’ academic press, classroom climate, instructional practices, and out-of-school activities. Moreover, SES composition was significantly related to student outcomes. Material and personnel resources as well as school climate yielded a close to zero effect. No differences were found between cognitive and motivational-affective outcome variables or between science and maths. The results point to the most promising school characteristics for promoting students’ outcomes and emphasise schools’ potential for students’ engagement in science and maths.     

Exploring the effects of online learning behaviors on short-term and long-term learning outcomes in flipped classrooms

ABSTRACT

The flipped classroom pedagogy has been widely used recently. Despite many researches have paid attention with the learning outcome of flipped classroom, there has been limited attention in regard to investigate the relationship between learning behavior and learning outcomes in a flipped classroom. In this paper, we proposed to investigate the influence of online learning behaviors on short-term and long-term learning outcomes in a flipped classroom. This study used Calculus and grade point average (GPA) scores to represent short-term and long-term learning outcomes, respectively. Multiple linear regression indicated that students’ online learning behavior does not have a significant effect on short-term learning outcomes, but has a significant effect on long-term learning outcomes. For applying multiple correspondence analysis, students were divided into groups according to five grade levels based on their scores. According to GPA grade level, students’ online learning behaviors had a significant effect on long-term learning outcomes for the five groups (GPAa, GPAb, GPAc, GPAd, GPAe). According to their Calculus grade level, students’ online learning behaviors had a significant effect on short-term learning outcomes for three groups (CALa, CALd, and CALe), but two groups (CALb and CALc) did not demonstrate this trend. For exploring the effects of online learning behaviors on future learning outcomes, GPA can be considered representative because the GPA was calculated for the entire academic year 2015. Students in the CALa group exhibited the highest frequency of online learning behaviors and obtained the highest GPA grade levels (GPAa and GPAb). For the CALb, CALc, CALd, and CALe groups, students with a higher frequency of online learning behaviors obtained a higher GPA grade level. These results indicate that students’ online learning behaviors have a positive effect on future learning outcomes.     

Which students benefit most from a flipped classroom approach to language learning?

Research has shown that the potential benefits of a flipped classroom could be diminished by the way students perceive and prepare information prior to class. This study aims to explore individual characteristics, such as learner motivation, self‐efficacy and epistemology beliefs, that might have an impact on learning outcomes in a flipped classroom. Data were collected using four instruments during a 7‐week flipped classroom conducted from mid‐September 2014 to mid‐November 2014 with a total enrolment of 85 students (10 females and 75 males) in the required course, Applied English for Vocational Education. After controlling for pre‐test and other covariates (eg, gender, grade and experience), an analysis based on structural equation modeling showed a positive and significant (β = 0.12, p < 0.05) indirect effect of instrumentality (promotion) on the change in test scores through averaged quiz scores. Beliefs had a significant positive effect on change in scores between pre‐ and post‐tests (β = 0.20, p < 0.05). Findings revealed that language learners with a high level of motivation in instrumentality (promotion) followed quiz mechanisms closely and thus benefited most from the flipped classroom. Learners with high beliefs exhibited improvement in the post‐tests without doing well in quizzes. The implications for a flipped language learning classroom are discussed, including multiple strategies for regulating learners of different personal traits to preview the online course content before class.     

Cognitive regulations in ICT-supported flipped classroom interactions: An activity theory perspective

In recent years, most studies on flipped classrooms or flipped learning have focused on learners’ motivation, academic achievement and teachers’ and students’ perceptions of the flipped classroom. Little research has involved interactions in the flipped classroom. This study aims to show that in-class collaborative learning in the flipped classroom can not only be conceived as a collective activity consisting of discrete actions, but also as a systematic activity. From the activity-theoretical perspective, contradictions were found between components of an activity such as subject, tools, rules, community, division of labor, object and outcome. This study conducted a pre-experiment, selecting one lesson of the seventh-grade Information Technology Course as the learning content, conducted in a flipped classroom. Three triads with different compositions were specifically observed by recording their interactions. More second-level contradictions existed between subject and tools, and division of labor within the low-level triad than within the other two triads, and more first-level contradictions also occurred within the low-level triad than within the other two triads. Most second-level contradictions within the high-level triad were resolved by self-regulation of cognition. Instead, most second-level contradictions within the mixed-level triad were overcome by co-regulation or shared regulation of cognition. This study sheds light on how to effectively design and implement flipped learning, and provides a new approach for analyzing interactions in the classroom.     

Individual participant data meta-analysis of the impact of educational interventions on pupils eligible for Free School Meals

Meta-analysis is the synthesis of findings from research projects, which enables an estimate of the average or pooled effect across various studies. This study presents findings from the intention to treat analysis for a series of educational evaluations in England using a two-stage meta-analysis with standardised outcome data and individual participant data meta-analyses. The research estimates the overall impact of educational trials on pupils eligible for Free School Meals (FSM) and the attainment gap in literacy and mathematics performance between FSM and non-FSM pupils based on analysis of 88 trials and data from over half a million pupils. For the meta-analyses, frequentist and Bayesian multilevel models were used to estimate the individual and pooled effect size across categories of explanatory variables such as age groups (key stages in England) and aspects of the type of interventions (one-to-one, small group, whole class). Results indicated that the overall impact of interventions on the literacy outcomes of FSM pupils was positive, with a pooled effect size of 0.06 (0.03, 0.08). However, for mathematics, no overall effect on FSM pupils was observed. Analysis of the attainment gap indicated that literacy outcomes for FSM pupils were improved by interventions marginally more than for non-FSM pupils (pooled attainment gap 0.01 (−0.01, 0.04)). The risk of bias assessment showed that estimates were consistent across different methodological approaches. Overall, evidence from this study can be used to identify, test and scale educational interventions in schools to improve educational outcomes for disadvantaged pupils.     

Motivation and cognitive load in the flipped classroom: definition,rationale and a call for research

Flipped classroom approaches remove the traditional transmissive lecture and replace it with active in-class tasks and pre-/post-class work. Despite the popularity of these approaches in the media, Google search, and casual hallway chats, there is very little evidence of effectiveness or consistency in understanding what a flipped classroom actually is. Although the flipped terminology is new, some of the approaches being labelled ‘flipped’ are actually much older. In this paper, we provide a catch-all definition for the flipped classroom, and attempt to retrofit it with a pedagogical rationale, which we articulate through six testable propositions. These propositions provide a potential agenda for research about flipped approaches and form the structure of our investigation. We construct a theoretical argument that flipped approaches might improve student motivation and help manage cognitive load. We conclude with a call for more specific types of research into the effectiveness of the flipped classroom approach.     

Establishing the Integrity of the Independent Variable in Early Intervention Programs

With the implementation of P.L. 99–457, early intervention programs are becoming more widespread; they are also being pressed to become more accountable. Persons evaluating early intervention programs have typically adopted outcome-oriented approaches focusing exclusively on measuring the dependent program variable (i.e., changes in student measures). This article highlights some of the problems inherent in these traditional approaches to program evaluation. A number of recommendations are offered for assessing the independent variable with the same methodological rigor that is typically reserved for measuring student outcomes in program evaluation studies. The implications of this shift in methodological approach are discussed in terms of improving the ability to replicate successful programs, design new ones with increased efficiency, and provide long-lasting support for early intervention.     

"扎根"课堂:翻转课堂的扎根式教学创新

新冠肺炎疫情影响下混合式教学在中国大学校园得到广泛应用,越来越多大学教师采用翻转课堂来开展教学活动,但在如何实现以学习者为中心方面仍面临着诸多挑战,进而影响翻转课堂的教学过程与质量。为改善翻转课堂在教学活动等方面存在的不足之处,运用扎根理论来改进翻转课堂的教学模式设计,基于扎根理论研究程序开发扎根式课堂学习规范和标准,构建以个体问题导向学习、小组合作翻转研讨、集体翻转研讨等为特征的翻转学习机制,并在课程教学中进行了实践应用,检验其必要性和可行性。研究表明,对翻转课堂进行扎根式教学设计,可以更有效地实现以学习者为中心的翻转教学,为大学生从浅层学习走向深度学习提供有益帮助。     

Classroom dialogue: a systematic review across four decades of research

Recognizing that empirical research into classroom dialogue has been conducted for about 40?years, a review is reported of 225 studies published between 1972 and 2011. The studies were identified through systematic search of electronic databases and scrutiny of publication reference lists. They focus on classroom dialogue in primary and secondary classrooms, covering the full age range of compulsory schooling. The methods of data collection and analysis used in the studies are described and discussed, with changes and continuities over time highlighted. Study results are then summarized and integrated to present a succinct picture of what is currently known and where future research might profitably be directed. One key message is that much more is known about how classroom dialogue is organized than about whether certain modes of organization are more beneficial than others. Moreover, epistemological and methodological change may be required if the situation is to be remedied.     

Webcasts promote in-class active participation and learning in an engineering elective course

This paper describes the design and outcomes of an educational intervention undertaken to improve the quality of delivery of a fourth-year engineering elective course – Industrial Wastewater and Solid Waste Management at the University of Queensland. The objective was to increase the level of active participation of students in planned active-learning classroom activities, including whole-class discussions and small group project-type work. According to a flipped classroom model, new online material in the form of webcasts was proposed to students before class. Students reacted very positively to the webcasts: the percentage of students viewing the webcast before planned workshop sessions ranged between 80% and 92% over the five weeks of the intervention. Enhanced engagement led also to increased attendance (85–92% at workshop sessions), and remarkable active participation in class (half of observed teams were ～80% active). Remarkably, team performance as quantified by their report marks linearly correlated with the level of active participation in class.     

Students’ Perceptions and Emotions Toward Learning in a Flipped General Science Classroom

Recently, the inverted instruction methodologies are gaining attentions in higher educations by claiming that flipping the classroom engages more effectively students with the learning process. Besides, students’ perceptions and emotions involved in their learning process must be assessed in order to gauge the usability of this relatively new instruction methodology, since it is vital in the educational formation. For this reason, this study intends to evaluate the students’ perceptions and emotions when a flipped classroom setting is used as instruction methodology. This research was conducted in a general science course, sophomore of the Primary Education bachelor degree in the Training Teaching School of the University of Extremadura (Spain). The results show that the students have the overall positive perceptions to a flipped classroom setting. Particularly, over 80 % of them considered that the course was a valuable learning experience. They also found this course more interactive and were willing to have more courses following a flipped model. According to the students’ emotions toward a flipped classroom course, the highest scores were given to the positive emotions, being fun and enthusiasm along with keyword frequency test. Then, the lowest scores were corresponded to negative emotions, being boredom and fear. Therefore, the students attending to a flipped course demonstrated to have more positive and less negative emotions. The results obtained in this study allow drawing a promising tendency about the students’ perceptions and emotions toward the flipped classroom methodology and will contribute to fully frame this relatively new instruction methodology.     

指向主动学习的大学英语校本翻转课堂新模式——以福建某高校大学英语教学改革为例

大学英语翻转课堂模式的构建应考虑地方院校校本化需求以及大学英语教学长期以来对主动学习的关注.在诸多关于翻转课堂的研究中,翻转课堂与主动学习之间的关系尚未得到足够重视.立足校本实际,探究由交互式教学环境、差异化教学内容、合作式学习活动和多元化学习评价四个要素构建而成的大学英语翻转课堂新模式,并以新模式下翻转课堂的教学组织...     

Improvement in Student Performance and Perceptions through a Flipped Anatomy Classroom: Shifting from Passive Traditional to Active Blended Learning

One of the major challenges facing anatomy educators is delivering the anatomy materials in fewer hours with a reduction of anatomy courses in the integrated curricula. The flipped classroom modality may be an innovative solution. However, its effectiveness remains under debate due to a lack of outcome-based research and the mixed results of students' performance. The present study aimed to determine the outcome of the flipped classroom based upon the level of student cognition. The study investigated performance on 17 multiple-choice anatomy questions as a part of the final examination of the musculoskeletal system module. The results were compared between the first-year female students of Qassim Medical College, specifically the flipped classroom group (46 students) of the academic year (2018–2019) and the traditional group (49 students) of the academic year (2017–2018). The mean differences in the students' grades on the anatomy questions at the level of knowledge, application, and analysis using Cohen's d test were 0.43, 1.41, and 1.01, respectively. These results suggest the positive impact of flipping the students' classrooms on improving their levels of thinking according to Bloom's taxonomy. Perception surveys also revealed students' enthusiasm for the pre-class activities, leading to a better performance in the class with more engagement with their peers and teachers. The present study suggested that the flipped classroom modality can be performed to compensate for the reduction of anatomy educational hours. However, further studies are recommended to investigate the best practices of the flipped classroom that fit with the students' needs and workloads.