A meta-analysis of the effects of various science teaching strategies on achievement

This study is a meta-analysis of the question “What are the effects on achievement of different teaching techniques?” Twelve categories of teaching techniques were specified. Among these are questioning, wait-time, testing, focusing, manipulative, presentation approach, inquiry or discovery, audio-visual, and teacher direction. A coding form was then developed to allow for the uniform examination and recording of 76 variables from each study. One or more effect sizes were calculated and coded for each study. Over 300 microfilmed dissertations covering the past 30 years were examined. Some 2000 ERIC science abstracts were reviewed and suitable studies obtained. Finally, journal articles were reviewed. A total of 400 effect sizes representing 160 studies were produced. The main effect size overall was 0.34. Thus, for all samples considered the experimental science teaching techniques on the average resulted in one-third of a standard deviation improvement over traditional techniques. Mean effect sizes also were calculated for each technique. More than 20 other variables such as class size, community type, and science subject area were cross-tabulated with effect size.     

The empirical review of meta-analysis published in Korea

Meta-analysis is a statistical method that is increasingly utilized to combine and compare the results of previous primary studies. However, because of the lack of comprehensive guidelines for how to use meta-analysis, many meta-analysis studies have failed to consider important aspects, such as statistical programs, power analysis, publication bias, model selection, test of heterogeneity, and identification of heterogeneity. Therefore, the current study reviewed 84 meta-analysis studies conducted in Korea to examine proper application of the six categories named above. With regard to the issue of effect sizes, it was found that most of the meta-analysis studies obtained more than ten effect sizes, which seem to be an adequate number for representing an issue. However, many studies failed to consider the other issues: power analysis, publication bias, model selection, test of heterogeneity, and identification of heterogeneity.     

Teaching and learning floating and sinking: A meta-analysis

Floating and sinking (FS) is a key topic in science education, both at primary and secondary levels. The interpretation of FS phenomena, however, is challenging due to the difficulty of the scientific concepts and explanatory models involved (e.g., density, buoyancy), along with students' everyday experiences, which conflict with scientific explanations. Consequently, many studies over the last few decades have investigated how FS could be taught effectively to students of different ages while utilizing multiple teaching approaches. This meta-analysis summarizes findings from 69 intervention studies on teaching FS conducted between 1977 and 2021. Over all studies, we estimated a mean effect size of g = 0.85 (95% CI = 0.71, 0.99). This large effect size demonstrates that, although FS is a challenging concept, teaching FS is effective even in elementary school. Moreover, in a moderator analysis, we investigate the effect of intervention characteristics, students' age, as well as study design, and assessment features on the mean study effect size. To analyze the effect of these moderator variables, we use a three-level hierarchical meta-regression model for dealing with multiple effect sizes from single studies. We found two intervention characteristics that explain variance in study effect sizes: longer lasting interventions result in larger effect sizes and interventions where hands-on experiments are applied are more effective than those utilizing virtual experiments. Furthermore, studies with a treatment-control group comparison have significantly smaller effect sizes than studies with a pre-post design. We discuss the implications of our findings regarding the moderator variables for effective teaching of FS and further research on FS.     

Gender differences in student attitudes toward science: A meta-analysis of the literature from 1970 to 1991

A meta-analysis covering the literature between 1970 and 1991 was conducted using an approach similar to that suggested by Glass, McGaw, and Smith (1981) and Hedges, Shymansky, and Woodworth (1989). This analysis examined gender differences in student attitudes toward science, and correlations between attitudes toward science and achievement in science. Thirty-one effect sizes and seven correlations representing the testing of 6,753 subjects were found in 18 studies. The mean of the unweighted effect sizes was .20 (SD = .50) and the mean of the weighted effect size was .16 (SD = .50), indicating that boys have more positive attitudes toward science than girls. The mean correlation between attitude and achievement was .50 for boys and .55 for girls, suggesting that the correlations are comparable. Results of the analysis of gender differences in attitude as a function of science type indicate that boys show a more positive attitude toward science than girls in all types of science. The correlation between attitude and achievement for boys and girls as a function of science type indicates that for biology and physics the correlation is positive for both, but stronger for girls than for boys. Gender differences and correlations between attitude and achievement by gender as a function of publication date show no pattern. The results for the analysis of gender differences as a function of the selectivity of the sample indicate that general level students reflect a greater positive attitude for boys, whereas the high-performance students indicate a greater positive attitude for girls. The correlation between attitude and achievement as a function of selectivity indicates that in all cases a positive attitude results in higher achievement. This is particularly true for low-performance girls. The implications of these finding are discussed and further research suggested.     

Bias Corrections for Standardized Effect Size Estimates Used With Single-Subject Experimental Designs

A multilevel meta-analysis can combine the results of several single-subject experimental design studies. However, the estimated effects are biased if the effect sizes are standardized and the number of measurement occasions is small. In this study, the authors investigated 4 approaches to correct for this bias. First, the standardized effect sizes are adjusted using Hedges’ small sample bias correction. Next, the within-subject standard deviation is estimated by a 2-level model per study or by using a regression model with the subjects identified using dummy predictor variables. The effect sizes are corrected using an iterative raw data parametric bootstrap procedure. The results indicate that the first and last approach succeed in reducing the bias of the fixed effects estimates. Given the difference in complexity, we recommend the first approach.     

Gender and science achievement: A reanalysis of studies from two meta-analyses

Data from 30 studies of the magnitude of gender differences in science achievement previously examined in two separate reviews were synthesized using modern methods for meta-analysis. Two meta-analysis methodologies were used: analyses of effect sizes (standardized mean differences) and vote-counting estimation procedures. Analyses revealed that magnitudes of gender differences in science achievement varied according to the subject matter under study. Males showed significant advantages in studies of biology, general science, and physics, but significant differences were not found for studies of mixed science content, and geology and earth sciences, or in a single study of chemistry. However, in all cases the numbers of effects examining each subject-matter area were quite small (seven or less). Also, studies which had focused on gender differences evidenced larger gender differences than other studies. Other possible predictors of the magnitudes of gender differences, such as grade level of the subjects and test length, did not account for significant amounts of variation among these study outcomes. Consequently, they also did not provide as strong an explanation of the patterns of gender differences as did subject matter and focus on gender.     

Meta-analysis of the effect of practical reasoning instruction on student outcome in Home Economics education in Korea

This meta-analysis reviewed the effect of practical reasoning instruction (PRI) on student outcome in Home Economics education in Korea. In this meta-analysis, 25 studies with 35 effect sizes were analyzed. The results of this review indicated that PRI is more effective than traditional instruction on student outcomes. A medium and significant mean effect was 0.60 (SE?=?0.06). Categorical analyses and regression analyses were employed to find the sources of variance and moderators that predict the effects of PRI. The moderator analyses revealed no statistically significant effects of publication status, study design, type of student outcome, gender of students, and location. The school level of the students and duration were revealed to be moderators. The effect of PRI on student outcome was found to be smaller in middle schools than in elementary or high schools. The results of regression analysis for middle school students indicated that the effect of PRI was predicted by the length of the intervention. A sensitivity analysis indicates that the method this review used for calculating effect sizes for nonexistent control groups was robust.     

Efficiency of digital and non-digital educational games: A comparative meta-analysis and a meta-thematic analysis

Abstract

The present article attempts to reinterpret the findings of most recent studies investigating effect of using games for teaching purposes. A methodological approach combining a meta-analysis of quantitative data with qualitative ones was adopted in order to present the broadest picture of the current research on educational use of games. To this end, we conducted a meta-analysis of 180 effect size comparisons out of 154 empirical studies on the effect of both digital and non-digital games on academic achievement conducted during the period from 2004 to 2019 in order to determine the overall effect size of using games for teaching various subjects. The overall sample size of the studies included a total number of 12800 participants. Some moderator analyses were also carried out to determine the exact efficiency of educational games in terms of student levels, durations of implementation of game activities, school subjects in which games were used, class sizes, kinds of games and achievement tests used. The findings suggest that educational games have a positive effect on academic achievement and this effect is at a medium level (g?=?0.695). The highest effect sizes were observed in foreign language courses (g?=?0.87), small (less than 50) class sizes (g?=?0.87), and in non-digital games (g?=?0.90). Moreover, we conducted a meta-thematic analysis based on document analysis of qualitative studies in order to further consolidate the findings of the meta-analysis. The meta-thematic dimension of our study reveals cognitive contributions as well as drawbacks of game-based teaching, and provides suggestions for conducting educational games in a better way.     

A Meta-analysis of the Effects of Computer Technology on School Students’ Mathematics Learning

This study examines the impact of computer technology (CT) on mathematics education in K-12 classrooms through a systematic review of existing literature. A meta-analysis of 85 independent effect sizes extracted from 46 primary studies involving a total of 36,793 learners indicated statistically significant positive effects of CT on mathematics achievement. In addition, several characteristics of primary studies were identified as having effects. For example, CT showed advantage in promoting mathematics achievement of elementary over secondary school students. As well, CT showed larger effects on the mathematics achievement of special need students than that of general education students, the positive effect of CT was greater when combined with a constructivist approach to teaching than with a traditional approach to teaching, and studies that used non-standardized tests as measures of mathematics achievement reported larger effects of CT than studies that used standardized tests. The weighted least squares univariate and multiple regression analyses indicated that mathematics achievement could be accounted for by a few technology, implementation and learner characteristics in the studies.     

Webinars in higher education and professional training: A meta-analysis and systematic review of randomized controlled trials

Digital learning environments are increasingly popular in higher education and professional training. Teaching and learning via webinars, and web conferencing more broadly, represents one widely used approach. Webinars are defined as web-based seminars, in which participants and facilitators communicate live over the Internet across distant geographical locations using shared virtual platforms and interact ubiquitously and synchronously in real time via voice over IP technology and web camera equipment. In the past, studies have reported mixed evidence concerning the effectiveness of webinars in promoting student achievement. As a remedy, this systematic literature review and meta-analysis cumulates observed effect sizes from previously published randomized controlled trials and corrects artifactual variance induced by sampling error. The research questions were: How effective are webinars in promoting student achievement? And which characteristics moderate webinar effectiveness? The findings suggest that webinars were slightly more effective than control conditions (online asynchronous learning management systems and offline face-to-face classroom instruction), but the differences were trivial in size. Differences were moderated by webinar, participant, achievement, and publication characteristics. This meta-analysis is the first to systematically review and meta-analyze the best evidence available for evaluating the effectiveness of webinars and video conferences in promoting student knowledge and skills. The implications of the study's findings can inform school teachers, lecturers, trainers, technologists, and theorists interested in the computer-supported design, implementation, delivery, tutoring, and assessment of webinar-based learning environments.     

Fostering scientific reasoning in education – meta-analytic evidence from intervention studies

Scientific reasoning skills are not just for researchers, they are also increasingly relevant for making informed decisions in our everyday lives. How can these skills be facilitated? The current state of research on supporting scientific reasoning includes intervention studies but lacks an integrated analysis of the approaches to foster scientific reasoning in primary through secondary education. This meta-analysis evaluates effect sizes taken from 30 interventions in experimental and quasi-experimental studies and shows a medium mean effect of interventions on scientific reasoning. Interventions successfully facilitate scientific discovery, scientific argumentation, or nature of science in all age groups. Moderator analyses show that interventions set in constructive and interactive learning activities yield positive effects but do not differ substantially. Although the meta-analysis is limited by the number of studies included, we can conclude that scientific reasoning can successfully be facilitated and we show which characteristics are beneficial in educational interventions.     

Does peer assessment promote student learning? A meta-analysis

Abstract

In recent years, there has been an increasing use of peer assessment in classrooms and other learning settings. Despite the prevailing view that peer assessment has a positive effect on learning across empirical studies, the results reported are mixed. In this meta-analysis, we synthesised findings based on 134 effect sizes from 58 studies. Compared to students who do not participate in peer assessment, those who participate in peer assessment show a .291 standard deviation unit increase in their performance. Further, we performed a meta-regression analysis to examine the factors that are likely to influence the peer assessment effect. The most critical factor is rater training. When students receive rater training, the effect size of peer assessment is substantially larger than when students do not receive such training. Computer-mediated peer assessment is also associated with greater learning gains than the paper-based peer assessment. A few other variables (such as rating format, rating criteria and frequency of peer assessment) also show noticeable, although not statistically significant, effects. The results of the meta-analysis can be considered by researchers and teachers as a basis for determining how to make effective use of peer assessment as a learning tool.     

Meta-analysis of faculty's teaching effectiveness: Student evaluation of teaching ratings and student learning are not related

Student evaluation of teaching (SET) ratings are used to evaluate faculty's teaching effectiveness based on a widespread belief that students learn more from highly rated professors. The key evidence cited in support of this belief are meta-analyses of multisection studies showing small-to-moderate correlations between SET ratings and student achievement (e.g., Cohen, 1980, Cohen, 1981; Feldman, 1989). We re-analyzed previously published meta-analyses of the multisection studies and found that their findings were an artifact of small sample sized studies and publication bias. Whereas the small sample sized studies showed large and moderate correlation, the large sample sized studies showed no or only minimal correlation between SET ratings and learning. Our up-to-date meta-analysis of all multisection studies revealed no significant correlations between the SET ratings and learning. These findings suggest that institutions focused on student learning and career success may want to abandon SET ratings as a measure of faculty's teaching effectiveness.     

Learning From Video: A Meta-Analysis of the Video Deficit in Children Ages 0 to 6 Years

Young children often learn less from video than face-to-face presentations. Meta-regression models were used to examine the average size of this difference (video deficit) and investigate moderators. An average deficit of about half of a standard deviation was reported across 122 independent effect sizes from 59 reports, involving children ages 0–6 years. Moderator analyses suggested (a) the deficit decreased with age, (b) object retrieval studies showed larger deficits than other domains, and (c) there was no difference between studies using live versus prerecorded video. Results are consistent with a multiple-mechanism explanation for the deficit. However, the analyses highlighted potential quality and publication bias issues that may have resulted in overestimation of the effect and should be addressed by future researchers.     

The development of students critical thinking abilities and dispositions through the concept mapping learning method – A meta-analysis

Our meta-analysis investigates the effectiveness of the concept mapping method compared with the traditional teaching method, for the improvement of critical thinking ability and disposition outcomes. We included studies measuring students' critical thinking through standardised tests, comparing an experimental (concept mapping) group with a control (traditional teaching) one. Effect sizes for critical thinking ability and disposition outcomes were pooled with a random-effects model. We included in our meta-analysis 21 studies (108 comparisons) involving 1695 students. The concept mapping method was more effective than the traditional teaching method at posttest for the improvement of critical thinking ability (g = 0.531, 95% CI 0.279 to 0.783) and disposition (g = 0.648, 95% CI 0.266 to 1.031). Heterogeneity was moderate to high, Egger's test did not indicate any evidence of publication bias; however, both visual inspection of the Funnel plot and Duval and Tweedie's trim and fill method indicated potentially three missing studies for critical thinking ability and two for critical thinking disposition. Additionally, we analysed the potential moderating effect of students' demographic characteristics, educational conditions, concept mapping elaboration methods and allocation type. Allocation type was a significant moderator, having a strong effect on concept mapping for critical thinking abilities in randomised studies (g = 0.739, 95% CI 0.356 to 1.122), but its effect is low in non-randomised studies (g = 0.265, 95% CI 0.014 to 0.517).     

The effect of inquiry teaching and advance organizers upon student outcomes in science education

Two areas of research were explored in this meta-analysis: comparison of inductive versus deductive teaching approachs, and the use of advance organizers. This study involved the coding of 128 characteristics for 39 studies which spanned the period from 1957 through 1980. The coding variables included 57 which were concerned with features of the treatment while 12 were concerned with outcome attributes. Aspects such as research methodology, sample characteristics, and instructional experiences were examined quantitatively in terms of their relationships to the treatment effects through the use of a common metric for all studies. Mean effect sizes were calculated and reported.     

A meta‐analysis of national research: Effects of teaching strategies on student achievement in science in the United States

This project consisted of a meta‐analysis of U.S. research published from 1980 to 2004 on the effect of specific science teaching strategies on student achievement. The six phases of the project included study acquisition, study coding, determination of intercoder objectivity, establishing criteria for inclusion of studies, computation of effect sizes for statistical analysis, and conducting the analyses. Studies were required to have been carried out in the United States, been experimental or quasi‐experimental, and must have included effect size or the statistics necessary to calculate effect size. Sixty‐one studies met the criteria for inclusion in the meta‐analysis. The following eight categories of teaching strategies were revealed during analysis of the studies (effect sizes in parentheses): Questioning Strategies (0.74); Manipulation Strategies (0.57); Enhanced Material Strategies (0.29); Assessment Strategies (0.51); Inquiry Strategies (0.65); Enhanced Context Strategies (1.48); Instructional Technology (IT) Strategies (0.48); and Collaborative Learning Strategies (0.95). All these effect sizes were judged to be significant. Regression analysis revealed that internal validity was influenced by Publication Type, Type of Study, and Test Type. External validity was not influenced by Publication Year, Grade Level, Test Content, or Treatment Categories. The major implication of this research is that we have generated empirical evidence supporting the effectiveness of alternative teaching strategies in science. © 2007 Wiley Periodicals, Inc. J Res Sci Teach 44: 1436–1460, 2007     

Evaluating the evidence base for evidence-based instructional practices in chemistry through meta-analysis

Investigating the effectiveness of instructional practices provides an evidence base to inform instructional decisions. Synthesizing research studies on instructional effectiveness provides an estimate of the generalizability of effectiveness across settings, along with an exploration of factors that may moderate the impact, which cannot be achieved within individual studies. This study sought to provide a synthesis of evidence-based instructional practices (EBIPs) particular to chemistry through meta-analysis. Ninety-nine studies were analyzed comprising a broader view of chemistry specific studies than past meta-analyses. The results showed that EBIPs feature a demonstrably positive impact on students' academic performance in chemistry, although assessment topic coverage and setting size emerged as relevant moderators of impact and prevented making definitive conclusions of the relative impact of each EBIP. In examining publication bias, an asymmetric distribution of studies based on standard error (SE) and effect size was found, indicative of potential publication bias. To explore the potential impact of bias, the trim and fill method was employed resulting in a range for the overall weighted effect size from 0.29 to 0.62. The study concludes that evidence-based instructional practices have demonstrated effectiveness even in consideration of potential publication bias, as the range of effect sizes remains positive, but highlights the continued need to publish null findings in the research literature.     

What limits the encoding effect of note-taking? A meta-analytic examination

Previous meta-analyses indicate that the overall encoding effect of note-taking is positive but modest. This meta-analysis of 57 note-taking versus no note-taking comparison studies explored what limits the encoding effect by examining the moderating influence of seven variables: intervention, schooling level, presentation mode and length, test mode, and publication year and source. It was found that (a) either positive interventions or rise in schooling level did not enhance the benefits of note-taking; (b) visual presentation of learning material interfered with the note-taking process, whereas longer presentation did not; (c) recall test detected the encoding effect more than recognition and higher-order performance tests; and (d) publication year and source contributed to the variation in effect sizes. These results suggest that the modest encoding effect is not due to the incompleteness of students’ spontaneous note-taking procedures, but mechanical demands of note-taking, type of learning outcome measure, and publication characteristics.