Communicating comparative findings from meta-analysis in educational research: some examples and suggestions

ABSTRACT

This article reviews some of the strengths and limitations of the comparative use of meta-analysis findings, using examples from the Sutton Trust-Education Endowment Foundation Teaching and Learning ‘Toolkit’ which summarizes a range of educational approaches to improve pupil attainment in schools. This comparative use of quantitative findings has similar characteristics to umbrella reviews which provide a succinct but applicable summary of the current state of evidence to inform practice or policy. Meta-analysis helps to identify which approaches have, on average, made the most difference to tested learning outcomes, in terms of effect size. We suggest that any comparative inferences made between meta-analyses should be treated cautiously, but taken seriously. Additionally, we present alternative ways of interpreting effect sizes, security ratings and cost-estimates to make research findings accessible, whilst retaining appropriate accuracy which is discussed using the ‘Toolkit’ as an example. We conclude by arguing that we should consider the available information not as ‘what works’, but ‘what has worked’ to understand its value and limits in terms of supporting the development of research-based practice.