For whom does “what works” work? The political economy of evidence-based education

ABSTRACT

What role does scientific evidence play in educational practice? Supporters of evidence-based education (EBE) see it as a powerful way of improving the quality of public services which is readily applicable to the education sector. Academic scholarship, however, points out important limits to this applicability. I offer an account inspired by Tullock’s theory of bureaucracy that helps explain EBE’s influence despite these limits. Recent configurations of EBE are an imperfect solution to 2 imperatives where policymakers are at an informational disadvantage: (a) guiding professionals working in the field and (b) evaluating evidence from academic researchers. EBE, especially in the form of RCTs and systematic reviews, offers a way of filtering a complex range of research to produce a determinate result that is transparent to policymakers. However, this impression of research transparency is misleading as it omits theoretical background that is critical for successfully interpreting the results of particular interventions. This comes at a cost of relevance to the frontline professionals whom this research evidence is supposed to inform and help.     

Introducing the mean absolute deviation ‘effect’ size

This paper revisits the use of effect sizes in the analysis of experimental and similar results, and reminds readers of the relative advantages of the mean absolute deviation as a measure of variation, as opposed to the more complex standard deviation. The mean absolute deviation is easier to use and understand, and more tolerant of extreme values. The paper then proposes the use of an easy to comprehend effect size based on the mean difference between treatment groups, divided by the mean absolute deviation of all scores. Using a simulation based on 1656 randomised controlled trials each with 100 cases, and a before and after design, the paper shows that the substantive findings from any such trial would be the same whether raw-score differences, a traditional effect size like Cohen's d, or the mean absolute deviation effect size is used. The same would be true for any comparison, whether for a trial or a simpler cross-sectional design. It seems that there is a clear choice over which effect size to use. The main advantage in using raw scores as an outcome measure is that they are easy to comprehend. However, they might be misleading and so perhaps require more judgement to interpret than traditional ‘effect’ sizes. Among the advantages of using the mean absolute deviation effect size are its relative simplicity, everyday meaning, and the lack of distortion of extreme scores caused by the squaring involved in computing the standard deviation. Given that working with absolute values is no longer the barrier to computation that it apparently was before the advent of digital calculators, there is a clear place for the mean absolute deviation effect size (termed ‘A’).     

知识产权案件“三审合一”对企业创新质量的影响

提升创新质量是我国经济实现高质量发展的关键,知识产权案件“三审合一”改革是提高知识产权司法保护力度、促进知识经济发展的重要举措。本文以“三审合一”改革作为准自然实验,采用倾向得分匹配基础上的“渐进式”双重差分法（PSM-DID）实证评估以“三审合一”改革为代表的知识产权保护制度创新对企业创新质量的影响。结果表明：“三审合一”改革能够显著提升企业的创新质量,这一结论在考虑了识别假设条件和一系列其他可能的干扰因素后依然成立;分析师关注度与研发投入强度在“三审合一”对企业创新质量的促进过程中发挥部分链式中介作用;该促进作用对大型企业、非国有企业和非高技术企业创新质量更加显著。本文为“法与创新”领域提供了新证据,对探寻创新升级新路径以及知识产权司法保护模式改革具有一定参考价值。     

Generalised Geometric Random Variables

Properties of the random variable representing the number of identical and independent Bernoulli trials necessary to obtain K consecutive successes are investigated. The results are interesting to students in a first course in probability or mathematical statistics.     

The importance of specifying and studying causal mechanisms in school-based randomised controlled trials: lessons from two studies of cross-age peer tutoring

Based on the experience of evaluating 2 cross-age peer-tutoring interventions, we argue that researchers need to pay greater attention to causal mechanisms within the context of school-based randomised controlled trials. Without studying mechanisms, researchers are less able to explain the underlying causal processes that give rise to results from randomised controlled trials. Studying implementation fidelity is necessary but not sufficient for causal explanation; the study of causal mechanisms through the application of mixed methods is also required. Due to the increasingly complicated nature of many classroom-based innovations that are subject to evaluation, and the potentially distal nature of hypothesised effects, particularly on attainment, programme theory and articulation of mechanisms are essential in enhancing causal explanation and promoting the accumulation of knowledge of what works and why in classroom settings.     

非布司他治疗中国高尿酸血症的有效性及安全性的Meta分析

目的:系统评价评价非布司他治疗高尿酸血症有效性及安全性。方法:检索1989~2016年国内发表的关于非布司他治疗高尿酸血症的临床疗效及安全性的随机对照试验(RCT)。按既定的纳入与排除标准选择,资料处理采用Rev Man 5.3统计软件做Meta分析处理。结果:研究共纳入5个RCT,各实验的基线均可比。Meta分析结果表明:与应用别嘌醇片方案相比,非布司他可以使血尿酸更容易降至正常范围并且差异有统计学意义。结论:非布司他具有良好的降低血清尿酸作用,并且对肝肾功能的损害更小,不良反应发生率更低,可以指导临床。     

Using Multisite Experiments to Study Cross-Site Variation in Treatment Effects: A Hybrid Approach With Fixed Intercepts and a Random Treatment Coefficient

The present article considers a fundamental question in evaluation research: “By how much do program effects vary across sites?” The article first presents a theoretical model of cross-site impact variation and a related estimation model with a random treatment coefficient and fixed site-specific intercepts. This approach eliminates several biases that can arise from unbalanced sample designs for multisite randomized trials. The article then describes how the approach operates, explores its assumptions, and applies the approach to data from three large welfare-to-work trials. The article also illustrates how to report cross-site impact findings and presents diagnostics for assessing these findings. To keep the article manageable, it focuses on experimental estimates of effects of program assignment (effects of intent to treat), although the ideas presented can be extended to analyses of multisite quasi-experiments and experimental estimates of effects of program participation (complier average causal effects).     

So much research,so little application: Barriers to dissemination and practical implementation of Tai Ji Quan

Despite the large number of articles published in the medical literature advocating the use of Tai Ji Quan for a wide variety of health-related outcomes, there has been little systematic broad-scale implementation of these programs. It may be argued that the lack of funding from organizations capable of implementing and overseeing large-scale programs, such as governmental health agencies or national non-governmental organizations concerned with healthcare for older adults, is to blame. However, the evidence these organizations need to justify underwriting such programs is in short supply because of conflicting priorities and standards related to determining the efficacy and effectiveness of Tai Ji Quan. Establishing efficacy through acceptable designs such as randomized controlled trials involves strict protocols to ensure meaningful internal validity but different approaches are needed to demonstrate meaningful effectiveness (external validity) outside the study setting. By examining the quality, quantity, and relative proportions of the randomized controlled trials, systematic reviews, and dissemination studies reported in the medical literature, this paper highlights the disparity in emphasis between efficacy and effectiveness research that has impeded the development of a cohesive literature on Tai Ji Quan and concludes that until more researchers develop a systematic, long-range commitment to investigating its health-related benefits, the research related to Tai Ji Quan will remain fractured and sporadic, limiting the incentive of large funding agencies to support its wide-spread use.     

远东审判的合法性、正义性及对未来的启迪

60年前开庭的远东审判是文明对野蛮的严正审判。在持续3年的审判中,尽管波澜起伏,最终正义战胜邪恶,战争罪犯得到了应有的惩罚。然而,远东审判在惩处日本战争罪犯和消灭军国主义势力上是不彻底的,最大的遗憾是没有追究天皇裕仁的战争责任。国际社会对日本发动的侵略战争性质,早有明确的结论,远东国际军事法庭的审判是公正的审判。     

The effect of the simultaneous Prompting–Constant time delay instructional sequence on the teaching of early academic skills to students with intellectual disabilities

This study investigated the effectiveness of combining simultaneous prompting (SP) and constant time delay (CTD) (hereafter SP–CTD instructional sequence) in teaching early academic skills to five students with intellectual disabilities (ID). A multiple-probe design was used to examine the functional relationship between the SP and CTD instructional sequence and the acquisition of letter–sound correspondences and number recognition. The results showed the effectiveness of the procedure, as all students acquired their target skills in 7–12 sessions, with an average error rate of 8%. Maintenance effects were assessed at 2, 6, and 10 weeks after the intervention to measure participants’ retention of these acquired skills. Three students preserved their skills with 100% accuracy, while two students did so with more than 90% accuracy. The data indicate that instruction consisting of probe trials, 0-s trials, and 5-s trials leads to improvements in early academic skills among students with ID.