The Power of the Test for Treatment Effects in Three-Level Cluster Randomized Designs

Abstract

Experiments that involve nested structures often assign entire groups (such as schools) to treatment conditions. Key aspects of the design of such experiments include knowledge of the intraclass correlation structure and the sample sizes necessary to achieve adequate power to detect the treatment effect. This study provides methods for computing power in three-level cluster randomized balanced designs (with two levels of nesting), where, for example, students are nested within classrooms and classrooms are nested within schools and schools are assigned to treatments. The power computations take into account nesting effects at the second (classroom) and at the third (school) level, sample size effects (e.g., number of schools, classrooms, and individuals), and covariate effects (e.g., pretreatment measures). The methods are applicable to quasi-experimental studies that examine group differences in an outcome.     

The Power of the Test for Treatment Effects in Three-Level Block Randomized Designs

Abstract

Experiments that involve nested structures may assign treatment conditions either to subgroups (such as classrooms) or individuals within subgroups (such as students). The design of such experiments requires knowledge of the intraclass correlation structure to compute the sample sizes necessary to achieve adequate power to detect the treatment effect. This study provides methods for computing power in three-level block randomized balanced designs (with two levels of nesting) where, for example, students are nested within classrooms and classrooms are nested within schools. The power computations take into account nesting effects at the second (classroom) and at the third (school) level, sample size effects (e.g., number of level-1, level-2, and level-3 units), and covariate effects (e.g., pretreatment measures). The methods are generalizable to quasi-experimental studies that examine group differences on an outcome.     

Power Analyses for Moderator Effects in Three-Level Cluster Randomized Trials

Researchers are often interested in whether the effects of an intervention differ conditional on individual- or group-moderator variables such as children's characteristics (e.g., gender), teacher's background (e.g., years of teaching), and school's characteristics (e.g., urbanity); that is, the researchers seek to examine for whom and under what circumstances an intervention works. Furthermore, the researchers are interested in understanding and interpreting variability in treatment effects through moderation analysis as an approach to exploring the sources of the treatment effect variability. This study develops formulas for power analyses to detect the moderator effects in designing three-level cluster randomized trials (CRTs). We develop the statistical formulas for calculating statistical power, minimum detectable effect size difference, and 95% confidence intervals for cluster or cross-level moderation, nonrandomly varying or random slopes, binary or continuous moderators, and designs with or without covariates. We demonstrate how the calculations can be used in the planning phase of three-level CRTs using the software PowerUp!-Moderator.     

Power Analysis in Two-Level Unbalanced Designs

Previous work on statistical power has discussed mainly single-level designs or 2-level balanced designs with random effects. Although balanced experiments are common, in practice balance cannot always be achieved. Work on class size is one example of unbalanced designs. This study provides methods for power analysis in 2-level unbalanced designs with random effects. Overall, the nesting affects power negatively, the treatment affects power positively, and the Level-2 units affect power more than Level-1 units. Computing power assuming balanced designs provides reasonable estimates only when imbalance is mild or moderate. When imbalance is large or extreme, computing power assuming balanced designs produces larger estimates of power. The use of the harmonic mean provides accurate estimates of power in unbalanced 2-level designs even when imbalance is large.     

Statistical Power for School-Based RCTs With Binary Outcomes

Abstract

This article develops a new approach for calculating appropriate sample sizes for school-based randomized control trials (RCTs) with binary outcomes using logit models with and without baseline covariates. The theoretical analysis develops sample size formulas for clustered designs where random assignment is at the school or teacher level using generalized estimating equation methods. The article focuses on the impact parameter pertaining to rates and proportions rather than to the log odds of response, which has been the focus of the previous literature. The article also compiles intraclass correlations (ICCs) for the clustered design for a range of binary outcomes using data from seven education RCTs. These ICCs and the power formulas are then used to conduct a power analysis using a provided SAS macro; the key finding is that sample sizes of 40 to 60 schools that are typically included in clustered RCTs for student test score or behavioral scale outcomes will often be insufficient for binary outcomes. A key reason is that the potential for precision gains from regression adjustment is likely to be smaller for binary outcomes.     

Statistical Power of Randomization Tests Used with Multiple-Baseline Designs

Statistical power was estimated for 3 randomization tests used with multiple-baseline designs. In 1 test, participants were randomly assigned to baseline conditions; in the 2nd, intervention points were randomly assigned; and in the 3rd, the authors used both forms of random assignment. Power was studied for several series lengths (N = 10, 20, 30), several effect sizes (d = 0, 0.5, 1.0, 1.5, 2.0), and several levels of autocorrelation among the errors (p ₁ = 0, .1, .2, .3, .4, and .5). Power was found to be similar among the 3 tests. Power was low for effect sizes of 0.5 and 1.0 but was often adequate (> .80) for effect sizes of 1.5 and 2.0.     

Detecting Intervention Effects Across Context: An Examination of the Precision of Cluster Randomized Trials

The Institute of Education Sciences has funded more than 100 experiments to evaluate educational interventions in an effort to generate scientific evidence of program effectiveness on which to base education policy and practice. In general, these studies are designed with the goal of having adequate statistical power to detect the average treatment effect. However, the average treatment effect may be less informative if the treatment effects vary substantially from site to site or if the intervention effects differ across context or subpopulations. This article considers the precision of studies to detect different types of treatment effect heterogeneity. Calculations are demonstrated using a set of Institute of Education Sciences funded cluster randomized trials. Strategies for planning future studies with adequate precision for estimating treatment effect heterogeneity are discussed.     

More Powerful Tests of Simple Interaction Contrasts in the Two-Way Factorial Design

For the two-way factorial design in analysis of variance, the current article explicates and compares three methods for controlling the Type I error rate for all possible simple interaction contrasts following a statistically significant interaction, including a proposed modification to the Bonferroni procedure that increases the power of statistical tests for deconstructing interaction effects when they are of primary substantive interest. Results indicate the general superiority of the modified Bonferroni procedure over Scheffé and Roy-type procedures, where the Bonferroni and Scheffé procedures have been modified to accommodate the logical implications of a false omnibus interaction null hypothesis. An applied example is provided and considerations for applied researchers are offered.     

Understanding Statistical Power in Cluster Randomized Trials: Challenges Posed by Differences in Notation and Terminology

Abstract

In this article I review the use of randomized field trials to evaluate school voucher interventions. I argue that although estimates of the effect of the voucher offer on achievement are unbiased in these trials, more specific interpretations such as the effect of attending private school may be difficult to obtain. I discuss several evaluation parameters in the context of voucher experiments, with particular attention to the theoretical assumptions needed to identify private school impacts. I argue that if there are direct effects of the voucher offer on student achievement, and compliance with voucher assignment is imperfect, instrumental variable estimates are likely to overstate the extent to which achievement gains may be attributable to private schooling.     

关于我国现代宾馆室内设计的新思考

改革开放以后，我国宾馆室内设计水平得到长足发展。但同时也出现许多问题与不足。本文针对当前宾馆室内设计中存在的误区，对注重人性设计和生态设计方面做了进一步的思考，并对未来的设计走向做了分析。     

西北少数民族地区三级课程实施的进一步思考

三级课程的提出为西北少数民族地区课程反映民族文化、满足不同民族学生发展需要提供了平台,但其初步的实施却并不尽如人意。西北少数民族地区三级课程应该进一步根据课程实施地区与对象的具体情况进行开发,满足民族地区以及学生的发展要求。     

德国政治教育课程设计分析

政治课在德国学校政治教育中发挥着主导作用,承载着重要的社会政治职能.政治课特殊的教学目标和任务对课程设计提出了很高的要求,课程设计对政治课教学能否取得良好效果至关重要.本文试对德国政治教育课程设计的一般阶段及其特点进行分析,希望能对我国政治课教学有所启示.     

Under What Circumstances Does External Knowledge About the Correlation Structure Improve Power in Cluster Randomized Designs?

Abstract

Recent publications have drawn attention to the idea of utilizing prior information about the correlation structure to improve statistical power in cluster randomized experiments. Because power in cluster randomized designs is a function of many different parameters, it has been difficult for applied researchers to discern a simple rule explaining when prior correlation information will substantially improve power. This article provides bounds on the maximum possible improvement in power as a function of a single parameter, the number of clusters at the highest level of a multilevel experiment. The maximum improvement in power is less than 0.05 unless the number of clusters at the highest level is less than 20. Thus, the utility of using prior correlation information is limited to experiments with very small cluster-level sample sizes. Situations where small cluster-level sample sizes could still result in experiments with good statistical power are discussed, as is the relative utility of prior information about intracluster correlations as compared with covariate information that can explain cluster level variability in the outcome.     

宽带压缩真空下运动梯型三能级原子的辐射压力

主要研究了宽带压缩真空下,双色驱动激光场对运动梯型三能级原子的辐射压力及其稳态表达式。从宽带压缩真空下,双色激光场对运动梯型三能级原子的驱动的主方程出发,得到了原子密度矩阵元!ij的布洛赫方程;利用合适的标度化变量,得到了关于原子布居的布洛赫方程,再由原子动量的海森堡运动方程,利用绝热近似重要条件,得出平圴辐射压力的表达式;考虑到稳态下,原子布居数不随时间变化以及"11 "22 "33=1的条件,得出原子的辐射压力的稳态表达式。     

工程力学教学中的几点设计

专业基础课与学生的学习兴趣和能力之间有一定的距离,如何能够唤起并始终保持学生适度的学习兴趣,需要教师对教学的方方面面进行精心设计.     

漳州非遗文化在钟表设计中的开发策略

漳州非物质文化遗产丰富,同时又是"中国钟表之城",开发具有漳州非遗文化特色的钟表产品,有利于本土非遗文化的传承发展,促进本土非遗文化和区域经济的协同发展,对助力漳州非遗文化系列钟表品牌的建立具有一定的现实意义。从文化融合和产品开发的角度探讨漳州非遗文化与钟表设计的结合,提出了相应的思路和方法。     

试论期刊设计中色彩的市场效应

色彩是期刊设计中最为直观也最具想象力的因素 ,它以其自身的机能和内在规律 ,在现代期刊装帧设计中发挥着愈来愈重要的作用 ,尤其体现在市场效应上 ,色彩的意义是其他形式不可替代的 .以几种杂志的装帧设计为例 ,具体阐述了色彩在期刊设计中的市场效应     

辽东山区的景观变化与景观设计

针对目前辽东山区的基质、斑块和廊道现状，并考虑它的历史演替进程，提出各自的景观设计方案，使之向着更有利于经济持续，生态持续的目标变化。     

Effects of Participation in After-School Programs for Middle School Students: A Randomized Trial

Abstract

This study assessed the effects of attending an after-school program (ASP) on a range of outcomes for middle school youths. The program operated for 9 hr per week for 30 weeks and included attendance monitoring and reinforcement, academic assistance, a prevention curriculum, and recreational programming. Participants were 447 students randomly assigned either to the ASP or to after-school activities as usual. Program attendance was sporadic. Although treatment students experienced increased exposure to ASPs relative to controls, nearly all youth in both groups reported participating in some organized activity during the after-school hours, and the number of different activities in which youths reported being involved did not differ across groups. Participation in the treatment reduced time spent with friends with no adults present during the after-school hours. No differences between treatment and control youths were found on measures of conduct problems, academic performance, school attendance, or any of the intermediate behaviors and attitudes targeted. The study replicates findings from the national evaluation of the 21st Century Community Learning Centers Program for middle school students using a more rigorous research design.