Analysis of Covariance and Randomized Block Design with Heterogeneous Slopes

The authors compared the Type I error rate and the power to detect differences in slopes and additive treatment effects of analysis of covariance (ANCOVA) and randomized block (RB) designs with a Monte Carlo simulation. For testing differences in slopes, 3 methods were compared: the test of slopes from ANCOVA, the omnibus Block × Treatment interaction, and the linear component of the Block × Treatment interaction of RB. In the test for adjusted means, 2 variations of both ANCOVA and RB were used. The power of the omnibus test of the interaction decreased dramatically as the number of blocks used increased and was always considerably smaller than the specific test of differences in slopes found in ANCOVA. Tests for means when there were concomitant differences in slopes showed that only ANCOVA uniformly controlled Type I error under all configurations of design variables. The most powerful option in almost all simulations for tests of both slopes and means was ANCOVA.     

The Effects of Type I Error Rate and Power of the ANCOVA F Test and Selected Alternatives Under Nonnormality and Variance Heterogeneity

The authors sought to identify through Monte Carlo simulations those conditions for which analysis of covariance (ANCOVA) does not maintain adequate Type I error rates and power. The conditions that were manipulated included assumptions of normality and variance homogeneity, sample size, number of treatment groups, and strength of the covariate-dependent variable relationship. Alternative tests studied were Quade's procedure, Puri and Sen's solution, Burnett and Barr's rank difference scores, Conover and Iman's rank transformation test, Hettmansperger's procedure, and the Puri-Sen-Harwell-Serlin test. For balanced designs, the ANCOVA F test was robust and was often the most powerful test through all sample-size designs and distributional configurations. With unbalanced designs, with variance heterogeneity, and when the largest treatment-group variance was matched with the largest group sample size, the nonparametric alternatives generally outperformed the ANCOVA test. When sample size and variance ratio were inversely coupled, all tests became very liberal; no test maintained adequate control over Type I error.     

The Power of the Test for Treatment Effects in Three-Level Block Randomized Designs

Abstract

Experiments that involve nested structures may assign treatment conditions either to subgroups (such as classrooms) or individuals within subgroups (such as students). The design of such experiments requires knowledge of the intraclass correlation structure to compute the sample sizes necessary to achieve adequate power to detect the treatment effect. This study provides methods for computing power in three-level block randomized balanced designs (with two levels of nesting) where, for example, students are nested within classrooms and classrooms are nested within schools. The power computations take into account nesting effects at the second (classroom) and at the third (school) level, sample size effects (e.g., number of level-1, level-2, and level-3 units), and covariate effects (e.g., pretreatment measures). The methods are generalizable to quasi-experimental studies that examine group differences on an outcome.     

Testing the Intervention Effect in Single-Case Experiments: A Monte Carlo Simulation Study

This article reports on a Monte Carlo simulation study, evaluating two approaches for testing the intervention effect in replicated randomized AB designs: two-level hierarchical linear modeling (HLM) and using the additive method to combine randomization test p values (RTcombiP). Four factors were manipulated: mean intervention effect, number of cases included in a study, number of measurement occasions for each case, and between-case variance. Under the simulated conditions, Type I error rate was under control at the nominal 5% level for both HLM and RTcombiP. Furthermore, for both procedures, a larger number of combined cases resulted in higher statistical power, with many realistic conditions reaching statistical power of 80% or higher. Smaller values for the between-case variance resulted in higher power for HLM. A larger number of data points resulted in higher power for RTcombiP.     

A Simulation Study on the Performance of the Simple Difference and Covariance‐Adjusted Scores in Randomized Experimental Designs

Research by Huck and McLean (1975) demonstrated that the covariance‐adjusted score is more powerful than the simple difference score, yet recent reviews indicate researchers are equally likely to use either score type in two‐wave randomized experimental designs. A Monte Carlo simulation was conducted to examine the conditions under which the simple difference and covariance‐adjusted scores were more or less powerful to detect treatment effects when relaxing certain assumptions made by Huck and McLean (1975) . Four factors were manipulated in the design including sample size, normality of the pretest and posttest distributions, the correlation between pretest and posttest, and posttest variance. A 5 × 5 × 4 × 3 mostly crossed design was run with 1,000 replications per condition, resulting in 226,000 unique samples. The gain score was nearly as powerful as the covariance‐adjusted score when pretest and posttest variances were equal, and as powerful in fan‐spread growth conditions; thus, under certain circumstances the gain score could be used in two‐wave randomized experimental designs.     

Under What Circumstances Does External Knowledge About the Correlation Structure Improve Power in Cluster Randomized Designs?

Abstract

Recent publications have drawn attention to the idea of utilizing prior information about the correlation structure to improve statistical power in cluster randomized experiments. Because power in cluster randomized designs is a function of many different parameters, it has been difficult for applied researchers to discern a simple rule explaining when prior correlation information will substantially improve power. This article provides bounds on the maximum possible improvement in power as a function of a single parameter, the number of clusters at the highest level of a multilevel experiment. The maximum improvement in power is less than 0.05 unless the number of clusters at the highest level is less than 20. Thus, the utility of using prior correlation information is limited to experiments with very small cluster-level sample sizes. Situations where small cluster-level sample sizes could still result in experiments with good statistical power are discussed, as is the relative utility of prior information about intracluster correlations as compared with covariate information that can explain cluster level variability in the outcome.     

A More Powerful Test in Three-Level Cluster Randomized Designs

Abstract

Field experiments that involve nested structures frequently assign treatment conditions to entire groups (such as schools). A key aspect of the design of such experiments includes knowledge of the clustering effects that are often expressed via intraclass correlation. This study provides methods for constructing a more powerful test for the treatment effect in three-level cluster randomized designs with two levels of nesting (at the second and third levels). When the intraclass correlation structure at the second and third level is assumed to be known, the proposed test provides higher estimates of power than those obtained from the typical test based on level-3 unit means, because it preserves the degrees of freedom associated with the number of level-2 and level-1 units. The advantage in power estimates is more pronounced when the number of level-3 units (e.g., schools) is small and the samples are homogeneous (e.g., low-achieving schools).     

“Resistance to Extinction” of Two Patterns of Verbal Reinforcement

Evaluation of large-scale educational programs is problematical because of inherent bias in the assignment of treatment and comparison groups. As a result, the ANOVA design is inapplicable, and even ANCOVA designs can give rise to serious regression artifacts. Data from the Follow Through Program are used to illustrate this point: samples were kindergarteners in the Responsive Education model and in best-match comparison classrooms. The criterion variable was MRT readiness level at posttest. Lord’s True Scores ANCOVA was shown to be a more powerful method in correcting for initial differences than the conventional ANCOVA. These data also were used to illustrate the problem of non-uniformity of program implementation across sites and classrooms. An index of implementation level by classroom was used to predict outcome levels, and the potential of this approach as an adjunct in comparative analysis was discussed.     

Risk-taking in explanation of biological events

Risk-taking by biology students as they attempt to explain observed biological events was investigated. The dependent variable, risk-taking as measured by the Extremity-Confidence of Hypothesis Test and the Risk-Taking Verbal Observation Scale, was analyzed within 2 × 2 and 2 × 3 factorial designs. The independent variables consisted of two levels of achievement motivation and status of information, and three levels of group arrangement for discussion. The results indicate that verbal risk-taking is influenced by differential knowledge of the information and the composition of a discussion group based on the members' measured achievement motivation. No differences were noted in the comparison groups on risk-taking as measured by the Extremity-Confidence of Hypothesis Test. A significant, but low, positive correlation was found to exist between IQ and risk-taking in verbal discourse by individuals. Based on the results of this study, the suggestion that risk-taking is a multidimensional trait is supported, and the magnitude of its expression may be influenced by differential treatments.     

The Power of the Test for Treatment Effects in Three-Level Cluster Randomized Designs

Abstract

Experiments that involve nested structures often assign entire groups (such as schools) to treatment conditions. Key aspects of the design of such experiments include knowledge of the intraclass correlation structure and the sample sizes necessary to achieve adequate power to detect the treatment effect. This study provides methods for computing power in three-level cluster randomized balanced designs (with two levels of nesting), where, for example, students are nested within classrooms and classrooms are nested within schools and schools are assigned to treatments. The power computations take into account nesting effects at the second (classroom) and at the third (school) level, sample size effects (e.g., number of schools, classrooms, and individuals), and covariate effects (e.g., pretreatment measures). The methods are applicable to quasi-experimental studies that examine group differences in an outcome.     

Conversational themes and attention focusing strategies: Predicting comprehension and recall

The specific purpose of this study was to examine the effects of conversational themes and attention focusing strategies on conversational comprehension and recall. A 2×2×2×2 between subjects factorial designs was used to test the research question with 247 participants. The results demonstrated that the unambiguous themes predicted conversational comprehensibility and recall accuracy. The results indicated that personal conversational information may be processed differently from content conversational information. Future conversational processing research should focus on discovering the reasons for the processing differences between content and personal conversational information.     

Applying the Mantel-Haenszel Procedure to Complex Samples of Items

This Monte Carlo study examined the effect of complex sampling of items on the measurement of differential item functioning (DIF) using the Mantel-Haenszel procedure. Data were generated using a 3-parameter logistic item response theory model according to the balanced incomplete block (BIB) design used in the National Assessment of Educational Progress (NAEP). The length of each block of items and the number of DIF items in the matching variable were varied, as was the difficulty, discrimination, and presence of DIF in the studied item. Block, booklet, pooled booklet, and extra-information analyses were compared to a complete data analysis using the transformed log-odds on the delta scale. The pooled booklet approach is recommended for use when items are selected for examinees according to a BIB design. This study has implications for DIF analyses of other complex samples of items, such as computer administered testing or another complex assessment design.     

Factors affecting treatment acceptability for psychostimulant medication versus psychoeducational intervention

Future teachers' judgments of acceptability for two common treatments for children with the Attention Deficit Hyperactivity Disorder (ADHD) label were examined. One hundred forty‐four pre‐service teachers were grouped according to their high school location at graduation (urban vs. rural) and were randomly assigned to read one of four vignettes. The content of the vignettes was held constant but label (ADHD vs no label) and treatments (special education placement vs. Ritalin) were varied. Results indicated a significant main effect on treatment acceptability for High School Location, a Label × Treatment interaction on the attention problems variable, a main effect for Label on the social problems variable, and a High School Location × Treatment interaction on the social problems variable. A number of implications can be made. Observer characteristics such as urban or rural high school experiences may influence judgments about a labeled child more than the characteristics of the child being observed. In this study, the ADHD label evoked greater expectations of attentional difficulties even when the pattern of functioning was similar to nonlabeled children. On the other hand, children with the ADHD label were judged as having better social functioning, which suggests that the ADHD label allows observers to attribute behavioral difficulties or the social problems displayed by these children to some factor that is outside the control of the child, or for which the child does not have personal responsibility. © 2001 John Wiley & Sons, Inc.     

Parallel processing architecture of H.264 adaptive deblocking filters

In H.264, computational complexity and memory access of deblocking filters are variable, dependent on video contents. This paper proposes a VLSI architecture of deblocking filters with adaptive dynamic power, which avoids redundant computations and memory accesses by precluding the blocks that can be skipped. The vertical and horizontal edges are simultaneously processed in an advanced scan order to speed up the decoder. As a result, dynamic power of the proposed architecture can be reduced adaptively （up to about 89%） for different videos, and the off-chip memory access is improved when compared to previous designs. Moreover, the processing capability of the proposed architecture is in particular appropriate for real-time deblocking of high-definition television （HDTV, 1920× 1080 pixels/frame, 60 frames/s video signals） video operation at 62 MHz. Using the proposed architecture, power can be reduced by up to about 89% and processing time by from 25% to 81% compared with previous designs.     

Analysis of Pre‐test‐Post‐test Control Group Designs in Educational Research

Pre‐test‐post‐test control group designs are well suited to investigating effects of educational innovations and are common in educational research. They are frequently analysed by means of an ANOVA on change scores, or, what amounts to the same thing, a repeated measures ANOVA to test the treatment by occasion interaction. Although the analysis of change scores has intuitive appeal, such analyses are often inappropriate. An ANCOVA on post‐test scores, with pre‐test scores as co‐variate usually provides a more appropriate and informative analysis. Advantages of the ANCOVA approach are explained and illustrated using SPSS‐X analyses of fabricated data sets.     

On the Generality of Univariate Eta

Eikeland, H. M. (1971). On the Generality of Univariate Eta. Scand. J. educ. Res. 15, 149‐167. The correlation ratio, or eta, is conceived of as a generic concept, covering varying complexities of data analysis designs, from a simple point‐biserial design to general multiclassification designs, orthogonal as well as nonorthogonal ones. When independent variables are categorical and the dependent variable metric, one can define a multiple eta as the correlation among observed and predicted scores, the prediction being based on two or more classification variables conjointly. Multiple eta is compared to multiple correlation. Worked examples of multiple eta are shown for both the orthogonal and the nonorthogonal case, illustrating the use of analysis of variance as a correlational technique. Only univariate eta is considered, but its two cases, bivariate and multiple eta, are related to the multivariate case.     

When teaching style matches students’ epistemic (in)dependence: The moderating effect of perceived epistemic gap

In a 2×2×2 factorial design, 3rd year Romanian psychology students (N=94) were assigned into 2 groups according to the extent to which they acknowledged an epistemic dependence (lowvs. high) toward their professor. They then compared the competence of 3rd year students to that of 1st year or 5th year students. Finally, they were exposed to a persuasive counter-attitudinal message from an epistemic authority, framed in an authoritarian vs. democratic style. The main dependent variable was the influence of the counter-attitudinal message. Results show an interaction between the three variables. No effects were found among students in the upward social comparison condition in which they felt particularly incompetent. The expected interaction between style and dependence was significant in the down ward comparison condition where participants felt more competent than 1st year students. Students high in perceived epistemic dependence were more influenced by the authoritarian style than those low in epistemic dependence. The reverse tended to be true for participants exposed to the democratic style.     

Testing the Immediate and Long-Term Efficacy of a Tier 2 Kindergarten Mathematics Intervention

ABSTRACT

This study examined the efficacy of a kindergarten mathematics intervention program, ROOTS, focused on developing whole-number understanding in the areas of counting and cardinality and operations and algebraic thinking for students at risk in mathematics. The study utilized a randomized block design with students within classrooms randomly assigned to treatment or control conditions. Measures of mathematics achievement were collected in the fall (pretest) and spring (posttest) in kindergarten and in the winter of first grade (delayed posttest). Significant differences between conditions favoring treatment students were found on four of six measures at posttest. Treatment students reduced the achievement gap with their not-at-risk peers. No effect was found on follow-up first-grade achievement scores. Implications for Tier 2 mathematics instruction in a Response to Intervention model are discussed.     

OFDM系统中峰值功率问题的研究

介绍了OFDM系统的基本原理,并对OFDM系统的峰值功率问题进行研究分析,提出了评估峰值功率的几种算法,以及控制峰值功率问题的2种解决方法:使用块编码方式和相位偏移方法,并且通过模拟和理论分析对这2种方法进行比较。结果表明,相位偏移方式比分组编码方式的CDF性能更好。     

Applications of the Hotelling T2 Statistic to Designs Involving Multiple Measures on Two Dimensions

Multiple measures taken on subjects are usually classified along two dimensions: (1) measures on the same dependent variable taken at different periods of time or occasions; and (2) measures on different dependent variables taken at one testing or observation period. In this paper an appropriate procedure for the analysis of “multivariate repeated measures” designs, i.e. designs in which measures are taken along both dimensions simultaneously, is discussed. Examples are given of the application of the procedure to quasi-experimental time-series designs and to the problem of determining rater agreement when a group of individuals are rated on more than one variable.