Progress in the past decade: an examination of the precision of cluster randomized trials funded by the U.S. Institute of Education Sciences

ABSTRACT

This article examines the statistical precision of cluster randomized trials (CRTs) funded by the Institute of Education Sciences (IES). Specifically, it compares the total number of clusters randomized and the minimum detectable effect size (MDES) of two sets of studies, those funded in the early years of IES (2002–2004) and those funded in the recent years (2011–2013). Overall, the average precision in terms of MDES of studies in the recent cohort was more than double that of the early cohort (i.e. 0.48 compared to 0.23). The findings suggest a consistent and substantial increase in the precision of CRTs funded by IES in the past decade which is a critical step towards designing studies that have the potential to yield high-quality evidence about the effectiveness of educational interventions.     

Detecting Intervention Effects Across Context: An Examination of the Precision of Cluster Randomized Trials

The Institute of Education Sciences has funded more than 100 experiments to evaluate educational interventions in an effort to generate scientific evidence of program effectiveness on which to base education policy and practice. In general, these studies are designed with the goal of having adequate statistical power to detect the average treatment effect. However, the average treatment effect may be less informative if the treatment effects vary substantially from site to site or if the intervention effects differ across context or subpopulations. This article considers the precision of studies to detect different types of treatment effect heterogeneity. Calculations are demonstrated using a set of Institute of Education Sciences funded cluster randomized trials. Strategies for planning future studies with adequate precision for estimating treatment effect heterogeneity are discussed.     

The function of individualized education programmes in special education: a discussion of premises∗

ABSTRACT

This paper focuses on the construction and implementation of written individual plans for teaching students in special education. Experiences from a number of countries are given, and from these experiences two premises for planning teaching are deduced: one regarding the link between diagnosis and instruction and one regarding teacher planning. The premises are discussed in some detail and consequences for individual planning in special education are formulated.

     

Large-scale change and survival of school principals in Turkey

ABSTRACT

Turkey possess one of the most vibrant educational systems in the World. This study specifically aims at revealing Turkish school principals’ perceptions about the large-scale changes imposed by the Ministry of Education and the strategies they follow for coping with these changes. The study was designed as a generic qualitative study. Twenty principals from different schools at different levels were identified through snowball sampling and semi-structured interviews were conducted with each of them. The decoupling between change interventions and the needed change, inappropriate change implementation process, the passive role of the school principals in the design phase of change interventions are some of the major challenges that school principals encounter. The passivity of the principals during design phase put them into a state of reluctant activeness to find out resources, create readiness and adopt a mediating role between top authority and the teachers in order to implement the imposed changes.     

Teacher collaboration in curriculum design teams: effects,mechanisms, and conditions

ABSTRACT

Collaborative design positively affects both professional development and the implementation of curriculum change, because teachers develop competencies and practice and develop ownership of the change. The current study was aimed to explore what empirical evidence is available about processes that take place when teachers co-design and how these contribute to professional development and curriculum change. Evidence from 14 PhD studies was collected to study their impact on teacher learning and curriculum change, by analysing effects, mechanisms, and conditions. Results showed that effects of curriculum design teams, in terms of learning outcomes for teachers in areas such as (pedagogical) content knowledge and design knowledge and skills, became manifest in the outcomes of the curriculum design process, and in the appreciation by the stakeholders. We concluded that professional development, through collaborative design in teams, which is specific and linked to the curriculum, influences teachers’ knowledge and practice and impacts implementation of curriculum change.     

Optimal Design for Two-Level Random Assignment and Regression Discontinuity Studies

An important concern when planning research studies is to obtain maximum precision of an estimate of a treatment effect given a budget constraint. When research designs have a multilevel or hierarchical structure changes in sample size at different levels of the design will impact precision differently. Furthermore, there will typically be differential costs of enrolling additional units at different levels of the hierarchy. The optimal design problem in multilevel research studies involves determining the optimal sample size at each level of the design given specified design parameters and a specified marginal cost of recruitment at each level. The current work extends existing results by considering optimal design for (a) unbalanced random assignment designs and (b) regression discontinuity designs.     

Are Power Analyses Reported With Adequate Detail? Evidence From the First Wave of Group Randomized Trials Funded by the Institute of Education Sciences

Abstract

This study examines the reporting of power analyses in the group randomized trials funded by the Institute of Education Sciences from 2002 to 2006. A detailed power analysis provides critical information that allows reviewers to (a) replicate the power analysis and (b) assess whether the parameters used in the power analysis are reasonable. Without a detailed power analysis, reviewers may have difficultly evaluating the accuracy of the power analysis and underpowered studies may inadvertently pass through the review process with a recommendation for funding. This study reveals that sample sizes are reported with high consistency; however, other important design parameters, including intraclass correlations, covariate-outcome correlations, and the percentage of variance explained by blocking are not reported with regularity. An analysis of reporting trends over time reveals that the reporting of intraclass correlations and covariate-outcome correlations dramatically increased over time. The reporting of blocking information was still extremely limited, even in the more recent studies.     

Arts practice as method,urban spaces and intra-active faiths

ABSTRACT

This paper examines the research design for an arts-based interfaith research project that is intended to build relationships between children from different faiths and to increase research participants’ understandings of faiths other than their own. The project is funded as an Australian Research Council Future Fellowship called Early Start Arts to Counter Radicalization and has a mixed method approach that brings arts-based workshop groups for children together with focus groups for parents. Early findings demonstrate the utility of art for developing a sense of belonging and self-worth in children and clearly show ways in which art facilitates comment on complex social issues even from primary school age. The nature of such socially engaged arts-based research means it must be developed or, at the least, refined, through engagement with community and social context. As such, consideration of the urban environment that shapes the lives of the young research participants and their families forms part of the discussion undertaken.     

Comprehensive Staff Development Programs: Implications for the Office of Institutional Research and Planning

Abstract

An increasing number of two-year colleges are drawing on an emerging model of comprehensive staff development. The model includes not only instructional improvement, but organizational and personal development as well. In this model, as staff development becomes an integral part of the college, the planning of such a program becomes an essential part of institutional planning.

The office of institutional research and planning should serve as a major resource in the formulation and implementation of the staff development program. Three major areas where this can occur are discussed: setting goals for the program, assessing staff development needs, and evaluating the impact of the program.     

The Power of the Test for Treatment Effects in Three-Level Cluster Randomized Designs

Abstract

Experiments that involve nested structures often assign entire groups (such as schools) to treatment conditions. Key aspects of the design of such experiments include knowledge of the intraclass correlation structure and the sample sizes necessary to achieve adequate power to detect the treatment effect. This study provides methods for computing power in three-level cluster randomized balanced designs (with two levels of nesting), where, for example, students are nested within classrooms and classrooms are nested within schools and schools are assigned to treatments. The power computations take into account nesting effects at the second (classroom) and at the third (school) level, sample size effects (e.g., number of schools, classrooms, and individuals), and covariate effects (e.g., pretreatment measures). The methods are applicable to quasi-experimental studies that examine group differences in an outcome.     

Mindfulness-based meditation to decrease stress and anxiety in college students: A narrative synthesis of the research

ObjectiveThe primary purpose of this paper was to narratively review the research testing the effects of mindfulness meditation on stress and anxiety in the college students; reviewing the inclusion of mindfulness was a secondary purpose.MethodsA literature search resulted in 57 studies on the effectiveness of mindfulness meditation in reducing stress and anxiety in college students.ConclusionsResearchers examined anxiety in 40 studies, self-reported stress in 34, physiological stress in 11, and mindfulness in 24. Thirty-three of 40 and 25 of 34 studies showed significant decreases in anxiety and stress respectively; 22 of 24 showed an increase in mindfulness. Physiological stress had inconsistent results indicating a need for further research. Overall, mindfulness meditation shows promise in reducing stress and anxiety in college students. Additionally, there are a number of differences in mindfulness interventions including frequency, duration, instructional method, and inclusion of yoga, that need quantitative examination (meta-analysis) to determine which is most effective.     

Office work and education 1848–1948

ABSTRACT

This paper, which is based on ESRC‐funded research, draws on quantitative and qualitative data to examine the transformation of the business studies curriculum in secondary schools. It links this transformation to the emergence of the ‘new vocationalism’ and to alleged changes in the nature of work, work organisation and the labour market. The paper argues that the changes which have taken place in the business studies curriculum are the result of interplay between the rhetoric of the new vocationalism, the resources mobilised to support this rhetoric, the activities of various mediating agencies, the characteristics and environments of schools and the professional and career concerns of teachers. The paper ends by considering the place of business studies within the curriculum in the aftermath of the 1988 Education Act.     

Managing Reform: Lessons from Urban High Schools

ABSTRACT

American high schools present an extremely difficult, nearly intractable setting for serious educational reform efforts. Yet such efforts can succeed. This article reports results from a survey of 179 urban comprehensive high schools that were successfully implementing change programs based on effective schools principles.

All schools encountered problems of implementation. Larger‐scale, instructionally‐oriented, and longer‐lasting programs induced more problems. Problems were only moderated, not eliminated, by good‐quality planning.

Given the problem‐richness of change efforts, the major determinant of successful implementation was good problem‐coping. It was made more likely by the presence of consensus (shared vision), support of key stakeholders in and out of the school, planning quality, external assistance, and administrative time spent. Case studies showed that the presence of an empowered change management group aided all these critical factors.

Schools with the most impact (defined as student outcomes, teacher improvement, and organizational improvement) had focused on teacher or organizational changes to begin with, coped with the inevitable problems well, had internal and external constituencies supporting the change effort, and sustained the work for a longer period of time (typically 4‐5 years).

The key variables appear to form a coherent causal network. Implications for change management are drawn.     

The Influence of Nursery School Attendance upon the Behavior and Personality of the Preschool Child

The power of analysis of covariance (ANCOVA) and 2 types of randomized block designs were compared as a function of the correlation between the concomitant variable and the outcome measure, the number of groups, the number of participants, and nominal power. ANCOVA had a small but consistent advantage over a randomized block design with 1 participant in each Block × Treatment combination (RB1). At correlations of .3 or greater, ANCOVA was superior to a randomized block design with n participants per Block × Treatment combination (RBn), with increasing differences as the correlation increased. RBn was superior to the other 2 designs only when the correlation was .2 or less. At those levels, however, the randomized group analysis of variance ignoring the concomitant variable was equally powerful. The findings held regardless of sample size, number of groups, or nominal power.     

The Effects of Team-Based Learning on Social Studies Knowledge Acquisition in High School

Abstract

This randomized control trial examined the efficacy of team-based learning implemented within 11th-grade social studies classes. A randomized blocked design was implemented with 26 classes randomly assigned to treatment or comparison. In the treatment classes teachers implemented team-based learning practices to support students in engaging in dialogue about course content, application of content to solve problems, and use of evidence to support responses. Significant differences in favor of the treatment group on content acquisition were noted (Hedges's g =.19). Examination of differences in response to the treatment indicated groups of students classified with high or moderate pretest scores benefitted from the treatment, whereas a group of students classified with low pretest scores did not benefit from the treatment.     

The implementation of the Bologna process

Abstract

This study identifies the responsibilities of the bodies and institutions involved in the implementation of the Bologna Process. They include the levels of Europe, nations, higher education institutions, departments, degree programmes, teachers and students. The future planning is analysed using the Balanced Scorecard approach designed for the communication and implementation of the strategy. The approach is used to analyse the objectives of European education policy from the different perspectives. Strategic planning and quality assurance are most efficient when they are transformed into action at the closest possible point to teaching and learning.     

Promoting Acceleration of Comprehension and Content Through Text in High School Social Studies Classes

Abstract:

The purpose of this study was to evaluate the efficacy of Promoting Acceleration of Comprehension and Content Through Text intervention implemented with 11th-grade students enrolled in U.S. History classes. Using a within-teacher randomized design, the study was conducted in 41 classes (23 treatment classes) with 14 teachers providing the treatment. Students in the treatment condition performed significantly better than students in the typical instruction comparison condition on a measure of content acquisition at posttest (ES = 0.36), as well as 4 (ES =.22) and 12 (ES =.24) weeks following treatment. There were no differences between treatment and comparison groups on measures of social studies reading comprehension or more general reading comprehension.     

Training the Ethanol Workforce: The Importance of Partners in Niche Program Development

Educational, industry, and state leaders worked together to design a program to meet the training needs of Nebraska's fast-growing ethanol industry. The statewide initiative, guided by Northeast Community College and funded through the President's Community-Based Job Training Grants program, is developing dual-credit, short-term training that leads to a certificate, and degree programs in renewable fuels technology.     

New Empirical Evidence for the Design of Group Randomized Trials in Education

Abstract

This article provides practical guidance for researchers who are designing studies that randomize groups to measure the impacts of educational interventions. The article (a) provides new empirical information about the values of parameters that influence the precision of impact estimates (intraclass correlations and R ² values) and includes outcomes other than standardized test scores and data with a three-level structure rather than a two-level structure, and (b) discusses the error (both generalizability and estimation error) that exists in estimates of key design parameters and the implications this error has for design decisions. Data for the paper come primarily from two studies: the Chicago Literacy Initiative: Making Better Early Readers Study (CLIMBERS) and the School Breakfast Pilot Project (SBPP). The analysis sample from CLIMBERS comprised 430 four-year-old children from 47 preschool classrooms in 23 Chicago public schools. The analysis sample from the SBPP study comprised 1,151 third graders from 233 classrooms in 111 schools from 6 school districts. Student achievement data from the Reading First Impact Study is also used to supplement the discussion.