The causal effect of an extra year of schooling on skills and knowledge in Latin America. Evidence from PISA

Abstract

In this paper, we estimate the causal effect of an extra year of schooling on mathematics performance for seven Latin American countries based on PISA 2012. To that end we exploit exogenous variation in students’ birthdates around the school entry cut-off date using both sharp and fuzzy Regression Discontinuity designs. We find strong effects of an extra year of schooling in most countries, which amount to a 30% increase in PISA test scores in Brazil, 18% in Uruguay, 7% in Argentina and 6% in Costa Rica. These effects differ from the typical estimates obtained from simple regressions or multilevel models and are large enough to allow 15-year-old students to reach higher proficiency levels, suggesting significant potential gains of reducing dropout rates in the region. Finally, we stress the importance of taking into account the effects of school entry cut-off dates on PISA samples to avoid making unfair international comparisons.     

Brief Research Report: Bayesian Versus REML Estimations With Noninformative Priors in Multilevel Single-Case Data

Abstract

Recently, researchers have used multilevel models for estimating intervention effects in single-case experiments that include replications across participants (e.g., multiple baseline designs) or for combining results across multiple single-case studies. Researchers estimating these multilevel models have primarily relied on restricted maximum likelihood (REML) techniques, but Bayesian approaches have also been suggested. The purpose of this Monte Carlo simulation study was to examine the impact of estimation method (REML versus Bayesian with noninformative priors) on the estimation of treatment effects (relative bias, root mean square error) and on the inferences about those effects (interval coverage) for autocorrelated multiple-baseline data. Simulated conditions varied with regard to the number of participants, series length, and distribution of the variance within and across participants. REML and Bayesian estimation led to estimates of the fixed effects that showed little to no bias but that differentially impacted the inferences about the fixed effects and the estimates of the variances. Implications for applied researchers and methodologists are discussed.     

New Confidence Intervals and Bias Comparisons Show That Maximum Likelihood Can Beat Multiple Imputation in Small Samples

When analyzing incomplete data, is it better to use multiple imputation (MI) or full information maximum likelihood (ML)? In large samples ML is clearly better, but in small samples ML’s usefulness has been limited because ML commonly uses normal test statistics and confidence intervals that require large samples. We propose small-sample t-based ML confidence intervals that have good coverage and are shorter than t-based confidence intervals under MI. We also show that ML point estimates are less biased and more efficient than MI point estimates in small samples of bivariate normal data. With our new confidence intervals, ML should be preferred over MI, even in small samples, whenever both options are available.     

The Severity of State Truancy Policies and Chronic Absenteeism

ABSTRACT

Since the Common School era, states have maintained truancy laws to ensure that students attend school. However, we know little about the severity of these laws and their relationship to student outcomes, particularly absenteeism. In this study, we survey state education statutes to document the severity of truancy policies. We estimate the relationship between these truancy policies and rates of chronic absenteeism via multilevel models using data on all U.S. public schools from the Office of Civil Rights. We find that most students live in states where they may be referred to legal authorities for levels of absenteeism below federal standards. We find little evidence that states with truancy policies have lower chronic absenteeism after adjusting for demographic and other characteristics. However, while non-White high-school students are overrepresented among chronic absences overall, they have rates of chronic absenteeism comparable to their White peers in states with less severe truancy policies.     

Engineering design skills coverage in K-12 engineering program curriculum materials in the USA

ABSTRACT

The current K-12 Science Education framework and Next Generation Science Standards (NGSS) in the United States emphasise the integration of engineering design in science instruction to promote scientific literacy and engineering design skills among students. As such, many engineering education programmes have developed curriculum materials that are being used in K-12 settings. However, little is known about the nature and extent to which engineering design skills outlined in NGSS are addressed in these K-12 engineering education programme curriculum materials. We analysed nine K-12 engineering education programmes for the nature and extent of engineering design skills coverage. Results show that developing possible solutions and actual designing of prototypes were the highly covered engineering design skills; specification of clear goals, criteria, and constraints received medium coverage; defining and identifying an engineering problem; optimising the design solution; and demonstrating how a prototype works, and making iterations to improve designs were lowly covered. These trends were similar across grade levels and across discipline-specific curriculum materials. These results have implications on engineering design-integrated science teaching and learning in K-12 settings.     

The Effect of Scale Referent on Tests of Mean Structure Parameters

Structured means analysis is a very useful approach for testing hypotheses about population means on latent constructs. In such models, a z test is most commonly used for testing the statistical significance of the relevant parameter estimates or of the differences between parameter estimates, where a z value is computed based on the asymptotic standard error estimate associated with the parameter of interest. In the current article, a series of population analyses demonstrate that the z tests for latent mean structure parameters or, more directly, the standard error estimates upon which those z tests are based are, not invariant to how factors are scaled. As such, circumstances exist in which latent mean inference is compromised solely as a result of scaling decisions. This problem is illustrated in the context of between-subjects (i.e., multisample) latent means models and within-subjects latent means models. Recommendations for practice are also offered.     

Social and Dimensional Comparisons in the Formation of German Students’ Language Self-Concepts

Abstract

This study extended the internal/external reference model to multiple languages including students’ language of instruction, first foreign language, and second foreign language. We examined whether social and dimensional comparisons play similar roles in the formation of students’ self-concept related to different languages and whether dimensional comparisons result in contrast or assimilation effects. All students had German as the language of instruction and English as the first foreign language. Students were divided into a subsample (N?=?487) learning French as a second foreign language and a subsample (N?=?481) learning Latin. Invariance tests demonstrated that the achievement–self-concept relations were similar across the subsamples, but interesting group differences became apparent when analyzing the French and Latin subsamples separately.     

MplusAutomation: An R Package for Facilitating Large-Scale Latent Variable Analyses in Mplus

MplusAutomation is a package for R that facilitates complex latent variable analyses in Mplus involving comparisons among many models and parameters. More specifically, MplusAutomation provides tools to accomplish 3 objectives: to create and manage Mplus syntax for groups of related models; to automate the estimation of many models; and to extract, aggregate, and compare fit statistics, parameter estimates, and ancillary model outputs. We provide an introduction to the package using applied examples including a large-scale simulation study. By reducing the effort required for large-scale studies, a broad goal of MplusAutomation is to support methodological developments in structural equation modeling using Mplus.     

Collaborative teaming and critical reflection: A model for practice in preservice programs and general education settings

Abstract

Inclusion settings for young children with mild to moderate educational needs are becoming the norm. Development of collaborative intervention teams has become essential to effective inclusion. Models of collaborative team development have described multiple aspects of team developmental stages (Tuckman & Jensen, 1977; Lowe & Herranen, 1982). What may be missing in these professional models is the understanding of intrinsic personal qualities that may support or inhibit team development. The model proposed in this reflection advocates a combination of professional and personal commitment models of team development. The authors also explore the benefits of critical reflection about team processes and the value of developing collaborative preservice preparation programs and collaborative intervention models at the preschool and primary levels.     

Assessing Differential Prediction of College Grades by Race/Ethnicity with a Multilevel Model

Previous research notes the importance of understanding racial/ethnic differential prediction of college grades across multiple institutions. Institutional variation in selection indices is especially important given some states' laws governing public institutions' admissions decisions. This paper employed multilevel moderated multiple regression to study the variation of selection indices across 30 institutions and the accuracy of selection indices in predicting college grades for students of different racial/ethnic backgrounds. Several benefits of multilevel models for cross-institutional differential prediction studies were described and include: controlling for institutional differences in range restriction, providing reliability estimates of least squares estimates, and adjusting criterion scores for differences in coursework difficulty. The findings from this study provide evidence of institutional variation in selection indices, which challenges current laws aimed at standardizing them. Specifically, there was evidence that the predictor slope coefficients varied across institutions, in addition to the estimates that measured intercept differences for African and Asian American students. Across universities, the results mirrored previous findings: high school grade point average (GPA) differentially predicted grades for African Americans, SAT verbal scores differentially predict grades for Asian Americans, and SAT math scores were better predictors of Asian Americans' grades.     

Directed acyclic graphs: An under-utilized tool for child maltreatment research

BackgroundChild maltreatment research involves modeling complex relationships between multiple interrelated variables. Directed acyclic graphs (DAGs) are one tool child maltreatment researchers can use to think through relationships among the variables operative in a causal research question and to make decisions about the optimal analytic strategy to minimize potential sources of bias.ObjectiveThe purpose of this paper is to highlight the utility of DAGs for child maltreatment research and to provide a practical resource to facilitate and support the use of DAGs in child maltreatment research.ResultsWe first provide an overview of DAG terminology and concepts relevant to child maltreatment research. We describe DAG construction and define specific types of variables within the context of DAGs including confounders, mediators, and colliders, detailing the manner in which each type of variable can be used to inform study design and analysis. We then describe four specific scenarios in which DAGs may yield valuable insights for child maltreatment research: (1) identifying covariates to include in multivariable models to adjust for confounding; (2) identifying unintended effects of adjusting for a mediator; (3) identifying unintended effects of adjusting for multiple types of maltreatment; and (4) identifying potential selection bias in data specific to children involved in the child welfare system.ConclusionsOverall, DAGs have the potential to help strengthen and advance the child maltreatment research and practice agenda by increasing transparency about assumptions, illuminating potential sources of bias, and enhancing the interpretability of results for translation to evidence-based practice.     

An Evaluation of the Linearity of Curriculum‐Based Measurement of Oral Reading (CBM‐R) Progress Monitoring Data: Idiographic Considerations

Educators use curriculum‐based measurement of oral reading (CBM‐R) to measure student response to instruction. Current decision‐making frameworks assume students demonstrate linear growth across a school year. However, growth appears nonlinear for at least a subgroup of students. We assessed the degree to which grade two (n = 800) and grade three (n = 800) students receiving intensive interventions experienced discontinuous growth. We also explored when discontinuous growth tended to occur, and whether students improved or regressed afterward. Results indicate linear patterns were adequate for most students (80 percent). Students who showed discontinuous growth early tended to improve afterward. Conversely, students who showed discontinuous growth later tended to plateau. Findings suggest multilevel models may obscure variability in growth patterns. Practice and research implications are discussed.     

Effectiveness of Schema-Based Instruction for Improving Seventh-Grade Students’ Proportional Reasoning: A Randomized Experiment

Abstract

This study examined the effect of schema-based instruction (SBI) on 7th-grade students’ mathematical problem-solving performance. SBI is an instructional intervention that emphasizes the role of mathematical structure in word problems and also provides students with a heuristic to self-monitor and aid problem solving. Using a pretest-intervention–posttest-retention test design, the study compared the learning outcomes for 1,163 students in 42 classrooms who were randomly assigned to treatment (SBI) or control condition. After 6 weeks of instruction, results of multilevel modeling indicated significant differences favoring the SBI condition in proportion problem solving involving ratios/rates and percents on an immediate posttest (g = 1.24) and on a 6-week retention test (g = 1.27). No significant difference between conditions was found for a test of transfer. These results demonstrate that SBI was more effective than students’ regular mathematics instruction.     

Cooperative Learning and Literacy: A Meta-Analytic Review

Abstract

We conducted a meta-analysis on the effectiveness of cooperative and collaborative learning to support enhanced literacy outcomes. Interventions considered were provided in regular education settings (i.e., not pull-out instruction) with students from Grades 2 through 12. Reviewing more than 30 years of literacy research, we located 18 intervention studies with 29 study cohorts. Included studies primarily used standardized assessments to report on students’ reading, vocabulary, or comprehension achievement, which we analyzed separately. Overall, students had significantly higher literacy achievement scores when instructional interventions utilized cooperative and collaborative activity structures. The overall weighted mean effect sizes ranged from 0.16 to 0.22 (p < .01) with more than 94% of the point estimates being positive. Because cooperative or collaborative learning was always one of multiple intervention components, it was impossible to estimate the unique, added effects of cooperative/collaborative learning. Although the small number of eligible studies precludes any claims about the effectiveness of specific forms of grouping and the circumstances under which programs have more impact, our findings suggest that cooperative and collaborative grouping was a core component of effective literacy interventions, particularly at the elementary level.     

The Effects of Oral Narrative Intervention on the Personal Narratives of Children with ASD and Severe Language Impairment: A Pilot Study

Abstract

The purpose of this pilot study was to examine and refine an oral narrative intervention addressing personal narratives of children with autism spectrum disorder (ASD) and severe language impairment. A multiple baseline across participants design investigated the effect of the intervention on the macrostructure of personal narratives. Three five–six year olds participated in a 1:1 intervention that targeted the where, who with, what and feelings of personal narrative. Intervention included macrostructure icons, pictures, modelling and participants telling the entire narrative. Participants received training with two or three narratives each session. The intervention was effective for all three participants. Two participants showed evidence of maintenance and generalisation across settings. The results show that children with autism and severe language impairment may benefit from oral narrative intervention targeting the macrostructure of personal narrative. These results are consistent with previous findings. Areas for future research include investigation of generalisation across people.     

Entering the Fray: The role of outdoor education in providing nature-based experiences that matter

Abstract

This article draws on different bodies of knowledge in order to review the potential role of outdoor education in providing nature-based experiences that might contribute to sustainable living A pragmatic perspective is adopted to critique what outdoor education is,and then what it might be. Phenomenology is used to challenge the belief that there is a causal relationship between activities and learning outcomes but foremost to consider what it is to be in nature in the first place. Aspects of both realism and social constructionism are presented as essential to environmental philosophy and the concomitant, but contested, relationship between people and planet. Through these multiple realities the moral significance of nature emerges not only as a theoretical consideration but as a practical one too. In this way I challenge dualisms that provide stumbling blocks to practice and celebrate instead pluralistic thinking where starting points are based on real-life work settings where theory and practice can emerge together through place-specific solutions.     

Toddlers’ peer engagement in Swiss childcare: contribution of individual and contextual characteristics

The present study observed 86 three-year-old children (M = 43.7, SD = 6.4) from 15 Swiss childcare groups, to investigate multiple individual and contextual contributions to toddlers’ positive engagement with peers. The children’s individual characteristics (age, sex and social skills) and childcare-related predictors (emotional and behavioural support from caregivers, and structural group features) were assessed. We employed the child-by-environment perspective and tested the hypothesis that high-quality behavioural and emotional support provided by caregivers benefits children with deficits in social abilities. Results of the multilevel structural equation modelling indicated that toddlers rated by caregivers as sociable and assertive showed more positive situation-specific peer engagement, especially with a concomitant higher quality of caregiver emotional and behavioural support. By contrast, being prosocial-cooperative was negatively associated with observed peer engagement. Thus, children’s social skills were found to be the most important factor for peer engagement in childcare settings. Important suggestions for future research are made, and practical implications are discussed.

     

Does learning begin at home? The use of ‘home international’ comparisons in UK policy making

Overseas comparisons have been increasingly influential in UK policy making for education and training, but they have limitations as a source of policy lessons. ‘Home international’ comparisons of the four home countries of the UK have been advocated as a more fruitful source of practical policy lessons. This paper examines the extent to which policy‐making processes in the four UK territories facilitate policy learning from such comparisons, drawing on interviews with senior policy makers. The policy makers agreed that home international comparisons were potentially valuable, but their actual use in policy making was occasional and unsystematic. The paper discusses the features of the policy‐making process which account for this conclusion.     

A “can-do” Attitude for Students with Disabilities: Special Education in Rural Charter Schools

ABSTRACT

Charter schools are often the only alternative to traditional public schools for students with disabilities in rural communities. Part of a larger study, we employed a qualitative multiple case study design to examine special education practices in five rural charter schools to understand the services provided, barriers to providing these services, and any novel practices used to provide these services. Data analysis revealed the following themes: (a) intentional school models facilitated individualized approaches, (b) schools responded to tight labor supply by prioritizing retention and making efficient use of staff time, and (c) schools demonstrate flexibility and willingness to problem solve.