The causal effect of an extra year of schooling on skills and knowledge in Latin America. Evidence from PISA

Abstract

In this paper, we estimate the causal effect of an extra year of schooling on mathematics performance for seven Latin American countries based on PISA 2012. To that end we exploit exogenous variation in students’ birthdates around the school entry cut-off date using both sharp and fuzzy Regression Discontinuity designs. We find strong effects of an extra year of schooling in most countries, which amount to a 30% increase in PISA test scores in Brazil, 18% in Uruguay, 7% in Argentina and 6% in Costa Rica. These effects differ from the typical estimates obtained from simple regressions or multilevel models and are large enough to allow 15-year-old students to reach higher proficiency levels, suggesting significant potential gains of reducing dropout rates in the region. Finally, we stress the importance of taking into account the effects of school entry cut-off dates on PISA samples to avoid making unfair international comparisons.     

Grades and Graduation: A Longitudinal Risk Perspective to Identify Student Dropouts

ABSTRACT

Studies of student risk of school dropout have shown that present predictors of at-risk status do not accurately identify a large percentage of students who eventually drop out. Through the analysis of the entire Grade 1–12 longitudinal cohort-based grading histories of the class of 2006 for two school districts in the United States, the author extends past longitudinal conceptions of dropout to a longitudinal risk perspective, using survival analysis, life tables, and discrete-time hazard modeling to appropriately account for student graduation, transfer, or dropout. The risk of dropout began in Grade 7, with the most hazardous years at Grades 8 and 11. A novel calculation of teacher-assigned grades, noncumulative GPA, is identified as a strong predictor of student dropout.     

The Effects of Busing and School Desegregation on Majority and Minority Student Dropout Rates

Abstract

This study evaluates the effects of school desegregation by court-ordered busing on the subsequent dropout rate of majority and minority students. Using before and after busing measures of dropout rates, school records and personal interviews, this research finds majority dropout rates are not affected by desegregation procedures. While the dropout rates of bused minority students appear to be identical to those of non-bused minority students, large disparities between minority rates in various bused sectors indicate highly uneven educational experiences of bused minority students. School socio-economic composition and the expectations of teachers concerning student behavior are used to analyze the disparities, with the conclusion reached that the more favorable expectations of teachers at higher socio-economic climate schools produce lower minority student dropout rates. Desegregation produces a positive benefit for this most crucial dimension of minority student educational accomplishment, when the school to which the minority student is bused is one where teachers' expectations are positive and supportive.     

Temporal development of student burnout symptoms: Sociodemographic differences and linkage to university dropout intentions

Considering the demanding higher education context, university students are at risk to experience burnout symptoms such as emotional exhaustion, cynicism, and reduced professional efficacy. Theoretical models (e.g., Job Demands-Resources Model) state that burnout symptoms develop over time, vary between individuals, and bear adverse consequences. To date, the temporal development of student burnout symptoms within semesters, inter-individual differences in their development, and the role of their development for academic outcomes like student dropout are understudied. To complement this limited research, we used date of a three-wave longitudinal study with N = 1435 undergraduate students over one semester. First, we modeled unconditional second-order latent growth curves to examine the initial levels and trajectories of each burnout symptom (emotional exhaustion, cynicism, and reduced professional efficacy). We then modeled conditional second-order latent growth curves to examine whether initial levels and trajectories differed depending on students’ sociodemographic characteristics (gender, age, study progress, and academic major) and associated with students’ intentions to drop out from university. Results indicated a linear increase in student burnout symptoms over the semester. Sociodemographic differences existed in the initial levels of emotional exhaustion and reduced professional efficacy and the trajectories of each symptom. Generally, female, STEM, and higher- and lower- semester students were particularly affected by burnout symptoms. Further, higher initial levels and an increase in each symptom corresponded with higher dropout intentions. Summarizing, the results highlighted that student burnout symptoms increase as the semester progresses, seem more pronounced among certain student groups, and operate as strong predictors for dropout intentions.     

Assessing Methods for Generalizing Experimental Impact Estimates to Target Populations

ABSTRACT

Randomized experiments are considered the gold standard for causal inference because they can provide unbiased estimates of treatment effects for the experimental participants. However, researchers and policymakers are often interested in using a specific experiment to inform decisions about other target populations. In education research, increasing attention is being paid to the potential lack of generalizability of randomized experiments because the experimental participants may be unrepresentative of the target population of interest. This article examines whether generalization may be assisted by statistical methods that adjust for observed differences between the experimental participants and members of a target population. The methods examined include approaches that reweight the experimental data so that participants more closely resemble the target population and methods that utilize models of the outcome. Two simulation studies and one empirical analysis investigate and compare the methods’ performance. One simulation uses purely simulated data while the other utilizes data from an evaluation of a school-based dropout prevention program. Our simulations suggest that machine learning methods outperform regression-based methods when the required structural (ignorability) assumptions are satisfied. When these assumptions are violated, all of the methods examined perform poorly. Our empirical analysis uses data from a multisite experiment to assess how well results from a given site predict impacts in other sites. Using a variety of extrapolation methods, predicted effects for each site are compared to actual benchmarks. Flexible modeling approaches perform best, although linear regression is not far behind. Taken together, these results suggest that flexible modeling techniques can aid generalization while underscoring the fact that even state-of-the-art statistical techniques still rely on strong assumptions.     

Large qualitative sample and thematic analysis to redefine student dropout and retention strategy in open online education

Open online education experiences persistently high dropout rates, and the efficacy of dropout interventions has been questioned. Despite considerable research, dropout reasons are not fully understood, and further in-depth investigation has been called for. Prior qualitative retention studies have frequently relied on smaller samples that are unable to generate deeper appreciation of dropout reasons. Over 200 in-depth interviews were therefore conducted with students that had dropped out of open online education. The probability-based qualitative sample facilitated capture of subthemes down to a 5% incidence level or frequency of occurrence. Thematic analysis revealed 41 subthemes within 10 broad dropout reasons. While the broad dropout themes have been identified previously, the subthemes are new and provide richer understanding. This study also captured students' suggestions for what might have prevented their dropout. Thematic analysis identified 19 subthemes within 5 broad intervention themes that respond to the root dropout causes. Many intervention subthemes address personal and learner context dropout factors that have often been considered uncontrollable and unavoidable. This paper therefore redefines dropout in open online education and offers new insights for improving retention. It also provides a strategic framework for evaluating dropouts and prioritising student-informed interventions that respond to the main dropout causes.

Practitioner notes

What is already known about this topic

• Persistently high open online education dropout suggests existing interventions are ineffective.
• Prior qualitative retention studies identified the main dropout reasons, but small sample sizes failed to generate deeper insights and more in-depth investigation has been called for.

What this paper adds

• A probability-based qualitative sample of 226 participants captured dropout subthemes down to a 5% level of incidence or frequency of occurrence.
• Thematic analysis identified 41 subthemes, within 10 broad dropout reasons. These subthemes are new and offer richer understanding.
• Thematic analysis also identified 19 subthemes within 5 broad dropout intervention areas that students suggested could have prevented their dropout. These include new insights for addressing dropout causes that have often previously been considered unavoidable.

Implications for practice and/or policy

• The strategic framework provides a retention management approach that prioritises responding to the main dropout causes with student-informed interventions.
• This approach and the deeper understanding afforded by robust qualitative investigation should help reduce persistently high dropout rates.

     

Exploring the Effects of Financial Aid on the Gap in Student Dropout Risks by Income Level

Using national survey data and discrete-time logit modeling, this research seeks to understand whether student aid mediates the relationship between parental income and student dropout behavior. Our analysis confirms that there is a gap in dropout rates for low-income students compared with their upper income peers, and suggests that some types of aid are associated with lower risks of dropout. Thus, we examine the interaction between financial aid type and parental income to explore whether, and if so how, different types of aid may reduce the dropout gap by income level group. We find that the receipt of a Pell grant is related to narrowing the dropout gap between students from low- and middle-income groups, although overall the interaction between Pell grant and income is not significant. Loans and work-study aid both have similar effects on student dropout across all income groups. Methodologically, our results demonstrate the need to model dropout behavior temporally and to avoid main-effect bias by incorporating interaction effects.

Contrasting school dropout: The protective role of perceived teacher justice

BackgroundSchool dropout is a serious issue with high individual and societal costs. Although numerous risk factors have been studied, those related to the proximal learning environment have mostly been neglected.AimIn this study we tested whether a feature of the learning environment, i.e., students’ perception of being treated fairly by teachers, could reduce their intention to drop out.Sample547 9th grade students (56% male, 94% born in Italy, M_age = 13.92).MethodParticipants answered a questionnaire on teacher justice and intentions to drop out at the beginning (T1), middle (T2) and end (T3) of the school year.ResultsA latent growth model indicated that intentions to dropout increased over time and perceptions of teacher justice had a negative association with the increase both at T2 and T3, even after controlling for gender, nationality, motivation, and perceived learning difficulty.ConclusionThese findings suggest that teacher justice can play a relevant protective role against early school leaving.     

Who makes it into PISA? Understanding the impact of PISA sample eligibility using Turkey as a case study (PISA 2003–PISA 2012)

ABSTRACT

New evidence presented in this paper shows that existing accounts of Programme for International Student Assessment (PISA) in developing countries underestimate both progress and inequality because they do not take into account the large proportion of 15–16-year olds who are ineligible for the PISA sampling frame, primarily due to dropout or delay. In such countries, who makes it into PISA (sample eligibility) heavily influences representivity and therefore the interpretation of PISA results. We use PISA in Turkey 2003–2012 as a case study to show how combining measures of access (from the Turkish Demographic and Health Surveys) and learning outcomes (from PISA) can account for changes in the underlying population. We find the percentage of 15–16-year olds reaching Level 2 in PISA is up to twice as large as that reflected in official PISA reports. These findings have strong implications for inter-country and inter-temporal comparisons using PISA, particularly for developing countries.     

The childhood roots of adult psychological distress: Interdisciplinary perspectives toward a better understanding of exposure to cumulative childhood adversity

BackgroundAlthough most children experience at least one adversity, it is the experience of multiple adversities that produces a context of disadvantage that increases the risk of various negative outcomes in adulthood. Previous measures of cumulative childhood adversity consider a limited number of adversities, overlook potential differences across experiences of adversity, and fail to measure the effects of multiple co-occurring childhood adversities. These limitations have led to inconsistent and incomplete conclusions regarding the impact of multiple adverse childhood experiences on adult mental health.ObjectiveThis study assesses how the operationalization and modeling of exposure to cumulative childhood adversity (CCA) influences estimates of the association between CCA and adult psychological distress and develops an improved measure of CCA.MethodsWe use data from the Panel Study of Income Dynamics, a nationally representative sample of households in the United States, and its supplement, the Childhood Retrospective Circumstances Study (N = 4219). We compare four measures of CCA that consider various distinct aspects of adverse experiences (additive, severity, type, and patterns of experience using latent class analysis).ResultsAll measures of CCA were associated with increases in adult psychological distress, but effects depend on the measurement of CCA. Results suggest the sum score overestimates the overall impact of CCA. Latent class analysis captures the co-occurrence of adversities across severity and type, providing an improved measure of CCA.ConclusionsThe heterogeneity across adversities impacts estimates of adult psychological distress. Measuring CCA as patterns of co-occurring adverse experiences is a promising approach.     

The Utility of Educational Resilience for Studying Degree Attainment in School Dropouts

Abstract

The few studies that have compared dropouts with and without degrees have provided information that has limited application for policy and prevention, because factors identified as associated with degree attainment are ones that are difficult to affect. The present study examined diploma and GED attainment in Mexican American and non-Latino White dropouts, hypothesizing that utilization of the educational resilience framework would increase knowledge about degree attainment in dropouts and provide factors that are more easily affected by practitioners. Logistic regression was used to determine factors associated with degree attainment, and multiple imputation was used to account for sample bias due to nonparticipation. Results indicated that viewing the returning dropout as a resilient student does provide a more useful set of factors associated with degree attainment.     

Spelling Performance in Children with Developmental Language Disorder: A Meta-Analysis across European Languages

ABSTRACT

Spelling difficulties often occur in children with developmental language disorder (DLD), the extent and drivers of these difficulties are underspecified. Meta-analyses were conducted to address this gap. We identified child-based factors (age, language spoken, profile of difficulties) and task-based factors (text production or dictation) as potential moderators of the impact of DLD on spelling performance. Thirty-two studies were analyzed, including 984 children with DLD. Large differences between the spelling of children with DLD and their age-matched peers were found. The average mean effect size was ?1.42, 95% confidence interval [?1.60, ?1.24]. Heterogeneity was large. Effect size estimates were larger when participants had additional phonological or reading difficulties than when they did not. No differences were found between participants with DLD and their language-matched peers. The results are discussed with respect to the underlying processes that impact on spelling across languages. Specific recommendations for future studies are made.     

A review of online course dropout research: implications for practice and future research

Although online learning is expanding in availability and popularity, the high dropout rates remain a challenging problem. In this paper, we reviewed the existing empirical studies on online course dropouts in post-secondary education that were published during the last 10 years. We identified 69 factors that influence students’ decisions to dropout and classified them into three main categories: (a) Student factors, (b) Course/Program factors, and (c) Environmental factors. We then examined the strategies proposed to overcome these dropout factors: (a) understanding each student’s challenges and potential, (b) providing quality course activities and well-structured supports, and (c) handling environmental issues and emotional challenges. Finally, we discussed issues regarding dropout factors and strategies for addressing these factors and offered recommendations for future research.     

Gerechtigkeit und Studienabbruch. Die Rolle der wahrgenommenen Fairness von Benotungsverfahren bei der Entstehung von Abbruchsintentionen

This article investigates the relationship between grading fairness and university students’ dropout intentions. Our focus is on the extent to which students assess the grading procedures as fair from a procedural justice perspective. We distinguish between control-related and validity-related aspects of procedural justice. The fairness of grading procedures is incorporated into a rational choice model on educational decision-making. We theorize that students consider questions of procedural justice in determining their chances of success. Thus, unfair grading practices lead to an increased risk of the students developing dropout intentions. Further, we investigate the relationship between students’ socioeconomic characteristics and procedural justice. Data from the first wave of the CampusPanel are used (n = 1393). Results show that dropout intentions are significantly affected by evaluations of procedural justice. Further, there is evidence that students with low parental socioeconomic status as well as students with an immigrant background are especially prone to exhibit dropout intentions when confronted with low procedural justice.     

A Technique for the Construction of a Teacher-Training Curriculum

ABSTRACT

Historically, students who fail to graduate from secondary school are considered as a single category of school dropouts. However, emerging literature indicates that there may be multiple subgroups of high school dropouts, termed a dropout typology. The authors’ purpose was to assess the extent to which a typology of dropouts was present in a large national dataset and to estimate the influence of the known covariates of dropping out on each of the subgroups. A growth mixture model was estimated using the Education Longitudinal Study 2002 (National Center for Education Statistics, n.d.) dataset and noncumulative grade point average during the first 3 semesters of high school. The model identified 2 main subgroups associated with dropping out, which accounted for 24.6% of the sample but contained 91.8% of the dropouts.     

Regression Discontinuity Designs With Multiple Rating-Score Variables

Abstract

In the absence of a randomized control trial, regression discontinuity (RD) designs can produce plausible estimates of the treatment effect on an outcome for individuals near a cutoff score. In the standard RD design, individuals with rating scores higher than some exogenously determined cutoff score are assigned to one treatment condition; those with rating scores below the cutoff score are assigned to an alternate treatment condition. Many education policies, however, assign treatment status on the basis of more than one rating-score dimension. We refer to this class of RD designs as “multiple rating score regression discontinuity” (MRSRD) designs. In this paper, we discuss five different approaches to estimating treatment effects using MRSRD designs (response surface RD; frontier RD; fuzzy frontier RD; distance-based RD; and binding-score RD). We discuss differences among them in terms of their estimands, applications, statistical power, and potential extensions for studying heterogeneity of treatment effects.     

Capable Dropouts and the Social Milieu of the High School

Abstract

Studies of high-school dropouts suggest that many of those who drop out of school have the requisite ability to do passing or even superior work. A conceptual framework is presented which attempts to explain why these capable students leave school.

The basic premise is that a capable dropout is a response to status deprivation experienced primarily by lower-class youth in the informal network of peer relations and in the academic system of the school. The socialization in middle-class families prepares youth to compete successfully in school, while in lower-class families children are not trained to conform to the academic and informal requirements of the school.     

Why We (Usually) Don't Have to Worry About Multiple Comparisons

Abstract

Applied researchers often find themselves making statistical inferences in settings that would seem to require multiple comparisons adjustments. We challenge the Type I error paradigm that underlies these corrections. Moreover we posit that the problem of multiple comparisons can disappear entirely when viewed from a hierarchical Bayesian perspective. We propose building multilevel models in the settings where multiple comparisons arise. Multilevel models perform partial pooling (shifting estimates toward each other), whereas classical procedures typically keep the centers of intervals stationary, adjusting for multiple comparisons by making the intervals wider (or, equivalently, adjusting the p values corresponding to intervals of fixed width). Thus, multilevel models address the multiple comparisons problem and also yield more efficient estimates, especially in settings with low group-level variation, which is where multiple comparisons are a particular concern.     

A longitudinal analysis of the reciprocal relationship between academic procrastination,study satisfaction,and dropout intentions in higher education

Student dropout is a multi-causal process. Different theoretical models on student dropout consider dysfunctional study behavior (e.g., academic procrastination) and low study satisfaction as possible determinants of students’ dropout intentions during their university studies. However, these models neglect contemporary conceptualizations that assume reverse relationships between dropout intentions and other determinants of the dropout process. Until now, empirical evidence on these assumptions is scant. The present three-wave longitudinal study explored the reciprocal relationships between academic procrastination, study satisfaction, and dropout intentions over one semester. To this end, we used data of N = 326 undergraduate students enrolled in mathematics and law. Our latent cross-lagged panel model replicated existing empirical cross-sectional findings between the variables (i.e., academic procrastination, study satisfaction, and dropout intentions). Regarding the longitudinal relations, as expected, the cross-lagged effects showed that higher dropout intentions significantly related to subsequent higher academic procrastination and lower study satisfaction. Unexpectedly, academic procrastination did not significantly relate to subsequent dropout intentions. Additionally, higher study satisfaction significantly associated with subsequent higher dropout intentions—possibly due to unfulfilled expectations. Further, higher study satisfaction significantly related to subsequent higher procrastination—possibly due to more confidence among satisfied students. Our results broaden the view on dropout intentions as part of the dynamic interplay of student dropout determinants and the need to refine dropout models’ assumptions accordingly. Practically, realistic expectations seem important to reduce dropout intentions. Further, student counselors should have a closer look at the reasons for academic procrastination to develop individual solutions for this dysfunctional behavior.