Examining Estimates of Intervention Effectiveness Using Sensitivity Analysis

Making causal inferences from a quasi‐experiment is difficult. Sensitivity analysis approaches to address hidden selection bias thus have gained popularity. This study serves as an introduction to a simple but practical form of sensitivity analysis using Monte Carlo simulation procedures. We examine estimated treatment effects for a school‐based support intervention designed to address student strengths and needs in academic and nonacademic areas by leveraging partnerships with community agencies. Middle school (Grades 6–8) statewide standardized test scores in mathematics and English language arts (ELA) were examined for students in a large urban district who participated in City Connects during elementary school. Results showed that the estimated treatment effects in both subjects were reduced slightly with the inclusion of U, a hypothesized unobserved binary variable. However, simulated effects fell within one‐sided 90% confidence intervals for original treatment effects, suggesting only a mild sensitivity to hidden bias. Moreover, almost identical estimated treatment effects were observed when the magnitude of the mathematical difference between each pair of the conditional probabilities of U given the treatment indicator Z was the same.     

Using an analogy to model causal mechanisms in a complex text

Two experiments examined the role of analogy inthe development of a situation model of atarget passage. Specifically, we examinedwhether an analogous source text could improvecomprehension and inferencing about causalmechanisms in the target. An ``analogy group'studied a fictional pseudo-scientific passageunder the guidance of an analogous passage(also fictional) which shared causal structureswith the target. A ``statement group' studiedthe target under the guidance of an abstractstatement that characterized the target causalstructures. A ``target-only group' received noadditional material. All groups received tasksthat assessed their representation of thetarget. In Experiment 1 (N = 92) participantswere asked to list the most important objectsand relations in the passage. In Experiment 2(N = 80) participants were directly asked aboutthe target causal structures. In bothexperiments, responses of analogy subjects weremore likely to include given and inferredinformation that comprised the target causalmechanisms. Analogies in which a sourcepassage models causal mechanisms correspondingto those of a target passage appear to promotebetter representation of the details of targetcausal mechanisms, allowing a more fullydeveloped situation model of the target.     

Causal Mediation Analysis With a Binary Outcome and Multiple Continuous or Ordinal Mediators: Simulations and Application to an Alcohol Intervention

We investigate a method to estimate the combined effect of multiple continuous/ordinal mediators on a binary outcome: (a) fit a structural equation model with probit link for the outcome and identity/probit link for continuous/ordinal mediators, (b) predict potential outcome probabilities, and (c) compute natural direct and indirect effects. Step 2 involves rescaling the latent continuous variable underlying the outcome to address residual mediator variance and covariance. We evaluate the estimation of risk-difference- and risk-ratio-based effects (RDs, RRs) using the maximum likelihood (ML), mean-and-variance-adjusted weighted least squares (WLSMV) and Bayes estimators in Mplus. Across most variations in path-coefficient and mediator-residual-correlation signs and strengths, and confounding situations investigated, the method performs well with all estimators, but favors ML/WLSMV for RDs with continuous mediators, and Bayes for RRs with ordinal mediators. Bayes outperforms ML/WLSMV regardless of mediator type when estimating RRs with small potential outcome probabilities and in two other special cases. An adolescent alcohol prevention study is used for illustration.     

Using conjoint analysis to examine bias in referral to a behavior intervention program

Research in the United States suggests that teachers view Black students' misbehavior as more likely to reoccur, but such research has not disentangled the effect of race from related factors. We used conjoint analysis to experimentally test the effect of multiple student characteristics on the chances of being referred for a behavior intervention program. Although preservice teacher participants were not more likely to refer Black students, they were more likely to refer students who had lower grades, lived in poverty, lived with a single mother, and had been retained. We interpret these results and highlight directions for future research and practice.     

On the Potential of Discrete Time Survival Analysis Using Latent Variable Modeling: An Application to the Study of the Vascular Depression Hypothesis

Analysis and modeling of time to event data have been traditionally associated with nonparametric, semiparametric, or parametric statistical frameworks. Recent advances in latent variable modeling have additionally provided unique analytic opportunities to methodologists and substantive researchers interested in survival time modeling. As a consequence, discrete time survival analyses can now be readily carried out using latent variable modeling, an approach that offers substantively important extensions to conventional survival models. Using data from the Health and Retirement Study, the discussed approach is applied to the study of the increasingly prominent vascular depression hypothesis in gerontology, geriatrics, and aging research, allowing examination of the unique predictive power of depression with respect to time to stroke in middle-aged and older adults.