Bayesian Model Averaging Over Directed Acyclic Graphs With Implications for the Predictive Performance of Structural Equation Models

This article examines Bayesian model averaging as a means of addressing predictive performance in Bayesian structural equation models. The current approach to addressing the problem of model uncertainty lies in the method of Bayesian model averaging. We expand the work of Madigan and his colleagues by considering a structural equation model as a special case of a directed acyclic graph. We then provide an algorithm that searches the model space for submodels and obtains a weighted average of the submodels using posterior model probabilities as weights. Our simulation study provides a frequentist evaluation of our Bayesian model averaging approach and indicates that when the true model is known, Bayesian model averaging does not yield necessarily better predictive performance compared to nonaveraged models. However, our case study using data from an international large-scale assessment reveals that the model-averaged submodels provide better posterior predictive performance compared to the initially specified model.     

Number of Subjects and Time Points Needed for Multilevel Time-Series Analysis: A Simulation Study of Dynamic Structural Equation Modeling

Dynamic structural equation modeling (DSEM) is a novel, intensive longitudinal data (ILD) analysis framework. DSEM models intraindividual changes over time on Level 1 and allows the parameters of these processes to vary across individuals on Level 2 using random effects. DSEM merges time series, structural equation, multilevel, and time-varying effects models. Despite the well-known properties of these analysis areas by themselves, it is unclear how their sample size requirements and recommendations transfer to the DSEM framework. This article presents the results of a simulation study that examines the estimation quality of univariate 2-level autoregressive models of order 1, AR(1), using Bayesian analysis in Mplus Version 8. Three features are varied in the simulations: complexity of the model, number of subjects, and number of time points per subject. Samples with many subjects and few time points are shown to perform substantially better than samples with few subjects and many time points.     

Goodness of Fit in Item Factor Analysis: Effect of the Number of Response Alternatives

The power of the chi-square test statistic used in structural equation modeling decreases as the absolute value of excess kurtosis of the observed data increases. Excess kurtosis is more likely the smaller the number of item response categories. As a result, fit is likely to improve as the number of item response categories decreases, regardless of the true underlying factor structure or χ²-based fit index used to examine model fit. Equivalently, given a target value of approximate fit (e.g., root mean square error of approximation ≤ .05) a model with more factors is needed to reach it as the number of categories increases. This is true regardless of whether the data are treated as continuous (common factor analysis) or as discrete (ordinal factor analysis). We recommend using a large number of response alternatives (≥ 5) to increase the power to detect incorrect substantive models.     

Fitting Structural Equation Model Trees and Latent Growth Curve Mixture Models in Longitudinal Designs: The Influence of Model Misspecification

When conducting longitudinal research, the investigation of between-individual differences in patterns of within-individual change can provide important insights. In this article, we use simulation methods to investigate the performance of a model-based exploratory data mining technique—structural equation model trees (SEM trees; Brandmaier, Oertzen, McArdle, & Lindenberger, 2013)—as a tool for detecting population heterogeneity. We use a latent-change score model as a data generation model and manipulate the precision of the information provided by a covariate about the true latent profile as well as other factors, including sample size, under the possible influences of model misspecifications. Simulation results show that, compared with latent growth curve mixture models, SEM trees might be very sensitive to model misspecification in estimating the number of classes. This can be attributed to the lower statistical power in identifying classes, resulting from smaller differences of parameters prescribed by the template model between classes.     

Bayesian Analysis of Mixture Structural Equation Models With an Unknown Number of Components

Multivariate heterogenous data with latent variables are common in many fields such as biological, medical, behavioral, and social-psychological sciences. Mixture structural equation models are multivariate techniques used to examine heterogeneous interrelationships among latent variables. In the analysis of mixture models, determination of the number of mixture components is always an important and challenging issue. This article aims to develop a full Bayesian approach with the use of reversible jump Markov chain Monte Carlo method to analyze mixture structural equation models with an unknown number of components. The proposed procedure can simultaneously and efficiently select the number of mixture components and conduct parameter estimation. Simulation studies show the satisfactory empirical performance of the method. The proposed method is applied to study risk factors of osteoporotic fractures in older people.     

Two Rules of Identification for Structural Equation Models

Identification of structural equation models remains a challenge to many researchers. Although empirical tests of identification are readily available in structural equation modeling software, these examine local identification and rely on sample estimates of parameters. Rules of identification are available, but do not include all models encountered in practice. In this article we provide 2 rules of identification: the 2+ emitted paths rule and the exogenous X rule. The former is a necessary condition of identification and the latter is a sufficient condition. We explain and prove each of these rules and provide illustrations of their application. These rules extend the coverage of structural equation models that we can check for identification. We also explain how they can be part of a piecewise identification strategy that extends their use even further.     

Unconstrained Structural Equation Models of Latent Interactions: Contrasting Residual- and Mean-Centered Approaches

Little, Bovaird and Widaman (2006) proposed an unconstrained approach with residual centering for estimating latent interaction effects as an alternative to the mean-centered approach proposed by Marsh, Wen, and Hau (2004, 2006). Little et al. also differed from Marsh et al. in the number of indicators used to infer the latent interaction factor and how they were represented, but this issue is separate from the mean versus residual centering distinction that was their primary focus. However, their implementation of the Marsh et al. mean-centered approach failed to incorporate the mean structure that Marsh et al. argued was necessary to obtain unbiased estimates. One might suppose that their new approach would suffer this same problem, an issue not addressed by Little et al. However, we demonstrate here why the Little et al. approach obviates this requirement that heretofore was thought to be necessary for all constrained, partially constrained, and unconstrained approaches. Both the Marsh et al. and Little et al. unconstrained approaches typically result in similar results and are much easier to implement than traditional constrained approaches. They differ primarily in that the Little et al. approach is a 2-step approach involving a potentially large number of separate analyses prior to estimating the structural equation model that apparently does not require the estimation of a mean structure, whereas the Marsh et al. approach is a 1-step approach that includes a mean structure.     

Bayesian Meta-Analytic SEM: A One-Stage Approach to Modeling Between-Studies Heterogeneity in Structural Parameters

Meta-analytic structural equation modeling (MASEM) refers to a set of meta-analysis techniques for combining and comparing structural equation modeling (SEM) results from multiple studies. Existing approaches to MASEM cannot appropriately model between-studies heterogeneity in structural parameters because of missing correlations, lack model fit assessment, and suffer from several theoretical limitations. In this study, we address the major shortcomings of existing approaches by proposing a novel Bayesian multilevel SEM approach. Simulation results showed that the proposed approach performed satisfactorily in terms of parameter estimation and model fit evaluation when the number of studies and the within-study sample size were sufficiently large and when correlations were missing completely at random. An empirical example about the structure of personality based on a subset of data was provided. Results favored the third factor structure over the hierarchical structure. We end the article with discussions and future directions.     

A Multilevel CFA–MTMM Approach for Multisource Feedback Instruments: Presentation and Application of a New Statistical Model

Multisource feedback instruments are a widely used tool in human resource management. However, comprehensive validation studies remain scarce and there is a lack of statistical models that account appropriately for the complex data structure. Because both peers and subordinates are nested within the target but stem from different populations, the assumption of traditional multilevel structural equation models that the sample on a lower level stems from the same population is violated. We present a multilevel confirmatory factor analysis multitrait–multimethod (ML–CFA–MTMM) model that considers this peculiarity of multisource feedback instruments. The model is applied to 2 scales of the Benchmarks® instrument and it is demonstrated how measures of reliability and of convergent and discriminant validity can be obtained using multilevel structural equation modeling software. We discuss the results as well as some implications and guidelines for the use of the model.     

Bayesian Estimation of the True Score Multitrait–Multimethod Model With a Split-Ballot Design

This article examines whether Bayesian estimation with minimally informed prior distributions can alleviate the estimation problems often encountered with fitting the true score multitrait–multimethod structural equation model with split-ballot data. In particular, the true score multitrait–multimethod structural equation model encounters an empirical underidentification when (a) latent variable correlations are homogenous, and (b) fitted to data from a 2-group split-ballot design; an understudied case of empirical underidentification due to a planned missingness (i.e., split-ballot) design. A Monte Carlo simulation and 3 empirical examples showed that Bayesian estimation performs better than maximum likelihood (ML) estimation. Therefore, we suggest using Bayesian estimation with minimally informative prior distributions when estimating the true score multitrait–multimethod structural equation model with split-ballot data. Furthermore, given the increase in planned missingness designs in psychological research, we also suggest using Bayesian estimation as a potential alternative to ML estimation for analyses using data from planned missingness designs.     

Effects of sample size,estimation methods,and model specification on structural equation modeling fit indexes

A Monte Carlo simulation study was conducted to investigate the effects on structural equation modeling (SEM) fit indexes of sample size, estimation method, and model specification. Based on a balanced experimental design, samples were generated from a prespecified population covariance matrix and fitted to structural equation models with different degrees of model misspecification. Ten SEM fit indexes were studied. Two primary conclusions were suggested: (a) some fit indexes appear to be noncomparable in terms of the information they provide about model fit for misspecified models and (b) estimation method strongly influenced almost all the fit indexes examined, especially for misspecified models. These 2 issues do not seem to have drawn enough attention from SEM practitioners. Future research should study not only different models vis‐à‐vis model complexity, but a wider range of model specification conditions, including correctly specified models and models specified incorrectly to varying degrees.     

Pick a Number … Any Number?

A recent classroom revelation caused us to reconsider the adequacy of the instructions offered in our textbooks for one of our most elementary quantitative methods. Specifically, we found that many students were mystified concerning how to pick an initial objective function value when plotting an isoprofit line in order to graphically solve a linear programming problem. We observed that this seemingly insignificant issue was causing a surprising amount of wasted time, confusion, and even anxiety. Our experience indicated that students do not necessarily possess any consistent intuition for choosing an appropriate initial value. We confirmed this perception through informal experimentation. We reviewed several textbooks and found that most simply do not bother to address this issue. Therefore, we devised a simple, comprehensible method which we have taught our students to resolve or avoid this problem. Although we quickly found the method to be helpful for many students, we conducted an experiment to formally measure its effect.     

Modeling Predictors of Latent Classes in Regression Mixture Models

The purpose of this study is to provide guidance on a process for including latent class predictors in regression mixture models. We first examine the performance of current practice for using the 1-step and 3-step approaches where the direct covariate effect on the outcome is omitted. None of the approaches show adequate estimates of model parameters. Given that Step 1 of the 3-step approach shows adequate results in class enumeration, we suggest using an alternative approach: (a) decide the number of latent classes without predictors of latent classes, and (b) bring the latent class predictors into the model with the inclusion of hypothesized direct covariate effects. Our simulations show that this approach leads to good estimates for all model parameters. The proposed approach is demonstrated by using empirical data to examine the differential effects of family resources on students’ academic achievement outcome. Implications of the study are discussed.     

Analyzing Mixed-Dyadic Data Using Structural Equation Models

Mixed-dyadic data, collected from distinguishable (nonexchangeable) or indistinguishable (exchangeable) dyads, require statistical analysis techniques that model the variation within dyads and between dyads appropriately. The purpose of this article is to provide a tutorial for performing structural equation modeling analyses of cross-sectional and longitudinal models for mixed independent variable dyadic data, and to clarify questions regarding various dyadic data analysis specifications that have not been addressed elsewhere. Artificially generated data similar to the Newlywed Project and the Swedish Adoption Twin Study on Aging were used to illustrate analysis models for distinguishable and indistinguishable dyads, respectively. Due to their widespread use among applied researchers, the AMOS and Mplus statistical analysis software packages were used to analyze the dyadic data structural equation models illustrated here. These analysis models are presented in sufficient detail to allow researchers to perform these analyses using their preferred statistical analysis software package.     

Individually Weighted-Average Models: Testing a Taxonomic SEM Approach Across Different Multidimensional/Global Constructs Because the Weights “Don’t Make No Nevermind”

From the time of William James, psychologists have posited individually importance-weighted-average models (IWAMs) in which weighting specific attributes by individual measures of importance improves prediction of the global outcome measures. Because IWAMs cause much confusion, we briefly review a general taxonomic paradigm and structural equation models for testing IWAMs, and demonstrate its application for 2 simulated and 3 diverse “real” data applications (multidimensional measures of self-concept, quality of life, and job satisfaction). Consistent across the real data applications and previous research more generally, there is surprisingly little support for IWAMs when tested appropriately. In these diverse tests of IWAMs we integrate new approaches such as exploratory structural equation modeling (SEM), alternative approaches to constructing latent interactions, application of bifactor models, modeling method and item-wording effects, and the juxtaposition of model evaluation in relation to goodness of fit (typically used in SEM studies) and variance explained (typically used in multiple regression tests of IWAMs).     

Direct and indirect effects of education on job satisfaction: A structural equation model for the Spanish case

Empirical literature has traditionally analyzed the effect of education on job satisfaction with single-equation models that ignore interrelationships between theoretical explanatory variables. Their results are somewhat inconclusive. We propose estimating a structural equation model to obtain both the direct effects and the set of indirect effects. Analysis of these effects allows us to explain the apparent contradictions that have existed to date, and to improve knowledge of the economic value of education. The proposed model shows that people with higher levels of formal education are more satisfied with their jobs, because they are more likely to access jobs with characteristics that provide greater satisfaction.     

On estimating finite mixtures of multivariate regression and simultaneous equation models

We propose a maximum likelihood framework for estimating finite mixtures of multivariate regression and simultaneous equation models with multiple endogenous variables. The proposed “semi‐parametric” approach posits that the sample of endogenous observations arises from a finite mixture of components (or latent‐classes) of unknown proportions with multiple structural relations implied by the specified model for each latent‐class. We devise an Expectation‐Maximization algorithm in a maximum likelihood framework to simultaneously estimate the class proportions, the class‐specific structural parameters, and posterior probabilities of membership of each observation into each latent‐class. The appropriate number of classes can be chosen using various information‐theoretic heuristics. A data set entailing cross‐sectional observations for a diverse sample of businesses is used to illustrate the proposed approach.     

Mostly Harmless Direct Effects: A Comparison of Different Latent Markov Modeling Approaches

We evaluate the performance of the most common estimators of latent Markov (LM) models with covariates in the presence of direct effects of the covariates on the indicators of the LM model. In LM modeling it is common practice not to model such direct effects, ignoring the consequences that might have on the overall model fit and the parameters of interest. However, in the general literature about latent variable modeling it is well known that unmodeled direct effects can severely bias the parameter estimates of the model at hand. We evaluate how the presence of direct effects in?uences the bias and efficiency of the 3 most common estimators of LM models, the 1-step, 2-step, and 3-step approaches. Furthermore, we propose amendments (that were thus far not used in the context of LM modeling) to the 2- and 3-step approaches that make it possible to account for direct effects and eliminate bias as a consequence. This is done by modeling the (possible) direct effects in the first step of the stepwise estimation procedures. We evaluate the proposed estimators through an extensive simulation study, and illustrate them via a real data application. Our results show, first, that the augmented 2-step and 3-step approaches are unbiased and efficient estimators of LM models with direct effects. Second, ignoring the direct effects leads to biased estimates with all existing estimators, the 1-step approach being the most sensitive.     

Fitting Unstructured Finite Mixture Models in Longitudinal Design: A Recommendation for Model Selection and Estimation of the Number of Classes

In longitudinal design, investigating interindividual differences of intraindividual changes enables researchers to better understand the potential variety of development and growth. Although latent growth curve mixture models have been widely used, unstructured finite mixture models (uFMMs) are also useful as a preliminary tool and are expected to be more robust in identifying classes under the influence of possible model misspecifications, which are very common in actual practice. In this study, large-scale simulations were performed in which various normal uFMMs and nonnormal uFMMs were fit to evaluate their utility and the performance of each model selection procedure for estimating the number of classes in longitudinal designs. Results show that normal uFMMs assuming invariance of variance–covariance structures among classes perform better on average. Among model selection procedures, the Calinski–Harabasz statistic, which has a nonparametric nature, performed better on average than information criteria, including the Bayesian information criterion.