Model Specification Searches in Structural Equation Modeling with R

This article introduces and demonstrates the application of an R statistical programming environment code for conducting structural equation modeling (SEM) specification searches. The implementation and flexibility of the provided code is demonstrated using the Tabu search procedure, although the underlying code can also be directly modified to implement other search procedures like Ant Colony Optimization, Genetic Algorithms, Ruin-and-Recreate, or Simulated Annealing. The application is illustrated using data with a known common factor structure. The results demonstrate the capabilities of the program for conducting specification searches in SEM. The programming codes are provided as open-source R functions.     

Reporting Monte Carlo Studies in Structural Equation Modeling

In structural equation modeling, Monte Carlo simulations have been used increasingly over the last two decades, as an inventory from the journal Structural Equation Modeling illustrates. Reaching out to a broad audience, this article provides guidelines for reporting Monte Carlo studies in that field. The framework of discourse is set by a number of steps to be taken in such research, matching outlines of experimental design by Paxton, Curran, Bollen, Kirby, and Chen (2001) Chen, F., Bollen, K. A., Paxton, P., Curran, P. J. and Kirby, J. 2001. Improper solutions in structural equation modeling: Causes, consequences, and strategies. Sociological Methods & Research, 29: 468–508. [Crossref], [Web of Science ®] , [Google Scholar] and Skrondal (2000) Skrondal, A. 2000. Design and analysis of Monte Carlo experiments: Attacking the conventional wisdom. Multivariate Behavioral Research, 35: 137–167. [Taylor & Francis Online], [Web of Science ®] , [Google Scholar]. Throughout the article, reference is made to exemplary publications and, occasionally, to imperfect reporting.     

Detecting Violations of Factorial Invariance Using Data-Based Specification Searches: A Monte Carlo Study

In testing factorial invariance, researchers have often used a reference variable strategy in which the factor loading for a variable (i.e., reference variable) is fixed to 1 for identification. This commonly used method can be misleading if the chosen reference variable is actually a noninvariant item. This simulation study suggests an alternative method for testing factorial invariance and evaluates the performance of the method in specification searches based on the modification index. The results of the study showed that the proposed specification searches performed well when the number of noninvariant variables was relatively small and this performance improved as sample size increased and the size of group differences increased. When the number of noninvariant variables was relatively large, however, the method rarely succeeded in detecting the noninvariant items in the specification searches. Implications of the findings are discussed along with the limitations of the study.     

Many-Level Multilevel Structural Equation Modeling: An Efficient Evaluation Strategy

Structural equation models are increasingly used for clustered or multilevel data in cases where mixed regression is too inflexible. However, when there are many levels of nesting, these models can become difficult to estimate. We introduce a novel evaluation strategy, Rampart, that applies an orthogonal rotation to the parts of a model that conform to commonly met requirements. This rotation dramatically simplifies fit evaluation in a way that becomes more potent as the size of the data set increases. We validate and evaluate the implementation using a 3-level latent regression simulation study. Then we analyze data from a statewide child behavioral health measure administered by the Oklahoma Department of Human Services. We demonstrate the efficiency of Rampart compared to other similar software using a latent factor model with a 5-level decomposition of latent variance. Rampart is implemented in OpenMx, a free and open source software package.     

Multilevel Mediation With Small Samples: A Cautionary Note on the Multilevel Structural Equation Modeling Framework

Multilevel structural equation modeling (ML-SEM) for multilevel mediation is noted for its flexibility over a system of multilevel models (MLMs). Sample size requirements are an overlooked limitation of ML-SEM (100 clusters is recommended). We find that 89% of ML-SEM studies have fewer than 100 clusters and the median number is 44. Furthermore, 75% of ML-SEM studies implement 2–1–1 or 1–1–1 models, which can be equivalently fit with MLMs. MLMs theoretically have lower sample size requirements, although studies have yet to assess small sample performance for multilevel mediation. We conduct a simulation to address this pervasive problem. We find that MLMs have more desirable small sample performance and can be trustworthy with 10 clusters. Importantly, many studies lack the sample size and model complexity to necessitate ML-SEM. Although ML-SEM is undeniably more flexible and uniquely positioned for difficult problems, small samples often can be more effectively and simply addressed with MLMs.     

Level-Specific Evaluation of Model Fit in Multilevel Structural Equation Modeling

This article examines the structure of several HIV risk behaviors in an ethnically and geographically diverse sample of 8,251 clients from 10 innovative demonstration projects intended for adolescents living with, or at risk for, HIV. Exploratory and confirmatory factor analyses identified 2 risk factors for men (sexual intercourse with men and a general risk factor) and 3 factors for women (sexual intercourse with men, substance abuse, and a high risky sex behavior factor). All factors except women engaging in risky sex with men strongly predicted known HIV status of clients for men and women. The findings from this investigation highlight the use of structural equation modeling for applied problems involving overlapping and complex sets of risk behaviors in youth who present at community health programs.     

Testing Factorial Invariance in Multilevel Data: A Monte Carlo Study

Testing factorial invariance has recently gained more attention in different social science disciplines. Nevertheless, when examining factorial invariance, it is generally assumed that the observations are independent of each other, which might not be always true. In this study, we examined the impact of testing factorial invariance in multilevel data, especially when the dependency issue is not taken into account. We considered a set of design factors, including number of clusters, cluster size, and intraclass correlation (ICC) at different levels. The simulation results showed that the test of factorial invariance became more liberal (or had inflated Type I error rate) in terms of rejecting the null hypothesis of invariance held between groups when the dependency was not considered in the analysis. Additionally, the magnitude of the inflation in the Type I error rate was a function of both ICC and cluster size. Implications of the findings and limitations are discussed.     

Applications of Multilevel Structural Equation Modeling to Cross-Cultural Research

Multilevel structural equation modeling (MSEM) has been proposed as an extension to structural equation modeling for analyzing data with nested structure. We have begun to see a few applications in cross-cultural research in which MSEM fits well as the statistical model. However, given that cross-cultural studies can only afford collecting data from a relatively small number of countries, the appropriateness of MSEM has been questioned. Using the data from the International Social Survey Program (1997; N = 15,244 from 27 countries), we first showed how Muth?n's MSEM procedure could be applied to a real data set on cross-cultural research. Given a small country-level sample size (27 countries) we then demonstrated that results on the individual level were quite stable even when using small individual-level sample sizes, whereas the group-level parameter estimates and their standard errors were affected unsystematically by varying individual-level sample sizes. Use of the findings for cross-cultural research and other areas with limited numbers of groups are discussed.     

模型设定偏误的蒙特卡洛模拟

设定偏误包括模型中遗漏变量和包含多余变量,会导致一些较严重的后果。在教学过程中,如果仅仅采用数学推导推演这些后果,由于公式复杂,学生接受较为困难;如果利用蒙特卡洛试验辅助教学,接受则会较为轻松。本研究利用GAUSS软件模拟模型设定偏误的后果。     

Multilevel Factorial Invariance in n-Level Structural Equation Modeling (nSEM)

We present a multigroup multilevel confirmatory factor analysis (CFA) model and a procedure for testing multilevel factorial invariance in n-level structural equation modeling (nSEM). Multigroup multilevel CFA introduces a complexity when the group membership at the lower level intersects the clustered structure, because the observations in different groups but in the same cluster are not independent of one another. nSEM provides a framework in which the multigroup multilevel data structure is represented with the dependency between groups at the lower level properly taken into account. The procedure for testing multilevel factorial invariance is illustrated with an empirical example using an R package xxm2.     

A Bayesian Approach to Multilevel Structural Equation Modeling With Continuous and Dichotomous Outcomes

Multilevel Structural equation models are most often estimated from a frequentist framework via maximum likelihood. However, as shown in this article, frequentist results are not always accurate. Alternatively, one can apply a Bayesian approach using Markov chain Monte Carlo estimation methods. This simulation study compared estimation quality using Bayesian and frequentist approaches in the context of a multilevel latent covariate model. Continuous and dichotomous variables were examined because it is not yet known how different types of outcomes—most notably categorical—affect parameter recovery in this modeling context. Within the Bayesian estimation framework, the impact of diffuse, weakly informative, and informative prior distributions were compared. Findings indicated that Bayesian estimation may be used to overcome convergence problems and improve parameter estimate bias. Results highlight the differences in estimation quality between dichotomous and continuous variable models and the importance of prior distribution choice for cluster-level random effects.     

Lower Level Mediation Effect Analysis in Two-Level Studies: A Note on a Multilevel Structural Equation Modeling Approach

This article presents a didactic discussion of a multilevel covariance structure modeling approach to estimation of lowest level mediation effect indexes in two-level studies. The procedure is useful when addressing questions about relations among total and indirect effects between variables of interest while accounting for the hierarchical structure of analyzed data. The discussed method also permits interval estimation and hypothesis tests with respect to related quantities of relevance when evaluating mediated effects with clustered data, and is illustrated on a two-level data set.     

A Systematic Evaluation and Comparison Between Exploratory Structural Equation Modeling and Bayesian Structural Equation Modeling

In this study, we contrast two competing approaches, not previously compared, that balance the rigor of CFA/SEM with the flexibility to fit realistically complex data. Exploratory SEM (ESEM) is claimed to provide an optimal compromise between EFA and CFA/SEM. Alternatively, a family of three Bayesian SEMs (BSEMs) replace fixed-zero estimates with informative, small-variance priors for different subsets of parameters: cross-loadings (CL), residual covariances (RC), or CLs and RCs (CLRC). In Study 1, using three simulation studies, results showed that (1) BSEM-CL performed more closely to ESEM; (2) BSEM-CLRC did not provide more accurate model estimation compared with BSEM-CL; (3) BSEM-RC provided unstable estimation; and (4) different specifications of targeted values in ESEM and informative priors in BSEM have significant impacts on model estimation. The real data analysis (Study 2) showed that the differences in estimation between different models were largely consistent with those in Study1 but somewhat smaller.     

Modeling Causal Error Structures in Longitudinal Panel Data: A Monte Carlo Study

This research was designed to investigate how much more suitable moving average (MA) and autoregressive-moving average (ARMA) models are for longitudinal panel data in which measurement errors correlate than AR, quasi-simplex, and 1-factor models. The conclusions include (a) when testing for a stochastic process hypothesized to occur in a longitudinal data set, testing for other processes is necessary, because incorrect models often fit other processes well enough to be deceiving; (b) when measurement error correlations are flagged to be relatively high in panel data, the fit and propriety of an MA or ARMA model should be considered and compared to the fit and propriety of other models; (c) when an MA model is fit to AR data, measurement error correlations may nonetheless be deceptively high, though fortunately MA model fit indexes are almost always lower than those for an AR model; and (d) the assumption that longitudinal panel data always contain measurement error correlations is patently false. In summary, whenever evaluating longitudinal panel data, the fit, propriety, and parsimony of all 5 models should be considered jointly and compared before a particular model is endorsed as most suitable.     

Modeling Latent Growth Curves With Incomplete Data Using Different Types of Structural Equation Modeling and Multilevel Software

This article offers different examples of how to fit latent growth curve (LGC) models to longitudinal data using a variety of different software programs (i.e., LISREL, Mx, Mplus, AMOS, SAS). The article shows how the same model can be fitted using both structural equation modeling and multilevel software, with nearly identical results, even in the case of models of latent growth fitted to incomplete data. The general purpose of this article is to provide a demonstration that integrates programming features from different software. The most immediate goal is to help researchers implement these LGC models as a useful way to test hypotheses of growth.     

Modeling Unobserved Heterogeneity Using Latent Profile Analysis: A Monte Carlo Simulation

Latent profile analysis (LPA) has become a popular statistical method for modeling unobserved population heterogeneity in cross-sectionally sampled data, but very few empirical studies have examined the question of how well enumeration indexes accurately identify the correct number of latent profiles present. This Monte Carlo simulation study examined the ability of several classes of enumeration indexes to correctly identify the number of latent population profiles present under 3 different research design conditions: sample size, the number of observed variables used for LPA, and the separation distance among the latent profiles measured in Mahalanobis D units. Results showed that, for the homogeneous population (i.e., the population has k = 1 latent profile) conditions, many of the enumeration indexes used in LPA were able to correctly identify the single latent profile if variances and covariances were freely estimated. However, for a heterogeneous population (i.e., the population has k = 3 distinct latent profiles), the correct identification rate for the enumeration indexes in the k = 3 latent profile conditions was typically very low. These results are compared with the previous cross-sectional mixture modeling studies, and the limitations of this study, as well as future cross-sectional mixture modeling and enumeration index research possibilities, are discussed.     

A Regularized GLS for Structural Equation Modeling

Ill conditioning of covariance and weight matrices used in structural equation modeling (SEM) is a possible source of inadequate performance of SEM statistics in nonasymptotic samples. A maximum a posteriori (MAP) covariance matrix is proposed for weight matrix regularization in normal theory generalized least squares (GLS) estimation. Maximum likelihood (ML), GLS, and regularized GLS test statistics (RGLS and rGLS) are studied by simulation in a 15-variable, 3-factor model with 15 levels of sample size varying from 60 to 100,000. A key result showed that in terms of nominal rejection rates, RGLS outperformed ML at all sample sizes below 500, and GLS at most sample sizes below 500. In larger samples, their performance was equivalent. The second regularization methodology (rGLS) performed well asymptotically, but poorly in small samples. Regularization in SEM deserves further study.     

Exploratory Structural Equation Modeling

This article examines the problem of specification error in 2 models for categorical latent variables; the latent class model and the latent Markov model. Specification error in the latent class model focuses on the impact of incorrectly specifying the number of latent classes of the categorical latent variable on measures of model adequacy as well as sample reallocation to latent classes. The results show that the clarity of remaining latent classes, as measured by the entropy statistic depends on the number of observations in the omitted latent class—but this statistic is not reliable. Specification error in the latent Markov model focuses on the transition probabilities when a longitudinal Guttman process is incorrectly specified. The findings show that specifying a longitudinal Guttman process that is not true in the population impacts other transition probabilities through the covariance matrix of the logit parameters used to calculate those probabilities.     

Regularized Structural Equation Modeling

A new method is proposed that extends the use of regularization in both lasso and ridge regression to structural equation models. The method is termed regularized structural equation modeling (RegSEM). RegSEM penalizes specific parameters in structural equation models, with the goal of creating easier to understand and simpler models. Although regularization has gained wide adoption in regression, very little has transferred to models with latent variables. By adding penalties to specific parameters in a structural equation model, researchers have a high level of flexibility in reducing model complexity, overcoming poor fitting models, and the creation of models that are more likely to generalize to new samples. The proposed method was evaluated through a simulation study, two illustrative examples involving a measurement model, and one empirical example involving the structural part of the model to demonstrate RegSEM’s utility.     

Number of Subjects and Time Points Needed for Multilevel Time-Series Analysis: A Simulation Study of Dynamic Structural Equation Modeling

Dynamic structural equation modeling (DSEM) is a novel, intensive longitudinal data (ILD) analysis framework. DSEM models intraindividual changes over time on Level 1 and allows the parameters of these processes to vary across individuals on Level 2 using random effects. DSEM merges time series, structural equation, multilevel, and time-varying effects models. Despite the well-known properties of these analysis areas by themselves, it is unclear how their sample size requirements and recommendations transfer to the DSEM framework. This article presents the results of a simulation study that examines the estimation quality of univariate 2-level autoregressive models of order 1, AR(1), using Bayesian analysis in Mplus Version 8. Three features are varied in the simulations: complexity of the model, number of subjects, and number of time points per subject. Samples with many subjects and few time points are shown to perform substantially better than samples with few subjects and many time points.