Comparing Exploratory Structural Equation Modeling and Existing Approaches for Multiple Regression with Latent Variables

Exploratory structural equation modeling (ESEM) is an approach for analysis of latent variables using exploratory factor analysis to evaluate the measurement model. This study compared ESEM with two dominant approaches for multiple regression with latent variables, structural equation modeling (SEM) and manifest regression analysis (MRA). Main findings included: (1) ESEM in general provided the least biased estimation of the regression coefficients; SEM was more biased than MRA given large cross-factor loadings. (2) MRA produced the most precise estimation, followed by ESEM and then SEM. (3) SEM was the least powerful in the significance tests; statistical power was lower for ESEM than MRA with relatively small target-factor loadings, but higher for ESEM than MRA with relatively large target-factor loadings. (4) ESEM showed difficulties in convergence and occasionally created an inflated type I error rate under some conditions. ESEM is recommended when non-ignorable cross-factor loadings exist.     

Exploratory Structural Equation Modeling,Integrating CFA and EFA: Application to Students' Evaluations of University Teaching

This study is a methodological-substantive synergy, demonstrating the power and flexibility of exploratory structural equation modeling (ESEM) methods that integrate confirmatory and exploratory factor analyses (CFA and EFA), as applied to substantively important questions based on multidimentional students' evaluations of university teaching (SETs). For these data, there is a well established ESEM structure but typical CFA models do not fit the data and substantially inflate correlations among the nine SET factors (median rs = .34 for ESEM, .72 for CFA) in a way that undermines discriminant validity and usefulness as diagnostic feedback. A 13-model taxonomy of ESEM measurement invariance is proposed, showing complete invariance (factor loadings, factor correlations, item uniquenesses, item intercepts, latent means) over multiple groups based on the SETs collected in the first and second halves of a 13-year period. Fully latent ESEM growth models that unconfounded measurement error from communality showed almost no linear or quadratic effects over this 13-year period. Latent multiple indicators multiple causes models showed that relations with background variables (workload/difficulty, class size, prior subject interest, expected grades) were small in size and varied systematically for different ESEM SET factors, supporting their discriminant validity and a construct validity interpretation of the relations. A new approach to higher order ESEM was demonstrated, but was not fully appropriate for these data. Based on ESEM methodology, substantively important questions were addressed that could not be appropriately addressed with a traditional CFA approach.     

A Comparison of CFA,ESEM, and BSEM in Test Structure Analysis

Minor cross-loadings on non-targeted factors are often found in psychological or other instruments. Forcing them to zero in confirmatory factor analyses (CFA) leads to biased estimates and distorted structures. Alternatively, exploratory structural equation modeling (ESEM) and Bayesian structural equation modeling (BSEM) have been proposed. In this research, we compared the performance of the traditional independent-clusters-confirmatory-factor-analysis (ICM-CFA), the nonstandard CFA, ESEM with the Geomin- or Target-rotations, and BSEMs with different cross-loading priors (correct; small- or large-variance priors with zero mean) using simulated data with cross-loadings. Four factors were considered: the number of factors, the size of factor correlations, the cross-loading mean, and the loading variance. Results indicated that ICM-CFA performed the worst. ESEMs were generally superior to CFAs but inferior to BSEM with correct priors that provided the precise estimation. BSEM with large- or small-variance priors performed similarly while the prior mean for cross-loadings was more important than the prior variance.     

Two-Level Dynamic Structural Equation Models with Small Samples

Advances in data collection have made intensive longitudinal data easier to collect, unlocking potential for methodological innovations to model such data. Dynamic structural equation modeling (DSEM) is one such methodology but recent studies have suggested that its small N performance is poor. This is problematic because small N data are omnipresent in empirical applications due to logistical and financial concerns associated with gathering many measurements on many people. In this paper, we discuss how previous studies considering small samples have focused on Bayesian methods with diffuse priors. The small sample literature has shown that diffuse priors may cause problems because they become unintentionally informative. Instead, we outline how researchers can create weakly informative admissible-range-restricted priors, even in the absence of previous studies. A simulation study shows that metrics like relative bias and non-null detection rates with these admissible-range-restricted priors improve small N properties of DSEM compared to diffuse priors.     

The Use of Incorrect Informative Priors in the Estimation of MIMIC Model Parameters with Small Sample Sizes

Recently, advancements in Bayesian structural equation modeling (SEM), particularly software developments, have allowed researchers to more easily employ it in data analysis. With the potential for greater use, come opportunities to apply Bayesian SEM in a wider array of situations, including for small sample size problems. Effective use of Bayseian estimation hinges on selection of appropriate prior distributions for model parameters. Researchers have suggested that informative priors may be useful with small samples, presuming that the mean of the prior is accurate with respect to the population mean. The purpose of this simulation study was to examine model parameter estimation for the Multiple Indicator Multiple Cause model when an informative prior distribution had an incorrect mean. Results demonstrated that the use of incorrect informative priors with somewhat larger variance than is typical, yields more accurate parameter estimates than do naïve priors, or maximum likelihood estimation. Implications for practice are discussed.     

Bayesian Versus Maximum Likelihood Estimation of Multitrait–Multimethod Confirmatory Factor Models

This article compares maximum likelihood and Bayesian estimation of the correlated trait–correlated method (CT–CM) confirmatory factor model for multitrait–multimethod (MTMM) data. In particular, Bayesian estimation with minimally informative prior distributions—that is, prior distributions that prescribe equal probability across the known mathematical range of a parameter—are investigated as a source of information to aid convergence. Results from a simulation study indicate that Bayesian estimation with minimally informative priors produces admissible solutions more often maximum likelihood estimation (100.00% for Bayesian estimation, 49.82% for maximum likelihood). Extra convergence does not come at the cost of parameter accuracy; Bayesian parameter estimates showed comparable bias and better efficiency compared to maximum likelihood estimates. The results are echoed via 2 empirical examples. Hence, Bayesian estimation with minimally informative priors outperforms enables admissible solutions of the CT–CM model for MTMM data.     

Variance Estimation Using Replication Methods in Structural Equation Modeling With Complex Sample Data

This article discusses replication sampling variance estimation techniques that are often applied in analyses using data from complex sampling designs: jackknife repeated replication, balanced repeated replication, and bootstrapping. These techniques are used with traditional analyses such as regression, but are currently not used with structural equation modeling (SEM) analyses. This article provides an extension of these methods to SEM analyses, including a proposed adjustment to the likelihood ratio test, and presents the results from a simulation study suggesting replication estimates are robust. Finally, a demonstration of the application of these methods using data from the Early Childhood Longitudinal Study is included. Secondary analysts can undertake these more robust methods of sampling variance estimation if they have access to certain SEM software packages and data management packages such as SAS, as shown in the article.     

A Bayesian Approach to Multilevel Structural Equation Modeling With Continuous and Dichotomous Outcomes

Multilevel Structural equation models are most often estimated from a frequentist framework via maximum likelihood. However, as shown in this article, frequentist results are not always accurate. Alternatively, one can apply a Bayesian approach using Markov chain Monte Carlo estimation methods. This simulation study compared estimation quality using Bayesian and frequentist approaches in the context of a multilevel latent covariate model. Continuous and dichotomous variables were examined because it is not yet known how different types of outcomes—most notably categorical—affect parameter recovery in this modeling context. Within the Bayesian estimation framework, the impact of diffuse, weakly informative, and informative prior distributions were compared. Findings indicated that Bayesian estimation may be used to overcome convergence problems and improve parameter estimate bias. Results highlight the differences in estimation quality between dichotomous and continuous variable models and the importance of prior distribution choice for cluster-level random effects.     

A Two-Stage Approach to Synthesizing Covariance Matrices in Meta-Analytic Structural Equation Modeling

A great obstacle for wider use of structural equation modeling (SEM) has been the difficulty in handling categorical variables. Two data sets with known structure between 2 related binary outcomes and 4 independent binary variables were generated. Four SEM strategies and resulting apparent validity were tested: robust maximum likelihood (ML), tetrachoric correlation matrix input followed by SEM ML analysis, SEM ML estimation for the sum of squares and cross-products (SSCP) matrix input obtained by the log-linear model that treated all variables as dependent, and asymptotic distribution-free (ADF) SEM estimation. SEM based on the SSCP matrix obtained by the log-linear model and SEM using robust ML estimation correctly identified the structural relation between the variables. SEM using ADF added an extra parameter. SEM based on tetrachoric correlation input did not specify the data generating process correctly. Apparent validity was similar for all models presented. Data transformation used in log-linear modeling can serve as an input for SEM.     

Using Structural Equation Modeling to Examine Group Differences in Assessment Booklet Designs with Sparse Data

The current research demonstrates the effectiveness of using structural equation modeling (SEM) for the investigation of subgroup differences with sparse data designs where not every student takes every item. Simulations were conducted that reflected missing data structures like those encountered in large survey assessment programs (e.g., National Assessment of Educational Progress). A maximum likelihood method of estimation was implemented that allowed all data to be used without performing any imputation. A multiple indicators multiple causes (MIMIC) model was used to examine group differences. There was no detriment to the estimation of the MIMIC model parameters under sparse data design conditions when compared to the design without missing data. The overall size of samples had more influence on the variability of estimates than did the data design.     

A Regularized GLS for Structural Equation Modeling

Ill conditioning of covariance and weight matrices used in structural equation modeling (SEM) is a possible source of inadequate performance of SEM statistics in nonasymptotic samples. A maximum a posteriori (MAP) covariance matrix is proposed for weight matrix regularization in normal theory generalized least squares (GLS) estimation. Maximum likelihood (ML), GLS, and regularized GLS test statistics (RGLS and rGLS) are studied by simulation in a 15-variable, 3-factor model with 15 levels of sample size varying from 60 to 100,000. A key result showed that in terms of nominal rejection rates, RGLS outperformed ML at all sample sizes below 500, and GLS at most sample sizes below 500. In larger samples, their performance was equivalent. The second regularization methodology (rGLS) performed well asymptotically, but poorly in small samples. Regularization in SEM deserves further study.     

The Impact of Moderate Priors For Bayesian Estimation and Testing of Item Factor Analysis Models When Maximum Likelihood is Unsuitable

In psychological research, available data are often insufficient to estimate item factor analysis (IFA) models using traditional estimation methods, such as maximum likelihood (ML) or limited information estimators. Bayesian estimation with common-sense, moderately informative priors can greatly improve efficiency of parameter estimates and stabilize estimation. There are a variety of methods available to evaluate model fit in a Bayesian framework; however, past work investigating Bayesian model fit assessment for IFA models has assumed flat priors, which have no advantage over ML in limited data settings. In this paper, we evaluated the impact of moderately informative priors on ability to detect model misfit for several candidate indices: posterior predictive checks based on the observed score distribution, leave-one-out cross-validation, and widely available information criterion (WAIC). We found that although Bayesian estimation with moderately informative priors is an excellent aid for estimating challenging IFA models, methods for testing model fit in these circumstances are inadequate.     

Digital ITEMS Module 2: Scale Reliability in Structural Equation Modeling

In this ITEMS module, we frame the topic of scale reliability within a confirmatory factor analysis and structural equation modeling (SEM) context and address some of the limitations of Cronbach's α. This modeling approach has two major advantages: (1) it allows researchers to make explicit the relation between their items and the latent variables representing the constructs those items intend to measure, and (2) it facilitates a more principled and formal practice of scale reliability evaluation. Specifically, we begin the module by discussing key conceptual and statistical foundations of the classical test theory model and then framing it within an SEM context; we do so first with a single item and then expand this approach to a multi‐item scale. This allows us to set the stage for presenting different measurement structures that might underlie a scale and, more importantly, for assessing and comparing those structures formally within the SEM context. We then make explicit the connection between measurement model parameters and different measures of reliability, emphasizing the challenges and benefits of key measures while ultimately endorsing the flexible McDonald's ω over Cronbach's α. We then demonstrate how to estimate key measures in both a commercial software program (Mplus) and three packages within an open‐source environment (R). In closing, we make recommendations for practitioners about best practices in reliability estimation based on the ideas presented in the module.     

Bayesian Data-Model Fit Assessment for Structural Equation Modeling

Bayesian approaches to modeling are receiving an increasing amount of attention in the areas of model construction and estimation in factor analysis, structural equation modeling (SEM), and related latent variable models. However, model diagnostics and model criticism remain relatively understudied aspects of Bayesian SEM. This article describes and illustrates key features of Bayesian approaches to model diagnostics and assessing data–model fit of structural equation models, discussing their merits relative to traditional procedures.     

A Structural Equation Modeling Evaluation of the General Model of Instructional Communication

The General Model of Instructional Communication introduced by McCroskey, Valencic, and Richmond (2004) is supported in its original conception by canonical data. This study, however, uses structural equation modeling (SEM) to provide a more detailed analysis. Although the model as originally hypothesized fits the data poorly, analysis of the SEM results suggests adjustments to the original model that substantially improve the model's fit. The revised model accounts for significant portions of the variance in the outcome variables, provides a more detailed explanation of the relationships involved, and has implications for future research. Bootstrapped parameter estimates suggest that the results are replicable.     

Introduction to Structural Equation Modeling: Issues and Practical Considerations

Structural equation modeling (SEM) is a versatile statistical modeling tool. Its estimation techniques, modeling capacities, and breadth of applications are expanding rapidly. This module introduces some common terminologies. General steps of SEM are discussed along with important considerations in each step. Simple examples are provided to illustrate some of the ideas for beginners. In addition, several popular specialized SEM software programs are briefly discussed with regard to their features and availability. The intent of this module is to focus on foundational issues to inform readers of the potentials as well as the limitations of SEM. Interested readers are encouraged to consult additional references for advanced model types and more application examples.     

Comparison of Frequentist and Bayesian Regularization in Structural Equation Modeling

Research in regularization, as applied to structural equation modeling (SEM), remains in its infancy. Specifically, very little work has compared regularization approaches across both frequentist and Bayesian estimation. The purpose of this study was to address just that, demonstrating both similarity and distinction across estimation frameworks, while specifically highlighting more recent developments in Bayesian regularization. This is accomplished through the use of two empirical examples that demonstrate both ridge and lasso approaches across both frequentist and Bayesian estimation, along with detail regarding software implementation. We conclude with a discussion of future research, advocating for increased evaluation and synthesis across both Bayesian and frequentist frameworks.     

Multilevel Factorial Invariance in n-Level Structural Equation Modeling (nSEM)

We present a multigroup multilevel confirmatory factor analysis (CFA) model and a procedure for testing multilevel factorial invariance in n-level structural equation modeling (nSEM). Multigroup multilevel CFA introduces a complexity when the group membership at the lower level intersects the clustered structure, because the observations in different groups but in the same cluster are not independent of one another. nSEM provides a framework in which the multigroup multilevel data structure is represented with the dependency between groups at the lower level properly taken into account. The procedure for testing multilevel factorial invariance is illustrated with an empirical example using an R package xxm2.     

Modeling of Nonrecursive Structural Equation Models With Categorical Indicators

Nonrecursive structural equation models generally take the form of feedback loops, involving 2 latent variables that are connected by 2 unidirectional paths, 1 starting with each variable and terminating in the other variable. Nonrecursive models belong to a larger class of path models that require the use of instrumental variables (IVs) to achieve model identification. Prior research has focused on SEM parameter estimation with IVs when indicators were continuous and normally distributed. Much less is known about how estimators function in the presence of categorical indicators, which are commonly used in the social sciences, such as with cognitive and affective instruments. In this study, there was specific interest in comparing the 2-stage least squares (2SLS) estimator and its categorical variant to other recommended estimators. This study compares the performance of several estimation approaches for fitting structural equation models with categorical indicator variables when IVs are necessary to obtain proper model estimates. Across conditions, 1 extension of the nonlinear 2SLS (N2SLS) approach, the nonlinear 3-stage least squares (N3SLS), which accounts for correlated errors among regressors within each model (as does the N2SLS), as well as correlations of errors across models, which N2SLS does not, appears to work the best among methods compared.