Bayesian Data-Model Fit Assessment for Structural Equation Modeling

Bayesian approaches to modeling are receiving an increasing amount of attention in the areas of model construction and estimation in factor analysis, structural equation modeling (SEM), and related latent variable models. However, model diagnostics and model criticism remain relatively understudied aspects of Bayesian SEM. This article describes and illustrates key features of Bayesian approaches to model diagnostics and assessing data–model fit of structural equation models, discussing their merits relative to traditional procedures.     

Application of Exploratory Structural Equation Modeling to Evaluate the Academic Motivation Scale

In this research, the authors examined the construct validity of scores of the Academic Motivation Scale using exploratory structural equation modeling. Study 1 and Study 2 involved 1,416 college students and 4,498 high school students, respectively. First, results of both studies indicated that the factor structure tested with exploratory structural equation modeling provides better fit to the data than the one tested with confirmatory factor analysis. Second, the factor structure was gender invariant in the exploratory structural equation modeling framework. Third, the pattern of convergent and divergent correlations among Academic Motivation Scale factors was more in line with theoretical expectations when computed with exploratory structural equation modeling rather than confirmatory factor analysis. Fourth, the configuration of convergent and divergent correlations connecting each Academic Motivation Scale factors to a validity criterion was more in line with theoretical expectations with exploratory structural equation modeling than with confirmatory factor analysis.     

A Note on Structural Equation Modeling Estimates of Reliability

Reliability can be estimated using structural equation modeling (SEM). Two potential problems with this approach are that estimates may be unstable with small sample sizes and biased with misspecified models. A Monte Carlo study was conducted to investigate the quality of SEM estimates of reliability by themselves and relative to coefficient alpha. The SEM approach showed minimal bias when the model was correctly specified if items were relatively well defined by their underlying factor(s). They tended to demonstrate somewhat greater bias when the model was misspecified, particularly underspecified. Overall, SEM estimates were more stable than anticipated. Researchers are more likely to obtain accurate estimates of reliability using SEM by conducting large-sample studies with well-constructed scales and critically assessing model fit.     

Dealing With Item Nonresponse in Large‐Scale Cognitive Assessments: The Impact of Missing Data Methods on Estimated Explanatory Relationships

Competence data from low‐stakes educational large‐scale assessment studies allow for evaluating relationships between competencies and other variables. The impact of item‐level nonresponse has not been investigated with regard to statistics that determine the size of these relationships (e.g., correlations, regression coefficients). Classical approaches such as ignoring missing values or treating them as incorrect are currently applied in many large‐scale studies, while recent model‐based approaches that can account for nonignorable nonresponse have been developed. Estimates of item and person parameters have been demonstrated to be biased for classical approaches when missing data are missing not at random (MNAR). In our study, we focus on parameter estimates of the structural model (i.e., the true regression coefficient when regressing competence on an explanatory variable), simulating data according to various missing data mechanisms. We found that model‐based approaches and ignoring missing values performed well in retrieving regression coefficients even when we induced missing data that were MNAR. Treating missing values as incorrect responses can lead to substantial bias. We demonstrate the validity of our approach empirically and discuss the relevance of our results.     

Evaluation of an Approximate Method for Synthesizing Covariance Matrices for Use in Meta-Analytic SEM

Meta-analytic structural equation modeling (MA-SEM) is increasingly being used to assess model-fit for variables' interrelations synthesized across studies. MA-SEM researchers have analyzed synthesized correlation matrices using structural equation modeling (SEM) estimation that is designed for covariance matrices. This can produce incorrect model-fit chi-square statistics, standard error estimates (Cudeck, 1989), or both for parameters that are not scale free or that describe a scale-noninvariant model unless corrected SEM estimation is used to analyze the correlations. This study introduced univariate and multivariate approximate methods for synthesizing covariance matrices for use in MA-SEM. A simulation study assessed the approximate methods by estimating parameters in a scale-noninvariant model using synthesized covariances versus synthesized correlations with and without the appropriate corrections. Standard error bias was noted only for uncorrected analyses of pooled correlations. Chi-square model-fit statistics were overly conservative except when covariance matrices were analyzed. Benefits and limitations of this approximate method are presented and discussed.     

Effects of sample size,estimation methods,and model specification on structural equation modeling fit indexes

A Monte Carlo simulation study was conducted to investigate the effects on structural equation modeling (SEM) fit indexes of sample size, estimation method, and model specification. Based on a balanced experimental design, samples were generated from a prespecified population covariance matrix and fitted to structural equation models with different degrees of model misspecification. Ten SEM fit indexes were studied. Two primary conclusions were suggested: (a) some fit indexes appear to be noncomparable in terms of the information they provide about model fit for misspecified models and (b) estimation method strongly influenced almost all the fit indexes examined, especially for misspecified models. These 2 issues do not seem to have drawn enough attention from SEM practitioners. Future research should study not only different models vis‐à‐vis model complexity, but a wider range of model specification conditions, including correctly specified models and models specified incorrectly to varying degrees.     

Efficacy of the indirect approach for estimating structural equation models with missing data: A comparison of five methods

Incomplete or missing data are routinely encountered in structural equation problems. Although current literature supports the use of a direct approach for modeling the missing values in a structural equation model, many situations are not applicable for the effective use of this approach. This leaves the use of an indirect approach for dealing with missing information. There is a general lack of knowledge regarding the efficacy of the use of the indirect approach in structural equation modeling. This article assesses the efficacy of five indirect methods for estimating parameters in a structural equation model with various levels of missing data.     

Comparing Exploratory Structural Equation Modeling and Existing Approaches for Multiple Regression with Latent Variables

Exploratory structural equation modeling (ESEM) is an approach for analysis of latent variables using exploratory factor analysis to evaluate the measurement model. This study compared ESEM with two dominant approaches for multiple regression with latent variables, structural equation modeling (SEM) and manifest regression analysis (MRA). Main findings included: (1) ESEM in general provided the least biased estimation of the regression coefficients; SEM was more biased than MRA given large cross-factor loadings. (2) MRA produced the most precise estimation, followed by ESEM and then SEM. (3) SEM was the least powerful in the significance tests; statistical power was lower for ESEM than MRA with relatively small target-factor loadings, but higher for ESEM than MRA with relatively large target-factor loadings. (4) ESEM showed difficulties in convergence and occasionally created an inflated type I error rate under some conditions. ESEM is recommended when non-ignorable cross-factor loadings exist.     

Are teachers’ approaches to teaching responsive to individual student variation? A two-level structural equation modeling

In the framework of teacher’s approaches to teaching, this study investigates the relationship between student-related variables (i.e., study time, class absence, domain knowledge, and homework completion), students’ approaches to learning, and teachers’ approaches to teaching using structural equation modeling (SEM) with two independent data samples. The participants were 61 biology teachers and their corresponding 1,518 high school students (12th grade). The first sample was used to fit the model, and the second sample was used to analyze the consistency of the data derived from the first sample. Using a two-level SEM analysis, we established whether the effects found at the individual level varied significantly at class level. The students’ approaches to learning were related to the teachers’ approaches to teaching as a function of the hypotheses established in the model, although the effect size was smaller than expected. However, approximately 48 % of the variance of the surface approach and 46 % of the deep approach sat at class level. At the individual level, the results of this study suggest that students’ approaches to learning significantly explain their teachers’ approaches to teaching and, thus, constitute important contextual variables. At the class level, the way students learn appears to be closely associated with class-related variables. Our data stresses the importance of promoting educational opportunities (e.g., school-based courses) for teachers to reflect upon the teaching methodologies used in class.     

Individually Weighted-Average Models: Testing a Taxonomic SEM Approach Across Different Multidimensional/Global Constructs Because the Weights “Don’t Make No Nevermind”

From the time of William James, psychologists have posited individually importance-weighted-average models (IWAMs) in which weighting specific attributes by individual measures of importance improves prediction of the global outcome measures. Because IWAMs cause much confusion, we briefly review a general taxonomic paradigm and structural equation models for testing IWAMs, and demonstrate its application for 2 simulated and 3 diverse “real” data applications (multidimensional measures of self-concept, quality of life, and job satisfaction). Consistent across the real data applications and previous research more generally, there is surprisingly little support for IWAMs when tested appropriately. In these diverse tests of IWAMs we integrate new approaches such as exploratory structural equation modeling (SEM), alternative approaches to constructing latent interactions, application of bifactor models, modeling method and item-wording effects, and the juxtaposition of model evaluation in relation to goodness of fit (typically used in SEM studies) and variance explained (typically used in multiple regression tests of IWAMs).     

Alternative approaches to structural modeling of ordinal data: A Monte Carlo study

In practice, several measures of association are used when analyzing structural equation models with ordinal variables: ordinary Pearson correlations (PE approach), polychoric and polyserial correlations (PO approach), and conditional polychoric correlations (CPO approach). In the case of structural equation models without latent variables, the literature has shown that the PE approach is outperformed by the alternatives. In this article we report a Monte Carlo study showing the comparative performance of the aforementioned alternative approaches under deviations from their respective assumptions in the case of structural equation models with latent variables when attention is restricted to point estimates of model parameters. The CPO approach is shown to be the most robust against nonnormality. It is also robust to randomness of the exogenous variables, but not to the existence of measurement errors in them. The PO approach lacks robustness against nonnormality. The PE approach lacks robustness against transformation errors but otherwise it can perform about as well as the alternative approaches.     

An SAS Macro for Implementing the Modified Bollen-Stine Bootstrap for Missing Data: Implementing the Bootstrap Using Existing Structural Equation Modeling Software

The Bollen-Stine bootstrap can be used to correct for standard error and fit statistic bias that occurs in structural equation modeling (SEM) applications due to nonnormal data. The purpose of this article is to demonstrate the use of a custom SAS macro program that can be used to implement the Bollen-Stine bootstrap with existing SEM software. Although this article focuses on missing data, the macro can be used with complete data sets as well. A series of heuristic analyses are presented, along with detailed programming instructions for each of the commercial SEM software packages.     

Sensitivity Analysis of Multiple Informant Models When Data Are Not Missing at Random

Missing data are common in studies that rely on multiple informant data to evaluate relationships among variables for distinguishable individuals clustered within groups. Estimation of structural equation models using raw data allows for incomplete data, and so all groups can be retained for analysis even if only 1 member of a group contributes data. Statistical inference is based on the assumption that data are missing completely at random or missing at random. Importantly, whether or not data are missing is assumed to be independent of the missing data. A saturated correlates model that incorporates correlates of the missingness or the missing data into an analysis and multiple imputation that might also use such correlates offer advantages over the standard implementation of SEM when data are not missing at random because these approaches could result in a data analysis problem for which the missingness is ignorable. This article considers these approaches in an analysis of family data to assess the sensitivity of parameter estimates and statistical inferences to assumptions about missing data, a strategy that could be easily implemented using SEM software.     

Longitudinal stability of latent means and individual differences: A unified approach

We examined the stability of responses to a multi‐item self‐esteem scale collected on five occasions over an 8‐year period. A wide variety of approaches were critically examined that considered the stability of means, individual differences (i.e., test‐retest correlations), and factor structures using traditional approaches (e.g., ANOVA and correlations) and structural equation models. Structural equation models based on multiple indicators provided a unified analytic approach for evaluating different aspects of stability and offered important advantages over traditional approaches. We describe a hierarchy of invariances and the nature of interpretations that are justified by different patterns of factor structure invariance associated with each level. We conclude that the assumptions underlying the typical repeated‐measures ANOVA approach to testing mean differences in longitudinal data are far more restrictive, less easily tested, and less likely to be met than those in the structural equation modeling approach advocated here, and that the use of ANOVA for this purpose requires a huge leap of faith that can rarely be justified on logical or empirical grounds.     

Maximum Likelihood Dynamic Factor Modeling for Arbitrary N and T Using SEM

This article has 3 objectives that build on each other. First, we demonstrate how to obtain maximum likelihood estimates for dynamic factor models (the direct autoregressive factor score model) with arbitrary T and N by means of structural equation modeling (SEM) and compare the approach to existing methods. Second, we go beyond standard time series analysis (T large and N = 1) and conventional SEM (N large and T = 1 or small) by integrating both approaches. The resulting combined model offers a variety of new modeling options including a direct test of the ergodicity hypothesis, according to which the factorial structure of an individual observed at many time points is identical to the factorial structure of a group of individuals observed at a single point in time. Third, we illustrate the flexibility of SEM time series modeling by extending the approach to account for complex error structures. We end with a discussion of current limitations and future applications of SEM-based time series modeling for arbitrary T and N.     

Writing and reading: connections between language by hand and language by eye

Four approaches to the investigation of connections between language by hand and language by eye are described and illustrated with studies from a decade-long research program. In the first approach, multigroup structural equation modeling is applied to reading and writing measures given to typically developing writers to examine unidirectional and bidirectional relationships between specific components of the reading and writing systems. In the second approach, structural equation modeling is applied to a multivariate set of language measures given to children and adults with reading and writing disabilities to examine how the same set of language processes is orchestrated differently to accomplish specific reading or writing goals, and correlations between factors are evaluated to examine the level at which the language-by-hand system and the language-by-eye system communicate most easily. In the third approach, mode of instruction and mode of response are systematically varied in evaluating effectiveness of treating reading disability with and without a writing component. In the fourth approach, functional brain imaging is used to investigate residual spelling problems in students whose problems with word decoding have been remediated. The four approaches support a model in which language by hand and language by eye are separate systems that interact in predictable ways.     

Integration of Stochastic Differential Equations Using Structural Equation Modeling: A Method to Facilitate Model Fitting and Pedagogy

Stochastic differential equation (SDE) models are a promising method for modeling intraindividual change and variability. Applications of SDEs in the social sciences are relatively limited, as these models present conceptual and programming challenges. This article presents a novel method for conceptualizing SDEs. This method uses structural equation modeling (SEM) conventions to simplify SDE specification, the flexibility of SEM to expand the range of SDEs that can be fit, and SEM diagram conventions to facilitate the teaching of SDE concepts. This method is a variation of latent difference scores (McArdle, 2009; McArdle & Hamagami, 2001) and the oversampling approach (Singer, 2012), and approximates the advantages of analytic methods such as the exact discrete model (Oud & Jansen, 2000) while retaining the modeling fiexibility of methods such as latent differential equation modeling (Boker, Neale, & Rausch, 2004). A simulation and empirical example are presented to illustrate that this method can be implemented on current computing hardware, produces good approximations of analytic solutions, and can flexibly accommodate novel models.     

Applying structural equation modeling in the context of the theory of reasoned action: Some problems and solutions

This study identifies and attempts to solve problems encountered in applications of structural equation modeling (SEM) to the theory of reasoned action. This theory is often used in social psychology and aims at explaining and predicting behavior. The few studies that test this theory with SEM have, in general, 2 methodological problems, which cast serious doubt on the validity of the conclusions. The first problem is that in most of the tests the data do not fit the model. The second problem is that part of the theory is formulated by multiplying 2 variables, which implies that the results are highly dependent on the arbitrarily chosen scale values. These problems are illustrated with a secondary analysis of survey data gathered by Burnkrant and Page (1988) and by new data presented in this study. In this article, an alternative model specification is proposed that strongly improves the fit of the data, but leaves intact the structural part of the model being tested. It is also advisable to omit 1 of the variables that forms part of the multiplicative composite.     

Model specification searches in structural equation modeling using tabu search

This article introduces an alternative structural equation modeling (SEM) specification search approach that is based on the Tabu search procedure. Using data with known structure, the performance of the Tabu search is illustrated. The results demonstrate the capabilities of the Tabu search procedure for conducting specification searches in SEM.     

Using Structural Equation Modeling to Examine Group Differences in Assessment Booklet Designs with Sparse Data

The current research demonstrates the effectiveness of using structural equation modeling (SEM) for the investigation of subgroup differences with sparse data designs where not every student takes every item. Simulations were conducted that reflected missing data structures like those encountered in large survey assessment programs (e.g., National Assessment of Educational Progress). A maximum likelihood method of estimation was implemented that allowed all data to be used without performing any imputation. A multiple indicators multiple causes (MIMIC) model was used to examine group differences. There was no detriment to the estimation of the MIMIC model parameters under sparse data design conditions when compared to the design without missing data. The overall size of samples had more influence on the variability of estimates than did the data design.