The Model Size Effect in SEM: Inflated Goodness-of-Fit Statistics Are Due to the Size of the Covariance Matrix

The size of a model has been shown to critically affect the goodness of approximation of the model fit statistic T to the asymptotic chi-square distribution in finite samples. It is not clear, however, whether this “model size effect” is a function of the number of manifest variables, the number of free parameters, or both. It is demonstrated by means of 2 Monte Carlo computer simulation studies that neither the number of free parameters to be estimated nor the model degrees of freedom systematically affect the T statistic when the number of manifest variables is held constant. Increasing the number of manifest variables, however, is associated with a severe bias. These results imply that model fit drastically depends on the size of the covariance matrix and that future studies involving goodness-of-fit statistics should always consider the number of manifest variables, but can safely neglect the influence of particular model specifications.     

On Fitting a Multivariate Two-Part Latent Growth Model

The purpose of this study was to explore the influence of the number of targets specified on the quality of exploratory factor analysis solutions with a complex underlying structure and incomplete substantive measurement theory. Three Monte Carlo studies were performed based on the ratio of the number of observed variables to the number of underlying factors. Within each study, communality, sample size, and the number of targets were manipulated. Outcomes included a measure of congruence and a measure of variability with regard to the rotated pattern matrix. The magnitude of the main effect for the influence of the number of targets on congruence and variability ranged from moderate to large. The magnitude of the interaction between the number of targets and level of communality ranged from small to moderate with regard to congruence and variability. Consistent with theoretical expectations, the minimum number of targets to specify to be reasonably assured of obtaining an accurate solution varied across study conditions.     

A Comparison of CFA,ESEM, and BSEM in Test Structure Analysis

Minor cross-loadings on non-targeted factors are often found in psychological or other instruments. Forcing them to zero in confirmatory factor analyses (CFA) leads to biased estimates and distorted structures. Alternatively, exploratory structural equation modeling (ESEM) and Bayesian structural equation modeling (BSEM) have been proposed. In this research, we compared the performance of the traditional independent-clusters-confirmatory-factor-analysis (ICM-CFA), the nonstandard CFA, ESEM with the Geomin- or Target-rotations, and BSEMs with different cross-loading priors (correct; small- or large-variance priors with zero mean) using simulated data with cross-loadings. Four factors were considered: the number of factors, the size of factor correlations, the cross-loading mean, and the loading variance. Results indicated that ICM-CFA performed the worst. ESEMs were generally superior to CFAs but inferior to BSEM with correct priors that provided the precise estimation. BSEM with large- or small-variance priors performed similarly while the prior mean for cross-loadings was more important than the prior variance.     

Growth Curve Modeling to Studying Change: A Comparison of Approaches Using Longitudinal Dyadic Data With Distinguishable Dyads

Although methodology articles have increasingly emphasized the need to analyze data from two members of a dyad simultaneously, the most popular method in substantive applications is to examine dyad members separately. This might be due to the underappreciation of the extra information simultaneous modeling strategies can provide. Therefore, the goal of this study was to compare multiple growth curve modeling approaches for longitudinal dyadic data (LDD) in both structural equation modeling and multilevel modeling frameworks. Models separately assessing change over time for distinguishable dyad members are compared to simultaneous models fitted to LDD from both dyad members. Furthermore, we compared the simultaneous default versus dependent approaches (whether dyad pairs’ Level 1 [or unique] residuals are allowed to covary and differ in variance). Results indicated that estimates of variance and covariance components led to conflicting results. We recommend the simultaneous dependent approach for inferring differences in change over time within a dyad.     

The Reliability of Observations of Talkativeness and Social Contact among Nursery School Children by the “Short Time Sample” Technique

In 1958, Page conducted a large multiple experiment: 74 teachers gave one class its normal quiz, scored and graded it in the usual way, assigned three comment treatments to students in stratified-random blocks, and then reported scores from the next objective quiz. There was a highly significant effect of comments. Others have borrowed some study features, with results that have appeared mixed. Here, a critical overall analysis shows much agreement with the ordered hypothesis of comments and with specified comments over no comments (p < .01). Despite great variety of designs and subtlety of effect, results broadly support teachers who comment. A typical effect size is demonstrated for ranks, and lessons are taken about the proper strategies for designs and the future of such research.     

Exploratory and Confirmatory Factor Analysis of the School Level Environment Questionnaire (SLEQ)

The purpose of the present study was to validate an existing school environment instrument, the School Level Environment Questionnaire (SLEQ). The SLEQ consists of 56 items, with seven items in each of eight scales. One thousand, one hundred and six (1106) teachers in 59 elementary schools in a southwestern USA public school district completed the instrument. An exploratory factor analysis was undertaken for a random sample of half of the completed surveys. Using principal axis factoring with oblique rotation, this analysis suggested that 13 items should be dropped and that the remaining 43 items could best be represented by seven rather than eight factors. A confirmatory factor analysis was run with the other half of the original sample using structural equation modeling. Examination of the fit indices indicated that the model came close to fitting the data, with goodness-of-fit (GOF) coefficients just below recommended levels. A second model was then run with two of the seven factors, with their associated items removed. That left five factors with 35 items. Model fit was improved. A third model was tried, using the same five factors with 35 items but with correlated residuals between some of the items within a factor. This model seemed to fit the data well, with GOF coefficients in recommended ranges. These results led to a refined, more parsimonious version of the SLEQ that was then used in a larger study. Future research is needed to see if this model would fit other samples in different elementary schools and in secondary schools both in the USA and in other countries. This revised version was published online in July 2006 with corrections to the Cover Date.     

Revisiting the Model Size Effect in Structural Equation Modeling

Fitting a large structural equation modeling (SEM) model with moderate to small sample sizes results in an inflated Type I error rate for the likelihood ratio test statistic under the chi-square reference distribution, known as the model size effect. In this article, we show that the number of observed variables (p) and the number of free parameters (q) have unique effects on the Type I error rate of the likelihood ratio test statistic. In addition, the effects of p and q cannot be fully explained using degrees of freedom (df). We also evaluated the performance of 4 correctional methods for the model size effect, including Bartlett’s (1950), Swain’s (1975), and Yuan’s (2005) corrected statistics, and Yuan, Tian, and Yanagihara’s (2015) empirically corrected statistic. We found that Yuan et al.’s (2015) empirically corrected statistic generally yields the best performance in controlling the Type I error rate when fitting large SEM models.     

Investigating Factorial Invariance of Latent Variables Across Populations When Manifest Variables Are Missing Completely

Difficulties arise in multiple-group evaluations of factorial invariance if particular manifest variables are missing completely in certain groups. Ad hoc analytic alternatives can be used in such situations (e.g., deleting manifest variables), but some common approaches, such as multiple imputation, are not viable. At least 3 solutions to this problem are viable: analyzing differing sets of variables across groups, using pattern mixture approaches, and a new method using random number generation. The latter solution, proposed in this article, is to generate pseudo-random normal deviates for all observations for manifest variables that are missing completely in a given sample and then to specify multiple-group models in a way that respects the random nature of these values. An empirical example is presented in detail comparing the 3 approaches. The proposed solution can enable quantitative comparisons at the latent variable level between groups using programs that require the same number of manifest variables in each group.     

Alternative Methods for Assessing Mediation in Multilevel Data: The Advantages of Multilevel SEM

Multilevel modeling (MLM) is a popular way of assessing mediation effects with clustered data. Two important limitations of this approach have been identified in prior research and a theoretical rationale has been provided for why multilevel structural equation modeling (MSEM) should be preferred. However, to date, no empirical evidence of MSEM's advantages relative to MLM approaches for multilevel mediation analysis has been provided. Nor has it been demonstrated that MSEM performs adequately for mediation analysis in an absolute sense. This study addresses these gaps and finds that the MSEM method outperforms 2 MLM-based techniques in 2-level models in terms of bias and confidence interval coverage while displaying adequate efficiency, convergence rates, and power under a variety of conditions. Simulation results support prior theoretical work regarding the advantages of MSEM over MLM for mediation in clustered data.