Examining Chi-Square Test Statistics Under Conditions of Large Model Size and Ordinal Data

This study examined the effect of model size on the chi-square test statistics obtained from ordinal factor analysis models. The performance of six robust chi-square test statistics were compared across various conditions, including number of observed variables (p), number of factors, sample size, model (mis)specification, number of categories, and threshold distribution. Results showed that the unweighted least squares (ULS) robust chi-square statistics generally outperform the diagonally weighted least squares (DWLS) robust chi-square statistics. The ULSM estimator performed the best overall. However, when fitting ordinal factor analysis models with a large number of observed variables and small sample size, the ULSM-based chi-square tests may yield empirical variances that are noticeably larger than the theoretical values and inflated Type I error rates. On the other hand, when the number of observed variables is very large, the mean- and variance-corrected chi-square test statistics (e.g., based on ULSMV and WLSMV) could produce empirical variances conspicuously smaller than the theoretical values and Type I error rates lower than the nominal level, and demonstrate lower power rates to reject misspecified models. Recommendations for applied researchers and future empirical studies involving large models are provided.     

Testing Recursive Path Models With Correlated Errors Using D-Separation

This article shows how to extend the inferential test of Shipley (2000b), which is applicable to recursive path models without correlated errors (a directed acyclic graph [DAG] model), to a class of recursive path models that include correlated errors (a semi-Markov model). The path model is first converted to a partial ancestral graph (PAG) and then, for PAGs that do not require latent variables, an inducing path DAG is obtained that is equivalent in its conditional independence relations to the original path model. The null probabilities of the k tests of independence that are implied by this DAG are combined using Fisher's test statistic C = -2ΣLn(pi), which is distributed as a chi-square variate with 2k degrees of freedom.     

Application of Latent Growth Curve Analysis With Categorical Responses in Social Behavioral Research

Latent growth modeling allows social behavioral researchers to investigate within-person change and between-person differences in within-person change. Typically, conventional latent growth curve models are applied to continuous variables, where the residuals are assumed to be normally distributed, whereas categorical variables (i.e., binary and ordinal variables), which do not hold to normal distribution assumptions, have rarely been used. This article describes the latent growth curve model with categorical variables, and illustrates applications using Mplus software that are applicable to social behavioral research. The illustrations use marital instability data from the Iowa Youth and Family Project. We close with recommendations for the specification and parameterization of growth models that use both logit and probit link functions.     

Goodness of Fit in Item Factor Analysis: Effect of the Number of Response Alternatives

The power of the chi-square test statistic used in structural equation modeling decreases as the absolute value of excess kurtosis of the observed data increases. Excess kurtosis is more likely the smaller the number of item response categories. As a result, fit is likely to improve as the number of item response categories decreases, regardless of the true underlying factor structure or χ²-based fit index used to examine model fit. Equivalently, given a target value of approximate fit (e.g., root mean square error of approximation ≤ .05) a model with more factors is needed to reach it as the number of categories increases. This is true regardless of whether the data are treated as continuous (common factor analysis) or as discrete (ordinal factor analysis). We recommend using a large number of response alternatives (≥ 5) to increase the power to detect incorrect substantive models.     

Using Fixed Thresholds with Grouped Data in Structural Equation Modeling

Valuable methods have been developed for incorporating ordinal variables into structural equation models using a latent response variable formulation. However, some model parameters, such as the means and variances of latent factors, can be quite difficult to interpret because the latent response variables have an arbitrary metric. This limitation can be particularly problematic in growth models, where the means and variances of the latent growth parameters typically have important substantive meaning when continuous measures are used. However, these methods are often applied to grouped data, where the ordered categories actually represent an interval-level variable that has been measured on an ordinal scale for convenience. The method illustrated in this article shows how category threshold values can be incorporated into the model so that interpretation is more meaningful, with particular emphasis given to the application of this technique with latent growth models.     

Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models

Ordinal variables are common in many empirical investigations in the social and behavioral sciences. Researchers often apply the maximum likelihood method to fit structural equation models to ordinal data. This assumes that the observed measures have normal distributions, which is not the case when the variables are ordinal. A better approach is to use polychoric correlations and fit the models using methods such as unweighted least squares (ULS), maximum likelihood (ML), weighted least squares (WLS), or diagonally weighted least squares (DWLS). In this simulation evaluation we study the behavior of these methods in combination with polychoric correlations when the models are misspecified. We also study the effect of model size and number of categories on the parameter estimates, their standard errors, and the common chi-square measures of fit when the models are both correct and misspecified. When used routinely, these methods give consistent parameter estimates but ULS, ML, and DWLS give incorrect standard errors. Correct standard errors can be obtained for these methods by robustification using an estimate of the asymptotic covariance matrix W of the polychoric correlations. When used in this way the methods are here called RULS, RML, and RDWLS.     

A Specification Error Test That Uses Instrumental Variables to Detect Latent Quadratic and Latent Interaction Effects

The relations between the latent variables in structural equation models are typically assumed to be linear in form. This article aims to explain how a specification error test using instrumental variables (IVs) can be employed to detect unmodeled interactions between latent variables or quadratic effects of latent variables. An empirical example is presented, and the results of a simulation study are reported to evaluate the sensitivity and specificity of the test and compare it with the commonly employed chi-square model test. The results show that the proposed test can identify most unmodeled latent interactions or latent quadratic effects in moderate to large samples. Furthermore, its power is higher when the number of indicators used to define the latent variables is large. Altogether, this article shows how the IV-based test can be applied to structural equation models and that it is a valuable tool for researchers using structural equation models.     

Using the friedman method of ranks for model comparison in structural equation modeling

Empirical researchers maximize their contribution to theory development when they compare alternative theory‐inspired models under the same conditions. Yet model comparison tools in structural equation modeling—χ² difference tests, information criterion measures, and screening heuristics—have significant limitations. This article explores the use of the Friedman method of ranks as an inferential procedure for evaluating competing models. This approach has attractive properties, including limited reliance on sample size, limited distributional assumptions, an explicit multiple comparison procedure, and applicability to the comparison of nonnested models. However, this use of the Friedman method raises important issues regarding the lack of independence of observations and the power of the test.     

Hierarchical Generalized Linear Models for the Analysis of Judge Ratings

It is known that the Rasch model is a special two-level hierarchical generalized linear model (HGLM). This article demonstrates that the many-faceted Rasch model (MFRM) is also a special case of the two-level HGLM, with a random intercept representing examinee ability on a test, and fixed effects for the test items, judges, and possibly other facets. This perspective suggests useful modeling extensions of the MFRM. For example, in the HGLM framework it is possible to model random effects for items and judges in order to assess their stability across examinees. The MFRM can also be extended so that item difficulty and judge severity are modeled as functions of examinee characteristics (covariates), for the purposes of detecting differential item functioning and differential rater functioning. Practical illustrations of the HGLM are presented through the analysis of simulated and real judge-mediated data sets involving ordinal responses.     

tetrad: A Set of Stata Commands for Confirmatory Tetrad Analysis

This article provides a brief overview of confirmatory tetrad analysis (CTA) and presents a new set of Stata commands for conducting CTA. The tetrad command allows researchers to use model-implied vanishing tetrads to test the overall fit of structural equation models (SEMs) and the relative fit of two SEMs that are tetrad-nested. An extension of the command, tetrad_matrix, allows researchers to conduct CTA using a sample covariance matrix as input rather than relying on raw data. Researchers can also use the tetrad_matrix command to input a polychoric correlation matrix and conduct CTA for SEMs involving dichotomous, ordinal, or censored outcomes. Another extension of the command, tetrad_bootstrap, provides a bootstrapped p value for the chi-square test statistic. With Stata’s recently developed commands for structural equation modeling, researchers can integrate CTA with data preparation, likelihood ratio tests for model fit, and the estimation of model parameters in a single statistical software package.     

Evaluating Interventions with Multimethod Data: A Structural Equation Modeling Approach

In many intervention and evaluation studies, outcome variables are assessed using a multimethod approach comparing multiple groups over time. In this article, we show how evaluation data obtained from a complex multitrait–multimethod–multioccasion–multigroup design can be analyzed with structural equation models. In particular, we show how the structural equation modeling approach can be used to (a) handle ordinal items as indicators, (b) test measurement invariance, and (c) test the means of the latent variables to examine treatment effects. We present an application to data from an evaluation study of an early childhood prevention program. A total of 659 children in intervention and control groups were rated by their parents and teachers on prosocial behavior and relational aggression before and after the program implementation. No mean change in relational aggression was found in either group, whereas an increase in prosocial behavior was found in both groups. Advantages and limitations of the proposed approach are highlighted.     

Comparing the Fit of Item Response Theory and Factor Analysis Models

Linear factor analysis (FA) models can be reliably tested using test statistics based on residual covariances. We show that the same statistics can be used to reliably test the fit of item response theory (IRT) models for ordinal data (under some conditions). Hence, the fit of an FA model and of an IRT model to the same data set can now be compared. When applied to a binary data set, our experience suggests that IRT and FA models yield similar fits. However, when the data are polytomous ordinal, IRT models yield a better fit because they involve a higher number of parameters. But when fit is assessed using the root mean square error of approximation (RMSEA), similar fits are obtained again. We explain why. These test statistics have little power to distinguish between FA and IRT models; they are unable to detect that linear FA is misspecified when applied to ordinal data generated under an IRT model.     

Evaluation of Test Statistics for Robust Structural Equation Modeling With Nonnormal Missing Data

A 2-stage robust procedure as well as an R package, rsem, were recently developed for structural equation modeling with nonnormal missing data by Yuan and Zhang (2012). Several test statistics that have been used for complete data analysis are employed to evaluate model fit in the 2-stage robust method. However, properties of these statistics under robust procedures for incomplete nonnormal data analysis have never been studied. This study aims to systematically evaluate and compare 5 test statistics, including a test statistic derived from normal-distribution-based maximum likelihood, a rescaled chi-square statistic, an adjusted chi-square statistic, a corrected residual-based asymptotical distribution-free chi-square statistic, and a residual-based F statistic. These statistics are evaluated under a linear growth curve model by varying 8 factors: population distribution, missing data mechanism, missing data rate, sample size, number of measurement occasions, covariance between the latent intercept and slope, variance of measurement errors, and downweighting rate of the 2-stage robust method. The performance of the test statistics varies and the one derived from the 2-stage normal-distribution-based maximum likelihood performs much worse than the other four. Application of the 2-stage robust method and of the test statistics is illustrated through growth curve analysis of mathematical ability development, using data on the Peabody Individual Achievement Test mathematics assessment from the National Longitudinal Survey of Youth 1997 Cohort.     

A New Inferential Test for Path Models Based on Directed Acyclic Graphs

This article introduces a new inferential test for acyclic structural equation models (SEM) without latent variables or correlated errors. The test is based on the independence relations predicted by the directed acyclic graph of the SEMs, as given by the concept of d-separation. A wide range of distributional assumptions and structural functions can be accommodated. No iterative fitting procedures are used, precluding problems involving convergence. Exact probability estimates can be obtained, thus permitting the testing of models with small data sets.     

Revisiting the Model Size Effect in Structural Equation Modeling

Fitting a large structural equation modeling (SEM) model with moderate to small sample sizes results in an inflated Type I error rate for the likelihood ratio test statistic under the chi-square reference distribution, known as the model size effect. In this article, we show that the number of observed variables (p) and the number of free parameters (q) have unique effects on the Type I error rate of the likelihood ratio test statistic. In addition, the effects of p and q cannot be fully explained using degrees of freedom (df). We also evaluated the performance of 4 correctional methods for the model size effect, including Bartlett’s (1950), Swain’s (1975), and Yuan’s (2005) corrected statistics, and Yuan, Tian, and Yanagihara’s (2015) empirically corrected statistic. We found that Yuan et al.’s (2015) empirically corrected statistic generally yields the best performance in controlling the Type I error rate when fitting large SEM models.     

Measurement Bias Detection Through Factor Analysis

Assessing the correctness of a structural equation model is essential to avoid drawing incorrect conclusions from empirical research. In the past, the chi-square test was recommended for assessing the correctness of the model but this test has been criticized because of its sensitivity to sample size. As a reaction, an abundance of fit indexes have been developed. The result of these developments is that structural equation modeling packages are now producing a large list of fit measures. One would think that this progression has led to a clear understanding of evaluating models with respect to model misspecifications. In this article we question the validity of approaches for model evaluation based on overall goodness-of-fit indexes. The argument against such usage is that they do not provide an adequate indication of the “size” of the model's misspecification. That is, they vary dramatically with the values of incidental parameters that are unrelated with the misspecification in the model. This is illustrated using simple but fundamental models. As an alternative method of model evaluation, we suggest using the expected parameter change in combination with the modification index (MI) and the power of the MI test.     

非精确数据包络分析研究概述

对非精确数据包络分析（IDEA）的研究进行了简要的归纳与总结．首先,简要回顾了有关I—DEA的研究进展;其次,给出了非精确数据的定义与分类,将非精确型数据分为：边界数据、弱序数据、强序数据和比值有界数据;同时给出了几个基本假设和相关模型;接着,构建了一个适用于非精确型数据的IDEA模型,并且给出了尺度变换法和变量交替法以实现将非线性规划问题转换成线性规划问题;最后,为了提高求解效率,对Zhu提出的简化变量交替法进行了改进,指出这将减少实际应用中的计算复杂度．     

The Effects of Selection Strategies for Bivariate Loglinear Smoothing Models on NEAT Equating Functions

In this study, eight statistical strategies were evaluated for selecting the parameterizations of loglinear models for smoothing the bivariate test score distributions used in nonequivalent groups with anchor test (NEAT) equating. Four of the strategies were based on significance tests of chi-square statistics (Likelihood Ratio, Pearson, Freeman-Tukey, and Cressie-Read) and four additional strategies were based on different evaluations of the Likelihood Ratio Chi-Square statistic (Akaike Information Criterion, Bayesian Information Criterion, Consistent Akaike Information Criterion, and an index traced to Goodman). The focus was the implications of the selection strategies' selection tendencies for the accuracy of chained and poststratification equating functions. The results differentiated the strategies in terms of their tendencies to select models with particular bivariate parameterizations and the implications of these tendencies for equating bias and variability .     

Longitudinal Models for Ordinal Data With Many Zeros and Varying Numbers of Response Categories

Ordinal response scales are often used to survey behaviors, including data collected in longitudinal studies. Advanced analytic methods are now widely available for longitudinal data. This study evaluates the performance of 4 methods as applied to ordinal measures that differ by the number of response categories and that include many zeros. The methods considered are hierarchical linear models (HLMs), growth mixture mixed models (GMMMs), latent class growth analysis (LCGA), and 2-part latent growth models (2PLGMs). The methods are evaluated by applying each to empirical response data in which the number of response categories is varied. The methods are applied to each outcome variable, first treating the outcome as continuous and then as ordinal, to compare the performance of the methods given both a different number of response categories and treatment of the variables as continuous versus ordinal. We conclude that although the 2PLGM might be preferred, no method might be ideal.     

Factor Models for Ordinal Variables With Covariate Effects on the Manifest and Latent Variables: A Comparison of LISREL and IRT Approaches

We consider a general type of model for analyzing ordinal variables with covariate effects and 2 approaches for analyzing data for such models, the item response theory (IRT) approach and the PRELIS-LISREL (PLA) approach. We compare these 2 approaches on the basis of 2 examples, 1 involving only covariate effects directly on the ordinal variables and 1 involving covariate effects on the latent variables in addition.