Model Misspecification and Invariance Testing Using Confirmatory Factor Analytic Procedures

Confirmatory factor analytic procedures are routinely implemented to provide evidence of measurement invariance. Current lines of research focus on the accuracy of common analytic steps used in confirmatory factor analysis for invariance testing. However, the few studies that have examined this procedure have done so with perfectly or near perfectly fitting models. In the present study, the authors examined procedures for detecting simulated test structure differences across groups under model misspecification conditions. In particular, they manipulated sample size, number of factors, number of indicators per factor, percentage of a lack of invariance, and model misspecification. Model misspecification was introduced at the factor loading level. They evaluated three criteria for detection of invariance, including the chi-square difference test, the difference in comparative fit index values, and the combination of the two. Results indicate that misspecification was associated with elevated Type I error rates in measurement invariance testing.     

Measurement Invariance in Confirmatory Factor Analysis: An Illustration Using IQ Test Performance of Minorities

Measurement invariance with respect to groups is an essential aspect of the fair use of scores of intelligence tests and other psychological measurements. It is widely believed that equal factor loadings are sufficient to establish measurement invariance in confirmatory factor analysis. Here, it is shown why establishing measurement invariance with confirmatory factor analysis requires a statistical test of the equality over groups of measurement intercepts. Without this essential test, measurement bias may be overlooked. A re-analysis of a study by Te Nijenhuis, Tolboom, Resing, and Bleichrodt (2004) on ethnic differences on the RAKIT IQ test illustrates that ignoring intercept differences may lead to the conclusion that bias of IQ tests with respect to minorities is small, while in reality bias is quite severe.     

Using Confirmatory Factor Analysis and the Rasch Model to Assess Measurement Invariance in a High Stakes Reading Assessment

The psychometric properties and multigroup measurement invariance of scores across subgroups, items, and persons on the Reading for Meaning items from the Georgia Criterion Referenced Competency Test (CRCT) were assessed in a sample of 778 seventh-grade students. Specifically, we sought to determine the extent to which score-based inferences on a high stakes state assessment hold across several subgroups within the population of students. To that end, both confirmatory factor analysis (CFA) and Rasch (1980 Rasch, G. 1980. Probabilistic models for some intelligence and attainment tests, Chicago: The University of Chicago Press (Original work published 1960).  [Google Scholar]) models were used to assess measurement invariance. Results revealed a unidimensional construct with factorial-level measurement invariance across disability status (students with and without specific learning disabilities), but not across test accommodations (resource guide, read-aloud, and standard administrations). Item-level analysis using the Rasch Model also revealed minimal differential item functioning across disability status, but not accommodation status.     

Using Multigroup Confirmatory Factor Analysis to Test Measurement Invariance in Raters: A Clinical Skills Examination Application

Psychometric models based on structural equation modeling framework are commonly used in many multiple-choice test settings to assess measurement invariance of test items across examinee subpopulations. The premise of the current article is that they may also be useful in the context of performance assessment tests to test measurement invariance of raters. The modeling approach and how it can be used for performance tests with less than optimal rater designs are illustrated using a data set from a performance test designed to measure medical students’ patient management skills. The results suggest that group-specific rater statistics can help spot differences in rater performance that might be due to rater bias, identify specific weaknesses and strengths of individual raters, and enhance decisions related to future task development, rater training, and test scoring processes.     

The Role of Referent Indicators in Tests of Measurement Invariance

Confirmatory factor analytic tests of measurement invariance (MI) require a referent indicator (RI) for model identification. Although the assumption that the RI is perfectly invariant across groups is acknowledged as problematic, the literature provides relatively little guidance for researchers to identify the conditions under which the practice is appropriate. Using simulated data, this study examined the effects of RI selection on both scale- and item-level MI tests. Results indicated that while inappropriate RI selection has little effect on the accuracy of conclusions drawn from scale-level tests of metric invariance, poor RI choice can produce very misleading results for item-level tests. As a result, group comparisons under conditions of partial invariance are highly susceptible to problems associated with poor RI choice.     

It Matters: Reference Indicator Selection in Measurement Invariance Tests

Conventional approaches for selecting a reference indicator (RI) could lead to misleading results in testing for measurement invariance (MI). Several newer quantitative methods have been available for more rigorous RI selection. However, it is still unknown how well these methods perform in terms of correctly identifying a truly invariant item to be an RI. Thus, Study 1 was designed to address this issue in various conditions using simulated data. As a follow-up, Study 2 further investigated the advantages/disadvantages of using RI-based approaches for MI testing in comparison with non-RI-based approaches. Altogether, the two studies provided a solid examination on how RI matters in MI tests. In addition, a large sample of real-world data was used to empirically compare the uses of the RI selection methods as well as the RI-based and non-RI-based approaches for MI testing. In the end, we offered a discussion on all these methods, followed by suggestions and recommendations for applied researchers.     

Teacher's Corner: Testing Measurement Invariance of Second-Order Factor Models

We illustrate testing measurement invariance in a second-order factor model using a quality of life dataset (n = 924). Measurement invariance was tested across 2 groups at a set of hierarchically structured levels: (a) configural invariance, (b) first-order factor loadings, (c) second-order factor loadings, (d) intercepts of measured variables, (e) intercepts of first-order factors, (f) disturbances of first-order factors, and (g) residual variances of observed variables. Given that measurement invariance at the factor loading and intercept levels was achieved, the latent factor mean difference on the higher order factor between the groups was also estimated. The analyses were performed on the mean and covariance structures within the framework of the confirmatory factor analysis using the LISREL 8.51 program. Implications of second-order factor models and measurement invariance in psychological research were discussed.     

The Effect of the Number of Observations per Parameter in Misspecified Confirmatory Factor Analytic Models

Abstract

Some authors have suggested that sample size in covariance structure modeling should be considered in the context of how many parameters are to be estimated (e.g., Kline, 2005 Kline, R. B., 2005. Principles and practice of structural equation modeling, . New York: Guilford; 2005. [Google Scholar]). Previous research has examined the effect of varying sample size relative to the number of parameters being estimated (N:q). Although some support has been found for this effect, the effect size appears to be small compared to other influences, such as indicator reliability and sample size (Jackson, 2003 Jackson, D. L., 2003. Revisiting sample size and the number of parameter estimates: Some support for the N:q hypothesis., Structural Equation Modeling: A Multidisciplinary Journal 10 (2003), pp. 128–141.[Taylor & Francis Online], [Web of Science ®] , [Google Scholar]). Efforts to extend this work to the case where models are intentionally misspecified are described in this article. In addition to varying the number of observations per estimated parameter, several other known influences on model fit were varied such as sample size, the degree of misspecification, number of variables per factor, and the communality of the measured variables. The results suggest that decreasing the number of parameters to be estimated while holding sample size constant can help detect misspecification errors, and some fit indexes were more sensitive to this manipulation than others. In general, the effects of N:q were small relative to other experimental effects.     

Testing Measurement Invariance in the Target Rotated Multigroup Exploratory Factor Model

We propose a method to investigate measurement invariance in the multigroup exploratory factor model, subject to target rotation. We consider both oblique and orthogonal target rotation. This method has clear advantages over other approaches, such as the use of congruence measures. We demonstrate that the model can be implemented readily in the freely available Mx program. We present the results of 2 illustrative analyses, one based on artificial data, and the other on real data relating to personality in male and female psychology students.     

Teichmüller映照之下解析曲线的不变性

研究拟共形映照之下曲线的变换性质 ,得到了Teichm櫣ｌｌｅｒ映照之下解析曲线的不变性质 ,推广了共形映照理论中的相应结果 .     

Measurement Invariance Testing Across Between-Level Latent Classes Using Multilevel Factor Mixture Modeling

This simulation study examines the efficacy of multilevel factor mixture modeling (ML FMM) for measurement invariance testing across unobserved groups when the groups are at the between level of multilevel data. To this end, latent classes are generated with class-specific item parameters (i.e., factor loading and intercept) across the between-level classes. The efficacy of ML FMM is evaluated in terms of class enumeration, class assignment, and the detection of noninvariance. Various classification criteria such as Akaike’s information criterion, Bayesian information criterion, and bootstrap likelihood ratio tests are examined for the correct enumeration of between-level latent classes. For the detection of measurement noninvariance, free and constrained baseline approaches are compared with respect to true positive and false positive rates. This study evidences the adequacy of ML FMM. However, its performance heavily depends on the simulation factors such as the classification criteria, sample size, and the magnitude of noninvariance. Practical guidelines for applied researchers are provided.     

一种提高功率测量精度的修正方法

针对电力参数测量中因附加相位引起的功率测量误差,提出了一种简单易行的功率算法的修正方法.通过对有功、无功的测量原理和测量系统中的硬件条件,分析了相位误差产生的因为,重点介绍了功率算法的修正原理,实验表明修正后的功率精度优于校正前.     

Cross-Level Invariance in Multilevel Factor Models

When modeling latent variables at multiple levels, it is important to consider the meaning of the latent variables at the different levels. If a higher-level common factor represents the aggregated version of a lower-level factor, the associated factor loadings will be equal across levels. However, many researchers do not consider cross-level invariance constraints in their research. Not applying these constraints when in fact they are appropriate leads to overparameterized models, and associated convergence and estimation problems. This simulation study used a two-level mediation model on common factors to show that when factor loadings are equal in the population, not applying cross-level invariance constraints leads to more estimation problems and smaller true positive rates. Some directions for future research on cross-level invariance in MLSEM are discussed.     

Testing Measurement Invariance Across Unobserved Groups: The Role of Covariates in Factor Mixture Modeling

Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed.     

Collapsing Categorical Variables and Measurement Invariance

Cross-cultural comparisons of latent variable means demands equivalent loadings and intercepts or thresholds. Although equivalence generally emphasizes items as originally designed, researchers sometimes modify response options in categorical items. For example, substantive research interests drive decisions to reduce the number of item categories. Further, categorical multiple-group confirmatory factor analysis (MG-CFA) methods generally require that the number of indicator categories is equal across groups; however, categories with few observations in at least one group can cause challenges. In the current paper, we examine the impact of collapsing ordinal response categories in MG-CFA. An empirical analysis and a complementary simulation study suggested meaningful impacts on model fit due to collapsing categories. We also found reduced scale reliability, measured as a function of Fisher’s information. Our findings further illustrated artifactual fit improvement, pointing to the possibility of data dredging for improved model-data consistency in challenging invariance contexts with large numbers of groups.     

自我监控量表的探索性和验证性因素分析

该文在指出Snyder(1974)自我监控构念存在的缺陷后,根据自我呈现的动机和自我呈现过程中是否权衡自我和谐与人际和谐,把自我监控分为三种类型:自我导向、他人导向和高自我监控。在分析每种自我监控者行为特征的基础上,编制出新的自我监控量表,探索性因子分析与验证性因子分析显示该量表具有较好的结构效度。     

A Monte Carlo Study of Recovery of Weak Factor Loadings in Confirmatory Factor Analysis

The recovery of weak factors has been extensively studied in the context of exploratory factor analysis. This article presents the results of a Monte Carlo simulation study of recovery of weak factor loadings in confirmatory factor analysis under conditions of estimation method (maximum likelihood vs. unweighted least squares), sample size, loading size, factor correlation, and model specification (correct vs. incorrect). The effects of these variables on goodness of fit and convergence are also examined. Results show that recovery of weak factor loadings, goodness of fit, and convergence are improved when factors are correlated and models are correctly specified. Additionally, unweighted least squares produces more convergent solutions and successfully recovers the weak factor loadings in some instances where maximum likelihood fails. The implications of these findings are discussed and compared to previous research.     

Assessing Measurement Invariance in Multiple-Group Latent Profile Analysis

The study of measurement invariance in latent profile analysis (LPA) indicates whether the latent profiles differ across known subgroups (e.g., gender). The purpose of the present study was to examine the impact of noninvariance on the relative bias of LPA parameter estimates and on the ability of the likelihood ratio test (LRT) and information criteria statistics to reject the hypothesis of invariance. A Monte Carlo simulation study was conducted in which noninvariance was defined as known group differences in the indicator means in each profile. Results indicated that parameter estimates were biased in conditions with medium and large noninvariance. The LRT and AIC detected noninvariance in most conditions with small sample sizes, while the BIC and adjusted BIC needed larger sample sizes to detect noninvariance. Implications of the results are discussed along with recommendations for future research.     

Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models

Ordinal variables are common in many empirical investigations in the social and behavioral sciences. Researchers often apply the maximum likelihood method to fit structural equation models to ordinal data. This assumes that the observed measures have normal distributions, which is not the case when the variables are ordinal. A better approach is to use polychoric correlations and fit the models using methods such as unweighted least squares (ULS), maximum likelihood (ML), weighted least squares (WLS), or diagonally weighted least squares (DWLS). In this simulation evaluation we study the behavior of these methods in combination with polychoric correlations when the models are misspecified. We also study the effect of model size and number of categories on the parameter estimates, their standard errors, and the common chi-square measures of fit when the models are both correct and misspecified. When used routinely, these methods give consistent parameter estimates but ULS, ML, and DWLS give incorrect standard errors. Correct standard errors can be obtained for these methods by robustification using an estimate of the asymptotic covariance matrix W of the polychoric correlations. When used in this way the methods are here called RULS, RML, and RDWLS.