Collapsing Categorical Variables and Measurement Invariance

Cross-cultural comparisons of latent variable means demands equivalent loadings and intercepts or thresholds. Although equivalence generally emphasizes items as originally designed, researchers sometimes modify response options in categorical items. For example, substantive research interests drive decisions to reduce the number of item categories. Further, categorical multiple-group confirmatory factor analysis (MG-CFA) methods generally require that the number of indicator categories is equal across groups; however, categories with few observations in at least one group can cause challenges. In the current paper, we examine the impact of collapsing ordinal response categories in MG-CFA. An empirical analysis and a complementary simulation study suggested meaningful impacts on model fit due to collapsing categories. We also found reduced scale reliability, measured as a function of Fisher’s information. Our findings further illustrated artifactual fit improvement, pointing to the possibility of data dredging for improved model-data consistency in challenging invariance contexts with large numbers of groups.     

Model Misspecification and Invariance Testing Using Confirmatory Factor Analytic Procedures

Confirmatory factor analytic procedures are routinely implemented to provide evidence of measurement invariance. Current lines of research focus on the accuracy of common analytic steps used in confirmatory factor analysis for invariance testing. However, the few studies that have examined this procedure have done so with perfectly or near perfectly fitting models. In the present study, the authors examined procedures for detecting simulated test structure differences across groups under model misspecification conditions. In particular, they manipulated sample size, number of factors, number of indicators per factor, percentage of a lack of invariance, and model misspecification. Model misspecification was introduced at the factor loading level. They evaluated three criteria for detection of invariance, including the chi-square difference test, the difference in comparative fit index values, and the combination of the two. Results indicate that misspecification was associated with elevated Type I error rates in measurement invariance testing.     

Evaluating Model Fit With Ordered Categorical Data Within a Measurement Invariance Framework: A Comparison of Estimators

A paucity of research has compared estimation methods within a measurement invariance (MI) framework and determined if research conclusions using normal-theory maximum likelihood (ML) generalizes to the robust ML (MLR) and weighted least squares means and variance adjusted (WLSMV) estimators. Using ordered categorical data, this simulation study aimed to address these queries by investigating 342 conditions. When testing for metric and scalar invariance, Δχ² results revealed that Type I error rates varied across estimators (ML, MLR, and WLSMV) with symmetric and asymmetric data. The Δχ² power varied substantially based on the estimator selected, type of noninvariant indicator, number of noninvariant indicators, and sample size. Although some the changes in approximate fit indexes (ΔAFI) are relatively sample size independent, researchers who use the ΔAFI with WLSMV should use caution, as these statistics do not perform well with misspecified models. As a supplemental analysis, our results evaluate and suggest cutoff values based on previous research.     

Measurement Invariance Across Groups in Latent Growth Modeling

This Monte Carlo study investigated the impacts of measurement noninvariance across groups on major parameter estimates in latent growth modeling when researchers test group differences in initial status and latent growth. The average initial status and latent growth and the group effects on initial status and latent growth were investigated in terms of Type I error and bias. The location and magnitude of noninvariance across groups was related to the location and magnitude of bias and Type I error in the parameter estimates. That is, noninvariance in factor loadings and intercepts was associated with the Type I error inflation and bias in the parameter estimates of the slope factor (or latent growth) and the intercept factor (or initial status), respectively. As noninvariance became large, the degree of Type I error and bias also increased. On the other hand, a correctly specified second-order latent growth model yielded unbiased parameter estimates and correct statistical inferences. Other findings and implications on future studies were discussed.     

It Matters: Reference Indicator Selection in Measurement Invariance Tests

Conventional approaches for selecting a reference indicator (RI) could lead to misleading results in testing for measurement invariance (MI). Several newer quantitative methods have been available for more rigorous RI selection. However, it is still unknown how well these methods perform in terms of correctly identifying a truly invariant item to be an RI. Thus, Study 1 was designed to address this issue in various conditions using simulated data. As a follow-up, Study 2 further investigated the advantages/disadvantages of using RI-based approaches for MI testing in comparison with non-RI-based approaches. Altogether, the two studies provided a solid examination on how RI matters in MI tests. In addition, a large sample of real-world data was used to empirically compare the uses of the RI selection methods as well as the RI-based and non-RI-based approaches for MI testing. In the end, we offered a discussion on all these methods, followed by suggestions and recommendations for applied researchers.     

Assessing Measurement Invariance in Multiple-Group Latent Profile Analysis

The study of measurement invariance in latent profile analysis (LPA) indicates whether the latent profiles differ across known subgroups (e.g., gender). The purpose of the present study was to examine the impact of noninvariance on the relative bias of LPA parameter estimates and on the ability of the likelihood ratio test (LRT) and information criteria statistics to reject the hypothesis of invariance. A Monte Carlo simulation study was conducted in which noninvariance was defined as known group differences in the indicator means in each profile. Results indicated that parameter estimates were biased in conditions with medium and large noninvariance. The LRT and AIC detected noninvariance in most conditions with small sample sizes, while the BIC and adjusted BIC needed larger sample sizes to detect noninvariance. Implications of the results are discussed along with recommendations for future research.     

Measurement Invariance Testing Across Between-Level Latent Classes Using Multilevel Factor Mixture Modeling

This simulation study examines the efficacy of multilevel factor mixture modeling (ML FMM) for measurement invariance testing across unobserved groups when the groups are at the between level of multilevel data. To this end, latent classes are generated with class-specific item parameters (i.e., factor loading and intercept) across the between-level classes. The efficacy of ML FMM is evaluated in terms of class enumeration, class assignment, and the detection of noninvariance. Various classification criteria such as Akaike’s information criterion, Bayesian information criterion, and bootstrap likelihood ratio tests are examined for the correct enumeration of between-level latent classes. For the detection of measurement noninvariance, free and constrained baseline approaches are compared with respect to true positive and false positive rates. This study evidences the adequacy of ML FMM. However, its performance heavily depends on the simulation factors such as the classification criteria, sample size, and the magnitude of noninvariance. Practical guidelines for applied researchers are provided.     

Testing Measurement Invariance Across Groups in Longitudinal Data: Multigroup Second-Order Latent Growth Model

In latent growth modeling, measurement invariance across groups has received little attention. Considering that a group difference is commonly of interest in social science, a Monte Carlo study explored the performance of multigroup second-order latent growth modeling (MSLGM) in testing measurement invariance. True positive and false positive rates in detecting noninvariance across groups in addition to bias estimates of major MSLGM parameters were investigated. Simulation results support the suitability of MSLGM for measurement invariance testing when either forward or iterative likelihood ratio procedure is applied.     

Investigating the Multidimensionality of Need Fulfillment: A Bifactor Exploratory Structural Equation Modeling Representation

This article assesses the multidimensionality of the Basic Psychological Need Satisfaction and Frustration Scale (BPNSFS) using bifactor exploratory structural equation modeling (bifactor ESEM). The first study relies on a sample of community adults (N = 2,301), and revealed the superiority of a bifactor ESEM representation, supporting the 6-factor structure of BPNSFS ratings, and the presence of a single continuum of need fulfillment relative to 2 distinct dimensions reflecting need satisfaction and frustration. These results were replicated in a second representative sample of the Hungarian adult population (N = 504), as well as across gender, and found no evidence of differential item functioning as a function of age. Relative to males, females presented higher levels of relatedness satisfaction and lower levels of competence satisfaction. Finally, autonomy frustration decreased with age, whereas competence satisfaction and frustration presented opposite curvilinear tendencies, showing that the fulfillment of this need increased sharply for younger participants, a tendency that became less pronounced with age.     

Measurement Invariance of Socioeconomic Status Across Migrational Background

Socioeconomic status (SES) is often used as control variable when relations between academic outcomes and students' migrational background are investigated. When measuring SES, indicators used must have the same meaning across groups. This study aims to examine the measurement invariance of SES, using data from TIMSS, 2003. The study shows that a latent SES variable has the same meaning across sub-populations with Swedish and non-Swedish background. However, the assumption of scalar invariance was rejected, which is essential for estimation of differences in latent means between groups. Comparisons between models assuming different degrees of scalar invariance indicated that models allowing partial scalar invariance should not be used when comparing latent variable means across groups of students with different migrational backgrounds.     

教育技术学实验研究工具的跨时测量恒等检测

基于跨时测量恒等视角与知识图谱分析,文章对我国教育技术学较常探讨的变量"自我效能"量表进行了工具检测,并以四川省某小学三年级的197名学生为被试,前后测时间间隔为6个月。文章采用结构方程模型的跨时测量恒等检验程序,依序针对不同恒等程度的模型进行比较,结果发现:数学自我效能量表不符合完全的度量恒等,放宽两道题项的参数限制后可达到部分的纯量恒等,但仍不及严格恒等的要求;跨时测量恒等性的结果会影响配对样本t检验的结论。基于此,文章提出建议:为了提升实验的内在效度,较长时间的实验研究应纳入工具的跨时测量恒等性检验。     

Testing Measurement Invariance Across Unobserved Groups: The Role of Covariates in Factor Mixture Modeling

Factor mixture modeling (FMM) has been increasingly used to investigate unobserved population heterogeneity. This study examined the issue of covariate effects with FMM in the context of measurement invariance testing. Specifically, the impact of excluding and misspecifying covariate effects on measurement invariance testing and class enumeration was investigated via Monte Carlo simulations. Data were generated based on FMM models with (1) a zero covariate effect, (2) a covariate effect on the latent class variable, and (3) covariate effects on both the latent class variable and the factor. For each population model, different analysis models that excluded or misspecified covariate effects were fitted. Results highlighted the importance of including proper covariates in measurement invariance testing and evidenced the utility of a model comparison approach in searching for the correct specification of covariate effects and the level of measurement invariance. This approach was demonstrated using an empirical data set. Implications for methodological and applied research are discussed.     

Rasch模型项目参数不变性的验证研究

Rasch模型具有被试参数和项目参数相互独立的性质,即被试能力与项目难度无关。本研究以某年度大学入学考试数学学科的实测成绩数据为例,在随机抽样、不同性别抽样、不同水平群体抽样等条件下,对Rasch模型项目参数不变性进行了验证研究。研究表明:Rasch模型项目参数不变性验证的前提条件较为严格,需要排除诸多干扰因素的影响;Rasch模型项目参数不变性的验证存在一定的误差,无法实现与理论一致的"不变性";Rasch模型项目参数不变性没有统一的标准,需依据实际问题确定。     

An Investigation of the Alignment Method With Polytomous Indicators Under Conditions of Partial Measurement Invariance

The alignment method (Asparouhov & Muthén, 2014) is an alternative to multiple-group factor analysis for estimating measurement models and testing for measurement invariance across groups. Simulation studies evaluating the performance of the alignment for estimating measurement models across groups show promising results for continuous indicators. This simulation study builds on previous research by investigating the performance of the alignment method’s measurement models estimates with polytomous indicators under conditions of systematically increasing, partial measurement invariance. We also present an evaluation of the testing procedure, which has not been the focus of previous simulation studies. Results indicate that the alignment adequately recovers parameter estimates under small and moderate amounts of noninvariance, with issues only arising in extreme conditions. In addition, the statistical tests of invariance were fairly conservative, and had less power for items with more extreme skew. We include recommendations for using the alignment method based on these results.     

Measurement Invariance Testing with Many Groups: A Comparison of Five Approaches

With the increasing use of international survey data especially in cross-cultural and multinational studies, establishing measurement invariance (MI) across a large number of groups in a study is essential. Testing MI over many groups is methodologically challenging, however. We identified 5 methods for MI testing across many groups (multiple group confirmatory factor analysis, multilevel confirmatory factor analysis, multilevel factor mixture modeling, Bayesian approximate MI testing, and alignment optimization) and explicated the similarities and differences of these approaches in terms of their conceptual models and statistical procedures. A Monte Carlo study was conducted to investigate the efficacy of the 5 methods in detecting measurement noninvariance across many groups using various fit criteria. Generally, the 5 methods showed reasonable performance in identifying the level of invariance if an appropriate fit criterion was used (e.g., Bayesian information criteron with multilevel factor mixture modeling). Finally, general guidelines in selecting an appropriate method are provided.     

The Impact of Varying the Number of Measurement Invariance Constraints on the Assessment of Between-Group Differences of Latent Means

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.     

Multilevel Factorial Invariance in n-Level Structural Equation Modeling (nSEM)

We present a multigroup multilevel confirmatory factor analysis (CFA) model and a procedure for testing multilevel factorial invariance in n-level structural equation modeling (nSEM). Multigroup multilevel CFA introduces a complexity when the group membership at the lower level intersects the clustered structure, because the observations in different groups but in the same cluster are not independent of one another. nSEM provides a framework in which the multigroup multilevel data structure is represented with the dependency between groups at the lower level properly taken into account. The procedure for testing multilevel factorial invariance is illustrated with an empirical example using an R package xxm2.     

考生分数误差指数的应用探讨

考分中难免会包含测量误差,论文旨在介绍几种量化个体考分中测量误差大小的指数,并举例说明这些指数在探测考生异常作答方面的应用。论文重点在于让读者对这些指数的运用原理有初步了解,熟悉其应用方向及在不同条件下的性能。在论文所介绍的几种指数中,扩展型警告指数在探测所讨论到的几种异常作答类型时比考生标准测量误差指数的灵敏度高;但这些指数都存在着较高的误通报率,即将本不属于异常类型的情况归属成异常类型,因此在实际应用中,研究者需要对被通报成异常类型的数据进一步分析,也即这些指数最终起到的是缩小研究者关注范围的作用。     

The Model Size Effect in SEM: Inflated Goodness-of-Fit Statistics Are Due to the Size of the Covariance Matrix

The size of a model has been shown to critically affect the goodness of approximation of the model fit statistic T to the asymptotic chi-square distribution in finite samples. It is not clear, however, whether this “model size effect” is a function of the number of manifest variables, the number of free parameters, or both. It is demonstrated by means of 2 Monte Carlo computer simulation studies that neither the number of free parameters to be estimated nor the model degrees of freedom systematically affect the T statistic when the number of manifest variables is held constant. Increasing the number of manifest variables, however, is associated with a severe bias. These results imply that model fit drastically depends on the size of the covariance matrix and that future studies involving goodness-of-fit statistics should always consider the number of manifest variables, but can safely neglect the influence of particular model specifications.