Measurement Invariance Across Groups in Latent Growth Modeling

This Monte Carlo study investigated the impacts of measurement noninvariance across groups on major parameter estimates in latent growth modeling when researchers test group differences in initial status and latent growth. The average initial status and latent growth and the group effects on initial status and latent growth were investigated in terms of Type I error and bias. The location and magnitude of noninvariance across groups was related to the location and magnitude of bias and Type I error in the parameter estimates. That is, noninvariance in factor loadings and intercepts was associated with the Type I error inflation and bias in the parameter estimates of the slope factor (or latent growth) and the intercept factor (or initial status), respectively. As noninvariance became large, the degree of Type I error and bias also increased. On the other hand, a correctly specified second-order latent growth model yielded unbiased parameter estimates and correct statistical inferences. Other findings and implications on future studies were discussed.     

It Matters: Reference Indicator Selection in Measurement Invariance Tests

Conventional approaches for selecting a reference indicator (RI) could lead to misleading results in testing for measurement invariance (MI). Several newer quantitative methods have been available for more rigorous RI selection. However, it is still unknown how well these methods perform in terms of correctly identifying a truly invariant item to be an RI. Thus, Study 1 was designed to address this issue in various conditions using simulated data. As a follow-up, Study 2 further investigated the advantages/disadvantages of using RI-based approaches for MI testing in comparison with non-RI-based approaches. Altogether, the two studies provided a solid examination on how RI matters in MI tests. In addition, a large sample of real-world data was used to empirically compare the uses of the RI selection methods as well as the RI-based and non-RI-based approaches for MI testing. In the end, we offered a discussion on all these methods, followed by suggestions and recommendations for applied researchers.     

Measurement Invariance Testing Across Between-Level Latent Classes Using Multilevel Factor Mixture Modeling

This simulation study examines the efficacy of multilevel factor mixture modeling (ML FMM) for measurement invariance testing across unobserved groups when the groups are at the between level of multilevel data. To this end, latent classes are generated with class-specific item parameters (i.e., factor loading and intercept) across the between-level classes. The efficacy of ML FMM is evaluated in terms of class enumeration, class assignment, and the detection of noninvariance. Various classification criteria such as Akaike’s information criterion, Bayesian information criterion, and bootstrap likelihood ratio tests are examined for the correct enumeration of between-level latent classes. For the detection of measurement noninvariance, free and constrained baseline approaches are compared with respect to true positive and false positive rates. This study evidences the adequacy of ML FMM. However, its performance heavily depends on the simulation factors such as the classification criteria, sample size, and the magnitude of noninvariance. Practical guidelines for applied researchers are provided.     

Latent Markov Factor Analysis for Exploring Measurement Model Changes in Time-Intensive Longitudinal Studies

When time-intensive longitudinal data are used to study daily-life dynamics of psychological constructs (e.g., well-being) within persons over time (e.g., by means of experience sampling methodology), the measurement model (MM)—indicating which constructs are measured by which items—can be affected by time- or situation-specific artifacts (e.g., response styles and altered item interpretation). If not captured, these changes might lead to invalid inferences about the constructs. Existing methodology can only test for a priori hypotheses on MM changes, which are often absent or incomplete. Therefore, we present the exploratory method “latent Markov factor analysis” (LMFA), wherein a latent Markov chain captures MM changes by clustering observations per subject into a few states. Specifically, each state gathers validly comparable observations, and state-specific factor analyses reveal what the MMs look like. LMFA performs well in recovering parameters under a wide range of simulated conditions, and its empirical value is illustrated with an example.     

Testing Measurement Invariance Across Groups in Longitudinal Data: Multigroup Second-Order Latent Growth Model

In latent growth modeling, measurement invariance across groups has received little attention. Considering that a group difference is commonly of interest in social science, a Monte Carlo study explored the performance of multigroup second-order latent growth modeling (MSLGM) in testing measurement invariance. True positive and false positive rates in detecting noninvariance across groups in addition to bias estimates of major MSLGM parameters were investigated. Simulation results support the suitability of MSLGM for measurement invariance testing when either forward or iterative likelihood ratio procedure is applied.     

A Comparison of Three-Step Approaches for Auxiliary Variables in Latent Class and Latent Profile Analysis

In this article, 3-step methods to include predictors and distal outcomes in commonly used mixture models are evaluated. Two Monte Carlo simulation studies were conducted to compare the pseudo class (PC), Vermunt’s (2010), and the Lanza, Tan, and Bray (LTB) 3-step approaches with respect to bias of parameter estimates in latent class analysis (LCA) and latent profile analysis (LPA) models with auxiliary variables. For coefficients of predictors of class membership, results indicated that Vermunt’s method yielded more accurate estimates for LCA and LPA compared to the PC method. With distal outcomes of latent classes and latent profiles, the LTB method produced the lowest relative bias of coefficient estimates and Type I error rates close to nominal levels.     

Multiple-Group Factor Analysis Alignment

This article presents a new method for multiple-group confirmatory factor analysis (CFA), referred to as the alignment method. The alignment method can be used to estimate group-specific factor means and variances without requiring exact measurement invariance. A strength of the method is the ability to conveniently estimate models for many groups. The method is a valuable alternative to the currently used multiple-group CFA methods for studying measurement invariance that require multiple manual model adjustments guided by modification indexes. Multiple-group CFA is not practical with many groups due to poor model fit of the scalar model and too many large modification indexes. In contrast, the alignment method is based on the configural model and essentially automates and greatly simplifies measurement invariance analysis. The method also provides a detailed account of parameter invariance for every model parameter in every group.     

The Impact of Varying the Number of Measurement Invariance Constraints on the Assessment of Between-Group Differences of Latent Means

The objective was to offer guidelines for applied researchers on how to weigh the consequences of errors made in evaluating measurement invariance (MI) on the assessment of factor mean differences. We conducted a simulation study to supplement the MI literature by focusing on choosing among analysis models with different number of between-group constraints imposed on loadings and intercepts of indicators. Data were generated with varying proportions, patterns, and magnitudes of differences in loadings and intercepts as well as factor mean differences and sample size. Based on the findings, we concluded that researchers who conduct MI analyses should recognize that relaxing as well as imposing constraints can affect Type I error rate, power, and bias of estimates in factor mean differences. In addition, fit indexes can be misleading in making decisions about constraints of loadings and intercepts. We offer suggestions for making MI decisions under uncertainty when assessing factor mean differences.     

Modeling Unobserved Heterogeneity Using Latent Profile Analysis: A Monte Carlo Simulation

Latent profile analysis (LPA) has become a popular statistical method for modeling unobserved population heterogeneity in cross-sectionally sampled data, but very few empirical studies have examined the question of how well enumeration indexes accurately identify the correct number of latent profiles present. This Monte Carlo simulation study examined the ability of several classes of enumeration indexes to correctly identify the number of latent population profiles present under 3 different research design conditions: sample size, the number of observed variables used for LPA, and the separation distance among the latent profiles measured in Mahalanobis D units. Results showed that, for the homogeneous population (i.e., the population has k = 1 latent profile) conditions, many of the enumeration indexes used in LPA were able to correctly identify the single latent profile if variances and covariances were freely estimated. However, for a heterogeneous population (i.e., the population has k = 3 distinct latent profiles), the correct identification rate for the enumeration indexes in the k = 3 latent profile conditions was typically very low. These results are compared with the previous cross-sectional mixture modeling studies, and the limitations of this study, as well as future cross-sectional mixture modeling and enumeration index research possibilities, are discussed.     

Student Engagement and School Burnout in Finnish Lower-Secondary Schools: Latent Profile Analysis

Self-ratings of behavioural engagement, cognitive engagement and school burnout were used in person-centred analyses to identify latent profiles among 2,485 Finnish lower-secondary school students. Three profiles were identified: high-engagement/low-burnout (40.6% of the sample), average-engagement/average-burnout (53.9%), and low-engagement/high-burnout (5.5%). Another sample of lower-secondary school students was used to validate the 3 profiles. The factors most strongly associated with the high-engagement/low-burnout profile of lower-secondary school students’ were high levels of support from teachers and family, good academic performance, and lack of truancy. The study indicated that teacher and family support and students’ academic achievement are pivotal in understanding student engagement and school burnout.     

青少年抑郁情绪与人际信任的关系:基于潜在剖面分析?

基于中国家庭追踪调查(CFPS)2016年数据,采用潜在剖面分析方法考察2769名青少年抑郁情绪的异质性及其人口学特征,分析不同抑郁情绪类别青少年的人际信任差异。结果发现:青少年可分为高抑郁情绪、中抑郁情绪与低抑郁情绪3种潜在类别,占比分别为6.68%、22.03%、71.29%;乡村,小学、初中、不在学,学业成绩在班级后25%等特征的青少年在中抑郁情绪型与高抑郁情绪型中所占比例更高;青少年人际信任表现为熟人信任与陌生人信任的差序结构,3类抑郁情绪类别青少年的熟人信任均处于中上水平,且低抑郁情绪型的信任水平最高、中抑郁情绪型次之、高抑郁情绪型最低,陌生人信任在3类青少年中不存在差异,均为较低水平。     

Statistical Power to Detect the Correct Number of Classes in Latent Profile Analysis

Little research has examined factors influencing statistical power to detect the correct number of latent classes using latent profile analysis (LPA). This simulation study examined power related to interclass distance between latent classes given true number of classes, sample size, and number of indicators. Seven model selection methods were evaluated. None had adequate power to select the correct number of classes with a small (Cohen's d = .2) or medium (d = .5) degree of separation. With a very large degree of separation (d = 1.5), the Lo–Mendell–Rubin test (LMR), adjusted LMR, bootstrap likelihood ratio test, Bayesian Information Criterion (BIC), and sample-size-adjusted BIC were good at selecting the correct number of classes. However, with a large degree of separation (d = .8), power depended on number of indicators and sample size. Akaike's Information Criterion and entropy poorly selected the correct number of classes, regardless of degree of separation, number of indicators, or sample size.     

Measurement Bias Detection Through Factor Analysis

Assessing the correctness of a structural equation model is essential to avoid drawing incorrect conclusions from empirical research. In the past, the chi-square test was recommended for assessing the correctness of the model but this test has been criticized because of its sensitivity to sample size. As a reaction, an abundance of fit indexes have been developed. The result of these developments is that structural equation modeling packages are now producing a large list of fit measures. One would think that this progression has led to a clear understanding of evaluating models with respect to model misspecifications. In this article we question the validity of approaches for model evaluation based on overall goodness-of-fit indexes. The argument against such usage is that they do not provide an adequate indication of the “size” of the model's misspecification. That is, they vary dramatically with the values of incidental parameters that are unrelated with the misspecification in the model. This is illustrated using simple but fundamental models. As an alternative method of model evaluation, we suggest using the expected parameter change in combination with the modification index (MI) and the power of the MI test.     

Assessing statistical aspects of test fairness with structural equation modelling

Test fairness and test bias are not synonymous concepts. Test bias refers to statistical evidence that the psychometrics or interpretation of test scores depend on group membership, such as gender or race, when such differences are not expected. A test that is grossly biased may be judged to be unfair, but test fairness concerns the broader, more subjective evaluation of assessment outcomes from perspectives of social justice. Thus, the determination of test fairness is not solely a matter of statistics, but statistical evidence is important when evaluating test fairness. This work introduces the use of the structural equation modelling technique of multiple-group confirmatory factor analysis (MGCFA) to evaluate hypotheses of measurement invariance, or whether a set of observed variables measures the same factors with the same precision over different populations. An example of testing for measurement invariance with MGCFA in an actual, downloadable data set is also demonstrated.     

中国4省市阅读教学模式探究:基于PISA2018测评的潜在剖面分析

教师的课堂教学行为对学生学习有着重要作用。基于课堂教学的三维理论模型,采用中国4省市PISA2018测试数据,使用潜在剖面分析,探索中国4省市阅读教学的典型模式,并就不同模式对学生学习的影响开展研究。结果发现:1)学生感知到的教师教学模式主要有综合发展型、普通支持型和纪律导向型;2)感知到综合发展型教学模式的学生在学科知识与理解、动机、情感与注意力和学习时间维度上均表现最佳,纪律导向型教学模式的学生在各指标上表现较弱;3)不同家庭社会经济地位和不同学校类型的学生对不同教学模式的感知存在显著差异,家庭社会经济地位中等及以下的学生、城镇和农村学校的学生更容易感知到纪律导向型教学模式,更难感知到综合发展型教学模式,男女生对不同教学模式的感知不存在显著差异。在此基础上,提出要为教师采用综合发展型教学模式创造条件、教师要强化元认知教学并关注课堂教学的群体差异与区域差异等建议。     

Collapsing Categorical Variables and Measurement Invariance

Cross-cultural comparisons of latent variable means demands equivalent loadings and intercepts or thresholds. Although equivalence generally emphasizes items as originally designed, researchers sometimes modify response options in categorical items. For example, substantive research interests drive decisions to reduce the number of item categories. Further, categorical multiple-group confirmatory factor analysis (MG-CFA) methods generally require that the number of indicator categories is equal across groups; however, categories with few observations in at least one group can cause challenges. In the current paper, we examine the impact of collapsing ordinal response categories in MG-CFA. An empirical analysis and a complementary simulation study suggested meaningful impacts on model fit due to collapsing categories. We also found reduced scale reliability, measured as a function of Fisher’s information. Our findings further illustrated artifactual fit improvement, pointing to the possibility of data dredging for improved model-data consistency in challenging invariance contexts with large numbers of groups.     

Measurement Invariance of Socioeconomic Status Across Migrational Background

Socioeconomic status (SES) is often used as control variable when relations between academic outcomes and students' migrational background are investigated. When measuring SES, indicators used must have the same meaning across groups. This study aims to examine the measurement invariance of SES, using data from TIMSS, 2003. The study shows that a latent SES variable has the same meaning across sub-populations with Swedish and non-Swedish background. However, the assumption of scalar invariance was rejected, which is essential for estimation of differences in latent means between groups. Comparisons between models assuming different degrees of scalar invariance indicated that models allowing partial scalar invariance should not be used when comparing latent variable means across groups of students with different migrational backgrounds.     

A Two-Step Latent Profile Method for Identifying Invalid Respondents in Self-Reported Survey Data

Insincere respondents can have an adverse impact on the validity of substantive inferences arising from self-administered questionnaires (SAQs). The current study introduces a new method for identifying potentially invalid respondents from their atypical response patterns. The two-step procedure involves generating a response inconsistency (RI) score for each participant and scale on the SAQ and subjecting the resulting scores to latent profile analysis to identify classes of atypical RI respondent profiles. The procedure can be implemented post–data collection and is illustrated through a survey of school climate that was administered to N = 52,102 high school students. Results of this screening procedure revealed high levels of specificity and expected levels of concordance when contrasted with the results of traditionally used methods of screening items and response time. Contrasts between valid and invalid respondents revealed similar patterns across the three screening procedures when compared across external measures of academics and risk behaviors.     

Two-Step Approach to Partial Factorial Invariance: Selecting a Reference Variable and Identifying the Source of Noninvariance

To date, no effective empirical method has been available to identify a truly invariant reference variable (RV) in testing measurement invariance under a multiple-group confirmatory factor analysis. This study proposes a method that, in selecting an RV, uses the smallest modification index (min-mod). The method’s performance is evaluated using 2 models: (a) a full invariance model, and (b) a partial invariance model. Results indicate that for both models the min-mod successfully identifies a truly invariant RV (Study 1). In Study 2, we use the RV found in Study 1 to further evaluate the performance of item-by-item Wald tests at locating a noninvariant variable. The results indicate that Wald tests overall performed better with an RV selected in a partial invariance model than an RV selected in a full invariance model, although in certain conditions their performances were rather similar. Implications and limitations of the study are also discussed.     

An Investigation of the Alignment Method With Polytomous Indicators Under Conditions of Partial Measurement Invariance

The alignment method (Asparouhov & Muthén, 2014) is an alternative to multiple-group factor analysis for estimating measurement models and testing for measurement invariance across groups. Simulation studies evaluating the performance of the alignment for estimating measurement models across groups show promising results for continuous indicators. This simulation study builds on previous research by investigating the performance of the alignment method’s measurement models estimates with polytomous indicators under conditions of systematically increasing, partial measurement invariance. We also present an evaluation of the testing procedure, which has not been the focus of previous simulation studies. Results indicate that the alignment adequately recovers parameter estimates under small and moderate amounts of noninvariance, with issues only arising in extreme conditions. In addition, the statistical tests of invariance were fairly conservative, and had less power for items with more extreme skew. We include recommendations for using the alignment method based on these results.