Bootstrapping Confidence Intervals for Fit Indexes in Structural Equation Modeling

Bootstrapping approximate fit indexes in structural equation modeling (SEM) is of great importance because most fit indexes do not have tractable analytic distributions. Model-based bootstrap, which has been proposed to obtain the distribution of the model chi-square statistic under the null hypothesis (Bollen & Stine, 1992), is not theoretically appropriate for obtaining confidence intervals (CIs) for fit indexes because it assumes the null is exactly true. On the other hand, naive bootstrap is not expected to work well for those fit indexes that are based on the chi-square statistic, such as the root mean square error of approximation (RMSEA) and the comparative fit index (CFI), because sample noncentrality is a biased estimate of the population noncentrality. In this article we argue that a recently proposed bootstrap approach due to Yuan, Hayashi, and Yanagihara (YHY; 2007) is ideal for bootstrapping fit indexes that are based on the chi-square. This method transforms the data so that the “parent” population has the population noncentrality parameter equal to the estimated noncentrality in the original sample. We conducted a simulation study to evaluate the performance of the YHY bootstrap and the naive bootstrap for 4 indexes: RMSEA, CFI, goodness-of-fit index (GFI), and standardized root mean square residual (SRMR). We found that for RMSEA and CFI, the CIs under the YHY bootstrap had relatively good coverage rates for all conditions, whereas the CIs under the naive bootstrap had very low coverage rates when the fitted model had large degrees of freedom. However, for GFI and SRMR, the CIs under both bootstrap methods had poor coverage rates in most conditions.     

Evaluating Goodness-of-Fit Indexes for Testing Measurement Invariance

Measurement invariance is usually tested using Multigroup Confirmatory Factor Analysis, which examines the change in the goodness-of-fit index (GFI) when cross-group constraints are imposed on a measurement model. Although many studies have examined the properties of GFI as indicators of overall model fit for single-group data, there have been none to date that examine how GFIs change when between-group constraints are added to a measurement model. The lack of a consensus about what constitutes significant GFI differences places limits on measurement invariance testing. We examine 20 GFIs based on the minimum fit function. A simulation under the two-group situation was used to examine changes in the GFIs (ΔGFIs) when invariance constraints were added. Based on the results, we recommend using Δcomparative fit index, ΔGamma hat, and ΔMcDonald's Noncentrality Index to evaluate measurement invariance. These three ΔGFIs are independent of both model complexity and sample size, and are not correlated with the overall fit measures. We propose critical values of these ΔGFIs that indicate measurement invariance.     

Improving the Root Mean Square Error of Approximation for Nonnormal Conditions in Structural Equation Modeling

In this study, the authors investigated incorporating adjusted model fit information into the root mean square error of approximation (RMSEA) fit index. Through Monte Carlo simulation, the usefulness of this adjusted index was evaluated for assessing model adequacy in structural equation modeling when the multivariate normality assumption underlying maximum likelihood estimation is violated. Adjustment to the RMSEA was considered in 2 forms: a rescaling adjustment via the Satorra-Bentler rescaled goodness-of-fit statistic and a bootstrap adjustment via the Bollen and Stine adjusted model p value. Both properly specified and misspecifed models were examined. The adjusted RMSEA was evaluated in terms of the average index value across study conditions and with respect to model rejection rates under tests of exact fit, close fit, and not-close fit.     

A Comparison of Four Estimators of a Population Measure of Model Fit in Covariance Structure Analysis

A major issue in the utilization of covariance structure analysis is model fit evaluation. Recent years have witnessed increasing interest in various test statistics and so-called fit indexes, most of which are actually based on or closely related to F ₀, a measure of model fit in the population. This study aims to provide a systematic investigation about the performance of 4 available estimators of F ₀. [Fcirc]₀₁ is the conventional estimator and is based on noncentral chi-square approximation. [Fcirc]₀₂ is newly proposed and does not assume noncentral chi-square approximation. [Fcirc]₀₃ and [Fcirc]₀₄ are variations of [Fcirc]₀₂. A Monte Carlo simulation study is conducted to examine how these four estimators of F ₀ perform across varying model misspecifications, data distributions, model sizes, and sample sizes. The results show that under normality all 4 quantities estimate F ₀ equally well, and under nonnormality [Fcirc]₀₂, [Fcirc]₀₃, and [Fcirc]₀₄ outperform [Fcirc]₀₁. Issues related to these findings are discussed.     

Validity evidence for the Chinese version Classroom Appraisal of Resources and Demands (CARD)

The Classroom Appraisal of Resources and Demands (CARD) was designed to evaluate teacher stress based on subjective evaluations of classroom demands and resources. However, the CARD has been mostly utilized in western countries. The aim of the current study was to provide aspects of the validity of responses to a Chinese version of the CARD that considers Chinese teachers’ unique vocational conditions in the classroom. A sample of 580 Chinese elementary school teachers (510 female teachers and 70 male teachers) were asked to respond to the Chinese version of the CARD. Confirmatory factor analyses showed that the data fit the theoretical model very well (e.g., CFI: .982; NFI: .977; GFI: .968; SRMR: .028; RMSEA: .075; where CFI is comparative fit index, NFI is normed fit index, GFI is goodness of fit, SRMR is standardized root mean square residual, RMSEA is root mean square error of approximation), thus providing evidence of construct validity. Latent constructs of the Chinese version of the CARD were also found to be significantly associated with other measures that are related to teacher stress such as self‐efficacy, job satisfaction, personal habits to deal with stress, and intention to leave their current job.     

Measurement Bias Detection Through Factor Analysis

Assessing the correctness of a structural equation model is essential to avoid drawing incorrect conclusions from empirical research. In the past, the chi-square test was recommended for assessing the correctness of the model but this test has been criticized because of its sensitivity to sample size. As a reaction, an abundance of fit indexes have been developed. The result of these developments is that structural equation modeling packages are now producing a large list of fit measures. One would think that this progression has led to a clear understanding of evaluating models with respect to model misspecifications. In this article we question the validity of approaches for model evaluation based on overall goodness-of-fit indexes. The argument against such usage is that they do not provide an adequate indication of the “size” of the model's misspecification. That is, they vary dramatically with the values of incidental parameters that are unrelated with the misspecification in the model. This is illustrated using simple but fundamental models. As an alternative method of model evaluation, we suggest using the expected parameter change in combination with the modification index (MI) and the power of the MI test.     

Performance of the Generalized S‐X2 Item Fit Index for Polytomous IRT Models

Orlando and Thissen's S‐X ² item fit index has performed better than traditional item fit statistics such as Yen's Q₁ and McKinley and Mill's G² for dichotomous item response theory (IRT) models. This study extends the utility of S‐X ² to polytomous IRT models, including the generalized partial credit model, partial credit model, and rating scale model. The performance of the generalized S‐X ² in assessing item model fit was studied in terms of empirical Type I error rates and power and compared to G². The results suggest that the generalized S‐X ² is promising for polytomous items in educational and psychological testing programs.     

A Structural Equation Model of the Factors Affecting Morale of a Select Group of Filipino Elderly in a Community Setting

While morale among the elderly has been widely and extensively studied, results are varied and at times conflicting. Hence, the purpose of this study is to explore the factors affecting elderly morale of a select group of Filipinos in a community setting. A 64-item questionnaire was utilized to survey 323 Filipinos aged 60 and above residing in the National Capital Region of the Philippines in May 2013. Respondents completed a robotfoto, a checklist of chronic illnesses, and measures of the social support, functional ability, geriatric depression, and morale. Structural equation modeling was used to test the hypothesized model. Two competing models emerged in the study. Model 1 followed causal relationships indicated in the hypothesized model while model 2 considered modification indices that surfaced more acceptable fit indices (X²/df = 1.414, GFI [goodness of fit index] = 0.988, CFI [comparative fit index] = 0.987, RMSEA [root mean square error of approximation] = 0.036). Chronic illness, social support, and depression were found to be major predictors of morale. Number of chronic illnesses and depression were also found to have a negative relationship with functional ability, and chronic illness and social support were negatively correlated. Findings can assist health professionals such as nurses to identify the factors that shape elderly morale vis-a-vis the use of effective strategies that promote the well-being of elderly people. The emerging model can serve as reference to assess the effectiveness of quality of care rendered as manifested by morale.     

Simultaneous Use of Multiple Answer Copying Indexes to Improve Detection Rates

Many of the currently available statistical indexes to detect answer copying lack sufficient power at small α levels or when the amount of copying is relatively small. Furthermore, there is no one index that is uniformly best. Depending on the type or amount of copying, certain indexes are better than others. The purpose of this article was to explore the utility of simultaneously using multiple copying indexes to detect different types and amounts of answer copying. This study compared eight copying indexes: S₁ and S₂ (Sotaridona & Meijer, 2003), K¯ ₂ (Sotaridona & Meijer, 2002), ω (Wollack, 1997),B and H (Angoff, 1974), and new indexes Runs and MaxStrings, plus all possible pairs and triplets of the 8 indexes using multiple comparison procedures (Dunn, 1961) to adjust the critical α level for each index in a pair or triplet. Empirical Type-I error rates and power of all indexes, pairs, and triplets were examined in a real data simulation (i.e., where actual examinee responses to items [rather than generated item response vectors] were changed to match the actual responses for randomly selected source examinees) for 2 test lengths, 9 sample sizes, 3 types of copying, 4 α levels, and 4 percentages of items copied. This study found that using both ω and H* (i.e., H with empirically derived critical values) can help improve power in the most realistic types of copying situations (strings and mixed copying). The ω-H* paired index improved power most particularly for small percentages of items copied and small amounts of copying, two conditions for which copying indexes tend to be underpowered.     

The Relation Among Fit Indexes,Power, and Sample Size in Structural Equation Modeling

The relation among fit indexes, power, and sample size in structural equation modeling is examined. The noncentrality parameter is required to compute power. The 2 existing methods of computing power have estimated the noncentrality parameter by specifying an alternative hypothesis or alternative fit. These methods cannot be implemented easily and reliably. In this study, 4 fit indexes (RMSEA, CFI, McDonald's Fit Index, and Steiger's gamma) were used to compute the noncentrality parameter and sample size to achieve certain level of power. The resulting power and sample size varied as a function of (a) choice of fit index, (b) number of variables/degrees of freedom, (c) relation among the variables, and (d) value of the fit index. However, if the level of misspecification were held constant, then the resulting power and sample size would be identical.     

Using Fit Indexes to Select a Covariance Model for Longitudinal Data

This study investigated the performance of fit indexes in selecting a covariance structure for longitudinal data. Data were simulated to follow a compound symmetry, first-order autoregressive, first-order moving average, or random-coefficients covariance structure. We examined the ability of the likelihood ratio test (LRT), root mean square error of approximation (RMSEA), comparative fit index (CFI), and Tucker–Lewis Index (TLI) to reject misspecified models with varying degrees of misspecification. With a sample size of 20, RMSEA, CFI, and TLI are high in both Type I and Type II error rates, whereas LRT has a high Type II error rate. With a sample size of 100, these indexes generally have satisfactory performance, but CFI and TLI are affected by a confounding effect of their baseline model. Akaike's Information Criterion (AIC) and Bayesian Information Criterion (BIC) have high success rates in identifying the true model when sample size is 100. A comparison with the mixed model approach indicates that separately modeling the means and covariance structures in structural equation modeling dramatically improves the success rate of AIC and BIC.     

Measuring Educators' Attitudes and Beliefs About Evaluation: Construct Validity and Reliability of the Teacher Evaluation Experience Scale

This study presents the reliability and validity of the Teacher Evaluation Experience Scale–Teacher Form (TEES-T), a multidimensional measure of educators' attitudes and beliefs about teacher evaluation. Confirmatory factor analyses of data from 583 teachers were conducted on the TEES-T hypothesized five-factor model, as well as on alternative models. The five- and four-factor model yielded acceptable fit to the data. Information-theory-based indices of relative fit (i.e., AIC₀, BCC₀, and BIC₀) indicated that the TEES-T four-factor model yielded superior fit to either the five-factor or one-factor models. The TEES-T evidenced good internal consistency, freedom from item bias, and convergent validity with the Collective Efficacy Scale. Implications are discussed.     

The Accuracy of Dixon’s Formula for Obtaining Critical F-Values

The F-distribution approximation suggested by Dixon was investigated at various combinations of alpha and degrees of freedom. Tabled values were compared with values computed utilizing the suggested formula. The results indicate that while the formula is not useful in all cases, the adequacy of the approximation generally increases as alpha, v₁, and v₂ increase. It is suggested that the approximation may be utilized when certain restrictions regarding alpha, v1, and v2 are met.     

A Note on Extending Steiger's (1998) Multiple Sample RMSEA Adjustment to Other Noncentrality Parameter-Based Statistics

This article considers the implications for other noncentrality parameter-based statistics from Steiger's (1998) multiple sample adjustment to the root mean square error of approximation (RMSEA) measure. When a structural equation model is fitted simultaneously in more than 1 sample, it is shown that the calculation of the noncentrality parameter used in tests of approximate fit and in point and interval estimators of other noncentral fit statistics (except the expected cross-validation index) also requires a likeminded adjustment. Furthermore, it is shown that an adjustment is needed in multiple sample models for correctly calculating MacCallum, Browne, and Sugawara's (1996) approach to power analysis. The accuracy of these proposals is investigated and demonstrated in a small Monte Carlo study in which particular attention is paid to using appropriately constructed covariance matrices that give specified nonzero population discrepancy values under maximum likelihood estimation.     

Assessing the Sensitivity of Weighted Least Squares Model Fit Indexes to Local Dependence in Item Response Theory Models

Given the relationships of item response theory (IRT) models to confirmatory factor analysis (CFA) models, IRT model misspecifications might be detectable through model fit indexes commonly used in categorical CFA. The purpose of this study is to investigate the sensitivity of weighted least squares with adjusted means and variance (WLSMV)-based root mean square error of approximation, comparative fit index, and Tucker–Lewis Index model fit indexes to IRT models that are misspecified due to local dependence (LD). It was found that WLSMV-based fit indexes have some functional relationships to parameter estimate bias in 2-parameter logistic models caused by violations of LD. Continued exploration into these functional relationships and development of LD-detection methods based on such relationships could hold much promise for providing IRT practitioners with global information on violations of local independence.     

The Model Size Effect in SEM: Inflated Goodness-of-Fit Statistics Are Due to the Size of the Covariance Matrix

The size of a model has been shown to critically affect the goodness of approximation of the model fit statistic T to the asymptotic chi-square distribution in finite samples. It is not clear, however, whether this “model size effect” is a function of the number of manifest variables, the number of free parameters, or both. It is demonstrated by means of 2 Monte Carlo computer simulation studies that neither the number of free parameters to be estimated nor the model degrees of freedom systematically affect the T statistic when the number of manifest variables is held constant. Increasing the number of manifest variables, however, is associated with a severe bias. These results imply that model fit drastically depends on the size of the covariance matrix and that future studies involving goodness-of-fit statistics should always consider the number of manifest variables, but can safely neglect the influence of particular model specifications.     

Detecting Misspecification in Mean Structures for Growth Curve Models: Performance of Pseudo R 2s and Concordance Correlation Coefficients

This study examined the performance of 4 correlation-based fit indexes (marginal and conditional pseudo R ²s; average and conditional concordance correlations) in detecting misspecification in mean structures in growth curve models. Their performance was also compared to that of 4 traditional SEM fit indexes. We found that the marginal pseudo R ² and average concordance correlation were able to detect misspecification in the marginal mean structure (average change trajectory). The conditional pseudo R ² and concordance correlation could detect misspecification when it occurred in the conditional mean structure (individual change trajectory) or in both mean structures. Compared to the SEM fit indexes, the correlation-based fit indexes were more robust to sample size but were less robust to data properties such as magnitude of population mean and measurement error. Theoretical and practical implications of the results and directions for future research are discussed.     

Anatomy education environment measurement inventory: A valid tool to measure the anatomy learning environment

Students' perceptions of the education environment influence their learning. Ever since the major medical curriculum reform, anatomy education has undergone several changes in terms of its curriculum, teaching modalities, learning resources, and assessment methods. By measuring students' perceptions concerning anatomy education environment, valuable information can be obtained to facilitate improvements in teaching and learning. Hence, it is important to use a valid inventory that specifically measures attributes of the anatomy education environment. In this study, a new 11‐factor, 132‐items Anatomy Education Environment Measurement Inventory (AEEMI) was developed using Delphi technique and was validated in a Malaysian public medical school. The inventory was found to have satisfactory content evidence (scale‐level content validity index [total] = 0.646); good response process evidence (scale‐level face validity index [total] = 0.867); and acceptable to high internal consistency, with the Raykov composite reliability estimates of the six factors are in the range of 0.604–0.876. The best fit model of the AEEMI is achieved with six domains and 25 items (X² = 415.67, P < 0.001, ChiSq/df = 1.63, RMSEA = 0.045, GFI = 0.905, CFI = 0.937, NFI = 0.854, TLI = 0.926). Hence, AEEMI was proven to have good psychometric properties, and thus could be used to measure the anatomy education environment in Malaysia. A concerted collaboration should be initiated toward developing a valid universal tool that, using the methods outlined in this study, measures the anatomy education environment across different institutions and countries. Anat Sci Educ 10: 423–432. © 2017 American Association of Anatomists.     

Confidence Intervals of Fit Indexes by Inverting a Bootstrap Test

Fit indexes are an important tool in the evaluation of model fit in structural equation modeling (SEM). Currently, the newest confidence interval (CI) for fit indexes proposed by Zhang and Savalei (2016) is based on the quantiles of a bootstrap sampling distribution at a single level of misspecification. This method, despite a great improvement over naive and model-based bootstrap methods, still suffers from unsatisfactory coverage. In this work, we propose a new method of constructing bootstrap CIs for various fit indexes. This method directly inverts a bootstrap test and produces a CI that involves levels of misspecification that would not be rejected in a bootstrap test. Similar in rationale to a parametric CI of root mean square error of approximation (RMSEA) based on a noncentral χ2 distribution and a profile-likelihood CI of model parameters, this approach is shown to have better performance than the approach of Zhang and Savalei (2016), with more accurate coverage and more efficient widths.