Factor Retention in Exploratory Factor Analysis With Missing Data

Determining the number of factors in exploratory factor analysis is arguably the most crucial decision a researcher faces when conducting the analysis. While several simulation studies exist that compare various so-called factor retention criteria under different data conditions, little is known about the impact of missing data on this process. Hence, in this study, we evaluated the performance of different factor retention criteria—the Factor Forest, parallel analysis based on a principal component analysis as well as parallel analysis based on the common factor model and the comparison data approach—in combination with different missing data methods, namely an expectation-maximization algorithm called Amelia, predictive mean matching, and random forest imputation within the multiple imputations by chained equations (MICE) framework as well as pairwise deletion with regard to their accuracy in determining the number of factors when data are missing. Data were simulated for different sample sizes, numbers of factors, numbers of manifest variables (indicators), between-factor correlations, missing data mechanisms and proportions of missing values. In the majority of conditions and for all factor retention criteria except the comparison data approach, the missing data mechanism had little impact on the accuracy and pairwise deletion performed comparably well as the more sophisticated imputation methods. In some conditions, especially small-sample cases and when comparison data were used to determine the number of factors, random forest imputation was preferable to other missing data methods, though. Accordingly, depending on data characteristics and the selected factor retention criterion, choosing an appropriate missing data method is crucial to obtain a valid estimate of the number of factors to extract.     

Performance of Modified Test Statistics in Covariance and Correlation Structure Analysis Under Conditions of Multivariate Nonnormality

Questions of whether hypothesized structure models are appropriate representations of the pattern of association among a group of variables can be addressed using a wide variety of statistical procedures. These procedures include covariance structure analysis techniques and correlation structure analysis techniques, in which covariance structure procedures are based on distribution theory for covariances, and correlation structure procedures are based on distribution theory for correlations. The present article provides an overview of standard and modified normal theory and asymptotically distribution-free covariance and correlation structure analysis techniques and also details Monte Carlo simulation results on the Type I and Type II error control as a function of structure model type, number of variables in the model, sample size, and distributional nonnormality. The present Monte Carlo simulation demonstrates clearly that the robustness and nonrobustness of structure analysis techniques vary as a function of the structure of the model and the data conditions. Implications of these results for users of structure analysis techniques are considered in the context of current software availability.     

Smoking and Cancers: Case-Robust Analysis of a Classic Data Set

A typical structural equation model is intended to reproduce the means, variances, and correlations or covariances among a set of variables based on parameter estimates of a highly restricted model. It is not widely appreciated that the sample statistics being modeled can be quite sensitive to outliers and influential observations, leading to bias in model parameter estimates. A classic public epidemiological data set on the relation between cigarette purchases and rates of 4 types of cancer among states in the United States is studied with case-weighting methods that reduce the influence of a few cases on the overall results. The results support and extend the original conclusions; the standardized effect of smoking on a factor underlying deaths from bladder and lung cancer is .79.     

State-Space Modeling of Dynamic Psychological Processes via the Kalman Smoother Algorithm: Rationale,Finite Sample Properties,and Applications

This article presents a state-space modeling (SSM) technique for fitting process factor analysis models directly to raw data. The Kalman smoother via the expectation-maximization algorithm to obtain maximum likelihood parameter estimates is used. To examine the finite sample properties of the estimates in SSM when common factors are involved, a Monte Carlo study is conducted. Results indicate that the estimates of factor loading matrix, transition matrix, and unique variances were asymptotically normal, accurate, precise, and robust, especially for moderate and long time series. The estimates of state residual variances were positively biased for shorter time series, but as the length of series increased, these estimates became accurate and precise. To illustrate the application of SSM the technique is applied to empirical multivariate time-series data on daily affect collected from 2 individuals in a dating couple.     

Standard Errors of IRT Parameter Scale Transformation Coefficients: Comparison of Bootstrap Method,Delta Method,and Multiple Imputation Method

The present study evaluated the multiple imputation method, a procedure that is similar to the one suggested by Li and Lissitz (2004), and compared the performance of this method with that of the bootstrap method and the delta method in obtaining the standard errors for the estimates of the parameter scale transformation coefficients in item response theory (IRT) equating in the context of the common‐item nonequivalent groups design. Two different estimation procedures for the variance‐covariance matrix of the IRT item parameter estimates, which were used in both the delta method and the multiple imputation method, were considered: empirical cross‐product (XPD) and supplemented expectation maximization (SEM). The results of the analyses with simulated and real data indicate that the multiple imputation method generally produced very similar results to the bootstrap method and the delta method in most of the conditions. The differences between the estimated standard errors obtained by the methods using the XPD matrices and the SEM matrices were very small when the sample size was reasonably large. When the sample size was small, the methods using the XPD matrices appeared to yield slight upward bias for the standard errors of the IRT parameter scale transformation coefficients.     

Effects of Missing Data Methods in SEM Under Conditions of Incomplete and Nonnormal Data

Using Monte Carlo simulations, this research examined the performance of four missing data methods in SEM under different multivariate distributional conditions. The effects of four independent variables (sample size, missing proportion, distribution shape, and factor loading magnitude) were investigated on six outcome variables: convergence rate, parameter estimate bias, MSE of parameter estimates, standard error coverage, model rejection rate, and model goodness of fit—RMSEA. A three-factor CFA model was used. Findings indicated that FIML outperformed the other methods in MCAR, and MI should be used to increase the plausibility of MAR. SRPI was not comparable to the other three methods in either MCAR or MAR.     

The Incorporation of Sample Weights Into Multilevel Structural Equation Models

Over the past decade and a half, methodologists working with structural equation modeling (SEM) have developed approaches for accommodating multilevel data. These approaches are particularly helpful when modeling data that come from complex sampling designs. However, most data sets that are associated with complex sampling designs also include observation weights, and methods to incorporate these sampling weights into multilevel SEM analyses have not been addressed. This article investigates the use of different weighting techniques and finds, through a simulation study, that the use of an effective sample size weight provides unbiased estimates of key parameters and their sampling variances. Also, a popular normalization technique of scaling weights to reflect the actual sample size is shown to produce negatively biased sampling variance estimates, as well as negatively biased within-group variance parameter estimates in the small group size case.     

Formulation of the DETECT Population Parameter and Evaluation of DETECT Estimator Bias

The development of the DETECT procedure marked an important advancement in nonparametric dimensionality analysis. DETECT is the first nonparametric technique to estimate the number of dimensions in a data set, estimate an effect size for multidimensionality, and identify which dimension is predominantly measured by each item. The efficacy of DETECT critically depends on accurate, minimally biased estimation of the expected conditional covariances of all the item pairs. However, the amount of bias in the DETECT estimator has been studied only in a few simulated unidimensional data sets. This is because the value of the DETECT population parameter is known to be zero for this case and has been unknown for cases when multidimensionality is present. In this article, integral formulas for the DETECT population parameter are derived for the most commonly used parametric multidimensional item response theory model, the Reckase and McKinley model. These formulas are then used to evaluate the bias in DETECT by positing a multidimensional model, simulating data from the model using a very large sample size (to eliminate random error), calculating the large-sample DETECT statistic, and finally calculating the DETECT population parameter to compare with the large-sample statistic. A wide variety of two- and three-dimensional models, including both simple structure and approximate simple structure, were investigated. The results indicated that DETECT does exhibit statistical bias in the large-sample estimation of the item-pair conditional covariances; but, for the simulated tests that had 20 or more items, the bias was small enough to result in the large-sample DETECT almost always correctly partitioning the items and the DETECT effect size estimator exhibiting negligible bias.     

Score Comparability of a State Reading Assessment Across Selected Groups of Students With Disabilities

This study investigated the factorial invariance of scores from a 7th-grade state reading assessment across general education students and selected groups of students with disabilities. Confirmatory factor analysis was used to assess the fit of a 2-factor model to each of the 4 groups. In addition to overall fit of this model, 5 levels of constraint, including equal factor loadings, intercepts, error variances, factor variances, and factor covariances, were investigated. Invariance across the factor loadings and intercepts was supported across the groups of students with disabilities and general education students. Invariance for these groups was not supported for the error variances. For the students with mental retardation, the lack of fit of the 2-factor model and the observed score results suggested a mismatch between the difficulty level of this test and the ability level of these students. Although the results generally supported the score comparability of the reading assessment across these groups, further research is needed into the nature of the larger error variances for the student with disabilities groups and into accommodations and modifications for the students with mental retardation.     

An evaluation of the Satorra‐Bentler distributional misspecification correction applied to the McDonald fit index

McDonald goodness‐of‐fit indices based on maximum likelihood, asymptotic distribution free, and the Satorra‐Bentler scale correction estimation methods are investigated. Sampling experiments are conducted to assess the magnitude of error for each index under variations in distributional misspecification, structural misspecification, and sample size. The Satorra‐Bentler correction‐based index is shown to have the least error under each distributional misspecification level when the model has correct structural specification. The scaled index also performs adequately when there is minor structural misspecification and distributional misspecification. However, when a model has major structural misspecification with distributional misspecification, none of the estimation methods perform adequately.     

A Comparison of Bridging Methods in the Analysis of NAEP Trends with the New Race Subgroup Definitions

Recent changes to federal guidelines for the collection of data on race and ethnicity allow respondents to select multiple race categories. Redefining race subgroups in this manner poses problems for research spanning both sets of definitions. NAEP long-term trends have used the single-race subgroup definitions for over thirty years. Little is known about the effects of redefining race subgroups on these trends. Bridging methods for reconciling the single and multiple race definitions have been developed. These methods treat single-race subgroup membership as unknown or missing. A simulation study was conducted to determine the effectiveness of four bridging methods: multiple imputation logistic regression, multiple imputation probabilistic whole assignment, deterministic whole assignment—smallest group, and deterministic whole assignment—largest group. Only the first of these methods incorporates covariate information about examinees into the bridging procedure. The other three methods only use information contained in the race item response. The simulation took into account the percentage of biracial examinees and the missing data mechanism. Results indicated that the multiple imputation logistic regression was often the best performing method. Given that all K-12 and higher education institutions will be required to use the multiple-race definitions by 2009, implications for No Child Left Behind and other federally mandated reporting are discussed.     

MIXED LINEAR MODEL APPROACHES FOR ANALYZING GENETIC MODELS OF COMPLEX QUANTITATIVE TRAITS

ＩＮＴＲＯＤＵＣＴＩＯＮＭａｎｙｇｅｎｅｔｉｃｍｏｄｅｌｓｂａｓｅｄｏｎｔｈｅａｐｐｒｏａｃｈｏｆＡＮＯＶＡ (ａｎａｌｙｓｉｓｏｆｖａｒｉａｎｃｅ)ｗｅｒｅｄｅｖｅｌ ｏｐｅｄｂｙＦｉｓｈｅｒ(1 92 5) .Ｓｏｍｅｏｆｔｈｅｓｅｍｏｄｅｌｓ,ｅ.ｇ .ＮＣｄｅｓｉｇｎＩａｎｄＩＩ(Ｃｏｍｓｔｏｃｋｅｔａｌ.,1 952 ;Ｈａｌｌａｕｅｒｅｔａｌ.,1 981 ) ,ｄｉａｌｌｅｌｍｏｄｅｌｓ(Ｙａｔｅｓ,1 94 7;Ｇｒｉｆｆｉｎｇ,1 956;ＧａｒｄｎｅｒａｎｄＥ ｂｅｒｈａｒｔ,1 966) ,ａｒｅｓｔｉｌｌｗｉｄｅｌｙｕｓｅｄｂｙｐｌａ…     

分布式协同应用中的Agent技术

Agent技术在基于网络的分布计算领域中,正发挥着越来越重要的作用．一方面,Agent技术为解决新的分布式应用问题提供了有效途径;另一方面,Agent技术为全面准确地研究分布计算系统的特点提供了合理的概念模型,本文讨论了Agent技术的概念、应用及其体系结构．     

Testing for factorial invariance of the Modified Leadership Scale for Sports: using a Japanese version

The objective of this study was to provide empirical evidence to support psychometric properties of a modified four-dimensional model of the Leadership Scale for Sports (LSS). The study tested invariance of all parameters (i.e., factor loadings, error variances, and factor variances–covariances) in the four-dimensional measurement model between two groups of student-athletes. For testing multi-group invariance of the proposed scale, 335 middle school and 320 high school student-athletes in Japan participated in this study. The modified version of the LSS consists of 35 items representing training instruction, democratic behaviour, positive feedback, and social support. A chi-square difference test was employed for model comparisons. The results supported configural, metric, scalar and factor variance–covariance invariance in the modified LSS across the two student-athlete groups.     

Exploring the contributions of job resources,job demands,and job self-efficacy to STEM teachers' job satisfaction: A commonality analysis

This study aims to examine how job resources, demands, and self-efficacy affect American STEM teachers' job satisfaction by analyzing the US TALIS 2018 data. Multiple regression and commonality analysis were used to analyze factors' significant contributions and their detailed real unique and common contributions to STEM teachers' job satisfaction. The results show that the final model explains 29.6% of the variances of STEM teachers' job satisfaction. The commonality analysis further showed that job resources, job demands, and job self-efficacy explained 23.5%, 8.6%, and 8.0% of variances of job satisfaction, respectively. However, these factor sets uniquely contributed 15.9%, 2.9%, and 2.1% of the variance, separately. This study confirms the validity of the revised job demands−resources model for STEM teachers' job satisfaction. Furthermore, the commonality analysis reveals the unique and independent contributions of job demands, resources, and self-efficacy to job satisfaction. Results from the research identified the significance of job resources contributing to the improvement of STEM teachers' job satisfaction.     

On Testability of Missing Data Mechanisms in Incomplete Data Sets

This article is concerned with the question of whether the missing data mechanism routinely referred to as missing completely at random (MCAR) is statistically examinable via a test for lack of distributional differences between groups with observed and missing data, and related consequences. A discussion is initially provided, from a formal logic standpoint, of the distinction between necessary conditions and sufficient conditions. This distinction is used to argue then that testing for lack of these group distributional differences is not a test for MCAR, and an example is given. The view is next presented that the desirability of MCAR has been frequently overrated in empirical research. The article is finalized with a reference to principled, likelihood-based methods for analyzing incomplete data sets in social and behavioral research.     

Comparisons of two statistical approaches to study growth curves: The multilevel model and the latent curve analysis

This article compares two statistical approaches for modeling growth across time. The two statistical approaches are the multilevel model (MLM) and latent curve analysis (LCA), which have been proposed to depict change or growth adequately. These two approaches were compared in terms of the estimation of growth profiles represented by the parameters of initial status and the rate of growth. A longitudinal data set obtained from a school‐based substance‐use prevention trial for adolescents was used to illustrate the similarities and differences between the two approaches. The results indicated that the two approaches yielded very compatible results. The parameter estimates associated with regression weights are the same, whereas those associated with variances and covariances are similar. The MLM approach is easier for model specification and is more efficient computationally in yielding results. The LCA approach, however, has the advantage of providing model evaluation, that is, an overall test of goodness of fit, and is more flexible in modeling and hypothesis testing as demonstrated in this study.     

AN INVESTIGATION OF AN EXTENSION OF ITEM SAMPLING WHICH YIELDS INDIVIDUAL SCORES1

The sampling procedures were designed so that the full matrix of item variances and covariances could be estimated. Three subtest sizes were investigated- subtests of size five, nine and sixteen items. In each of these implementations a double cross validation was used yielding two predicted scores for each individual. Discrepancy measures were also computed showing the difference between the observed and the predicted scores. The prediction of individual scores was accomplished within various ranges of error. The correlations between predicted scores and observed scores ranged from the .70′s to the .90′s, depending on the number of predictor variables used. The procedure is applicable in situations in which large numbers of individuals are tested or in situations where multiple measures are taken.     

Why might relative fit indices differ between estimators?

Previous research indicates that relative fit indices in structural equation modeling may vary across estimation methods. Sugawara and MacCallum (1993) explained that the discrepancy arises from difference in the function values for the null model with no further derivation given. In this study, we derive explicit solutions for parameters of the null model. The null model specifies the variances of the observed variables as model parameters and fixes all the covariances to be zero. Three methods of estimation are considered: the maximum likelihood (ML) method, the ordinary least squares (OLS) method, and the generalized least squares (GLS) method. Results indicate that ML and LS yield an identical estimator, which is different from GLS. Function values and associated chi‐square statistics of the null model vary across estimation methods. Consequently, relative fit indices using the null model as the reference point in computation may yield different results depending on the estimation method chosen. An illustration example is given and implications of this study are discussed.     

Handling missing data: analysis of a challenging data set using multiple imputation

Missing data is endemic in much educational research. However, practices such as step-wise regression common in the educational research literature have been shown to be dangerous when significant data are missing, and multiple imputation (MI) is generally recommended by statisticians. In this paper, we provide a review of these advances and their implications for educational research. We illustrate the issues with an educational, longitudinal survey in which missing data was significant, but for which we were able to collect much of these missing data through subsequent data collection. We thus compare methods, that is, step-wise regression (basically ignoring the missing data) and MI models, with the model from the actual enhanced sample. The value of MI is discussed and the risks involved in ignoring missing data are considered. Implications for research practice are discussed.