Differential Item Functioning Effect Size From the Multigroup Confirmatory Factor Analysis for a Meta-Analysis: A Simulation Study

This study presents a new approach to synthesizing differential item functioning (DIF) effect size: First, using correlation matrices from each study, we perform a multigroup confirmatory factor analysis (MGCFA) that examines measurement invariance of a test item between two subgroups (i.e., focal and reference groups). Then we synthesize, across the studies, the differences in the estimated factor loadings between the two subgroups, resulting in a meta-analytic summary of the MGCFA effect sizes (MGCFA-ES). The performance of this new approach was examined using a Monte Carlo simulation, where we created 108 conditions by four factors: (1) three levels of item difficulty, (2) four magnitudes of DIF, (3) three levels of sample size, and (4) three types of correlation matrix (tetrachoric, adjusted Pearson, and Pearson). Results indicate that when MGCFA is fitted to tetrachoric correlation matrices, the meta-analytic summary of the MGCFA-ES performed best in terms of bias and mean square error values, 95% confidence interval coverages, empirical standard errors, Type I error rates, and statistical power; and reasonably well with adjusted Pearson correlation matrices. In addition, when tetrachoric correlation matrices are used, a meta-analytic summary of the MGCFA-ES performed well, particularly, under the condition that a high difficulty item with a large DIF was administered to a large sample size. Our result offers an option for synthesizing the magnitude of DIF on a flagged item across studies in practice.     

Using Multigroup Confirmatory Factor Analysis to Test Measurement Invariance in Raters: A Clinical Skills Examination Application

Psychometric models based on structural equation modeling framework are commonly used in many multiple-choice test settings to assess measurement invariance of test items across examinee subpopulations. The premise of the current article is that they may also be useful in the context of performance assessment tests to test measurement invariance of raters. The modeling approach and how it can be used for performance tests with less than optimal rater designs are illustrated using a data set from a performance test designed to measure medical students’ patient management skills. The results suggest that group-specific rater statistics can help spot differences in rater performance that might be due to rater bias, identify specific weaknesses and strengths of individual raters, and enhance decisions related to future task development, rater training, and test scoring processes.     

Applying Multigroup Confirmatory Factor Models for Continuous Outcomes to Likert Scale Data Complicates Meaningful Group Comparisons

Treating Likert rating scale data as continuous outcomes in confirmatory factor analysis violates the assumption of multivariate normality. Given certain requirements pertaining to the number of categories, skewness, size of the factor loadings, and so forth, it seems nevertheless possible to recover true parameter values if the data stem from a single homogeneous population. It is shown that, in a multigroup context, an analysis of Likert data under the assumption of multivariate normality may distort the factor structure differently across groups. In that case, investigations of measurement invariance (MI), which are necessary for meaningful group comparisons, are problematic. Analyzing subscale scores computed from Likert items does not seem to solve the problem.     

A Bayesian Approach for Multigroup Nonlinear Factor Analysis

The main purpose of this article is to develop a Bayesian approach for a general multigroup nonlinear factor analysis model. Joint Bayesian estimates of the factor scores and the structural parameters subjected to some constraints across different groups are obtained simultaneously. A hybrid algorithm that combines the Metropolis-Hastings algorithm and the Gibbs sampler is implemented to produce these joint Bayesian estimates. It is shown that this algorithm is computationally efficient. The Bayes factor approach is introduced for comparing models under various degrees of invariance across groups. The Schwarz criterion (BIC), a simple and useful approximation of the Bayes factor, is calculated on the basis of simulated observations from the Gibbs sampler. Efficiency and flexibility of the proposed Bayesian procedure are illustrated by some simulation results and a real-life example.     

Extension Procedures for Confirmatory Factor Analysis

We present factor extension procedures for confirmatory factor analysis that provide estimates of the relations of common and unique factors with external variables that do not undergo factor analysis. We present identification strategies that build upon restrictions of the pattern of correlations between unique factors and external variables. The first restriction minimizes the sum of squared correlations between unique factors and external variables. This approach is similar to the traditional factor extension procedure. The second restriction minimizes the complexity of the pattern of external correlations of unique factors. This approach has similarities with the simple structure ideal imposed on most factor rotation strategies. The procedures are illustrated with a real data example that demonstrates their applicability to real-world research questions.     

A Psychometric Investigation of the FACES-III: Confirmatory Factor Analysis with Replication

The Family Adaptability and Cohesion Evaluation Scales—III has been frequently used to measure the family environment. Although some data exist attesting to the reliability and validity of FACES-III data, the underlying factor structure of the instrument has not been assessed using a confirmatory methodology. The present research used data from a sample of 922 families with children having disabilities to conduct a confirmatory factor analysis. Results suggest that the structure of the instrument proposed by the author did not provide an acceptable fit to the data. Post-hoc model fitting identified 3-Factor solutions for the Actual and Ideal Scales as superior based on fit statistics and replication. The Cohesion Scale as originally proposed by the authors of the instrument remained intact. The original Adaptability Scale was divided into two scales, Adaptability, which focused on change in leadership and rules within the family, and Discipline and Control which targeted the role of children in the family system. The present results are consistent with challenges to the adaptability scale, and supports other exploratory analyses of the instrument.     

自我监控量表的探索性和验证性因素分析

该文在指出Snyder(1974)自我监控构念存在的缺陷后,根据自我呈现的动机和自我呈现过程中是否权衡自我和谐与人际和谐,把自我监控分为三种类型:自我导向、他人导向和高自我监控。在分析每种自我监控者行为特征的基础上,编制出新的自我监控量表,探索性因子分析与验证性因子分析显示该量表具有较好的结构效度。     

Confirmatory Factor Analysis of Ordinal Variables With Misspecified Models

Ordinal variables are common in many empirical investigations in the social and behavioral sciences. Researchers often apply the maximum likelihood method to fit structural equation models to ordinal data. This assumes that the observed measures have normal distributions, which is not the case when the variables are ordinal. A better approach is to use polychoric correlations and fit the models using methods such as unweighted least squares (ULS), maximum likelihood (ML), weighted least squares (WLS), or diagonally weighted least squares (DWLS). In this simulation evaluation we study the behavior of these methods in combination with polychoric correlations when the models are misspecified. We also study the effect of model size and number of categories on the parameter estimates, their standard errors, and the common chi-square measures of fit when the models are both correct and misspecified. When used routinely, these methods give consistent parameter estimates but ULS, ML, and DWLS give incorrect standard errors. Correct standard errors can be obtained for these methods by robustification using an estimate of the asymptotic covariance matrix W of the polychoric correlations. When used in this way the methods are here called RULS, RML, and RDWLS.     

用多组验证性因素分析模型比较教学评价结果

使用一种新的统计模型——多组验证性因素分析比较学生评教的数据,以判断教师间授课效果的差异。相比于传统T检验的方法,多组验证性因素分析模型消除了测量误差和起评点的影响,得出的结论更为可靠。使用两种方法对两位教师的教学评价的真实数据进行了比较,结果发现,在教学常规因子和总分两项上,两种统计方法得出了相反的结论。     

Measuring Teachers' Meta-Orientations to Curriculum: Application of Hierarchical Confirmatory Factor Analysis

Although curriculum scholars generally agree that teachers' meta-orientations to curriculum are hidden forces guiding their selection of curriculum goals, curriculum content, teaching methods, and assessment strategies, the existence of such a meta-orientation construct has not been empirically confirmed. The author used McNeil's (1996) 4 curriculum orientations to demonstrate how hierarchical confirmatory factor analysis can be applied to measure the meta-orientation construct. A hierarchical model was hypothesized that consisted of 4 1st-order factors and 1 2nd-order factor. The 1st-order factors represented McNeil's 4 separate curriculum orientations: academic, social reconstructionist, humanistic, and technological. The 2nd-order factor denoted the meta-orientation construct. Hierarchical confirmatory factor analysis of teacher data collected by a curriculum orientation inventory provided clear support for the hypothesized model.     

《大学教师教学关键环节行为评价问卷》的验证性因素分析

对某高校942名学生进行了《大学教师教学关键环节行为评价问卷》的测量,采用Anos4．0程序进行验证性因素分析,结果表明,该问卷的模型建构比较理想且各因子指标拟合较好,可以作为下一步研究的良好测量工具。     

The Construct Validity of Student Engagement: A Confirmatory Factor Analysis Approach

As institutions seek to promote student engagement on campus, the National Survey of Student Engagement (NSSE) is increasingly being used to chart progress and compare results using the Five Benchmark Scores. While recent research has begun to decompose the five benchmarks in a variety of ways; few research studies have sought to explore the underlying structure of these five benchmarks, their interdependence, and the extent to which the items do reflect those five dimensions. This study begins to address the instrument’s construct validity by submitting a single, first-time freshman cohort’s NSSE responses to a confirmatory factor analysis, and proposes as an alternative, eight “dimensions” of student engagement that fit this set of data slightly better and in a more useful way. Results have practical implications for institutions utilizing NSSE, but also contain conceptual implications pertaining to the application of these benchmarks.     

影响大学生出勤率的客观因素验证性因子分析

近年来高校学生课堂出勤率低这一现象有蔓延之势,长此下去会使部分学生丧失学习兴趣,导致高校教学质量下降,甚至是形成不良学风。利用结构方程建模的统计学方法建立影响大学生出勤率的客观因素模型,能够揭示影响大学生出勤率的各种因素,对提高大学生课堂出勤率策略的制定与实施具有启示意义。     

C.TEST语言能力的验证性因素分析

本文根据前期探索性因素分析成果和相关语言测试理论,提出了两个C.TEST语言能力结构假设模型,分别是C.TEST(A～D级)模型1和C.TEST(E～F级)模型1。然后,使用LISREL统计软件分别对两个假设模型和两个对比模型进行了验证性因素分析。统计数据表明:本研究中提出的两个假设模型都能较好地拟合数据,且优于两个对比模型,这说明研究中提出的两个假设模型是合理的,也说明C.TEST的确具有较高的构想效度。     

A Monte Carlo Study of Recovery of Weak Factor Loadings in Confirmatory Factor Analysis

The recovery of weak factors has been extensively studied in the context of exploratory factor analysis. This article presents the results of a Monte Carlo simulation study of recovery of weak factor loadings in confirmatory factor analysis under conditions of estimation method (maximum likelihood vs. unweighted least squares), sample size, loading size, factor correlation, and model specification (correct vs. incorrect). The effects of these variables on goodness of fit and convergence are also examined. Results show that recovery of weak factor loadings, goodness of fit, and convergence are improved when factors are correlated and models are correctly specified. Additionally, unweighted least squares produces more convergent solutions and successfully recovers the weak factor loadings in some instances where maximum likelihood fails. The implications of these findings are discussed and compared to previous research.     

Measuring Teacher Attitudes Toward Instructional Technology: A Confirmatory Factor Analysis of the TAC and TAT

In this article the authors present a confirmatory factor analysis of the Teachers’ Attitudes Toward Computers (TAC) and the Teachers’ Attitudes Toward Information Technology (TAT) scales by Christensen and Knezek (1996, 1998) using large samples from three states. The TAC was reduced from 98 items and nine factors to 35 items and eight factors, while maintaining superior psychometric properties. The TAT was reduced from 50 items and five factors to 20 items measuring five factors, similarly maintaining excellent psychometric properties. The authors also developed a combined short form of both instruments using 42 items to measure 11 factors for use in situations where parsimony is critical.     

Testing Measurement Invariance in the Target Rotated Multigroup Exploratory Factor Model

We propose a method to investigate measurement invariance in the multigroup exploratory factor model, subject to target rotation. We consider both oblique and orthogonal target rotation. This method has clear advantages over other approaches, such as the use of congruence measures. We demonstrate that the model can be implemented readily in the freely available Mx program. We present the results of 2 illustrative analyses, one based on artificial data, and the other on real data relating to personality in male and female psychology students.     

Measurement Invariance in Confirmatory Factor Analysis: An Illustration Using IQ Test Performance of Minorities

Measurement invariance with respect to groups is an essential aspect of the fair use of scores of intelligence tests and other psychological measurements. It is widely believed that equal factor loadings are sufficient to establish measurement invariance in confirmatory factor analysis. Here, it is shown why establishing measurement invariance with confirmatory factor analysis requires a statistical test of the equality over groups of measurement intercepts. Without this essential test, measurement bias may be overlooked. A re-analysis of a study by Te Nijenhuis, Tolboom, Resing, and Bleichrodt (2004) on ethnic differences on the RAKIT IQ test illustrates that ignoring intercept differences may lead to the conclusion that bias of IQ tests with respect to minorities is small, while in reality bias is quite severe.