Skills Diagnosis Using IRT-Based Continuous Latent Trait Models

This article summarizes the continuous latent trait IRT approach to skills diagnosis as particularized by a representative variety of continuous latent trait models using item response functions (IRFs). First, several basic IRT-based continuous latent trait approaches are presented in some detail. Then a brief summary of estimation, model checking, and assessment scoring aspects are discussed. Finally, the University of California at Berkeley multidimensional Rasch-model-grounded SEPUP middle school science-focused embedded assessment project is briefly described as one significant illustrative application.     

Using Restricted Latent Class Models to Map the Skill Structure of Achievement Items

This paper presents a new method for using certain restricted latent class models, referred to as binary skills models, to determine the skills required by a set o f test items. The method is applied to reading achievement data from a nationally representative sample o f fourth-grade students and offers useful perspectives on test structure and examinee ability, distinct from those provided by other methods o f analysis. Models fitted to small, overlapping sets o f items are integrated into a common skill map, and the nature o f each skill is then inferred from the characteristics o f the items for which it is required. The reading comprehension items examined conform closely to a unidimensional scale with six discrete skill levels that range from an inability to comprehend or match isolated words in a reading passage to the abilities required to integrate passage content with general knowledge and to recognize the main ideas o f the most difficult passages on the test.     

Conducting Confirmatory Latent Class Analysis Using Mplus

Latent class analysis (LCA) is an increasingly popular tool that researchers can use to identify latent groups in the population underlying a sample of responses to categorical observed variables. LCA is most commonly used in an exploratory fashion whereby no parameters are specified a priori. Although this exploratory approach is reasonable when very little prior research has been conducted in the area under study, it can be very limiting when much is already known about the variables and population. Confirmatory latent class analysis (CLCA) provides researchers with a tool for modeling and testing specific hypotheses about response patterns in the observed variables. CLCA is based on placing specific constraints on the parameters to reflect these hypotheses. The popular and easy-to-use latent variable modeling software package Mplus can be used to conduct a variety of CLCA types using these parameter constraints. This article focuses on the basic principles underlying the use of CLCA, and the Mplus programming code necessary for carrying it out.     

MIMIC Latent Class Analysis Model for Alcoholic Diagnosis

The multiple indicators multiple causes (MIMIC) latent class analysis (LCA) model is an excellent classification method when researchers cannot find a "gold standard" to classify participants. The MIMIC-LCA model includes features of a typical LCA model and also introduces a new relation between the latent class and covariates. In other words, a logistic regression type of analysis between participants' categorical latent status and their background information is added. Detailed statistical setups of the MIMIC-LCA model and algorithmic procedures are derived. The model features, parameter estimations, and model selections for MIMIC-LCA models are also presented. Specifically, the MIMIC-LCA model is estimated by a generalized expectation-maximization algorithm under the maximum likelihood frameworks. A substantive application of the MIMIC-LCA model in diagnosing alcoholics and, in particular, examining potential risk factors for alcoholism is demonstrated.     

Using LISREL to Fit Nonlinear Latent Curve Models

Latent curve models offer a flexible approach to the study of longitudinal data when the form of change in a response is nonlinear. This article considers such models that are conditionally linear with regard to the random coefficients at the 2nd level. This framework allows fixed parameters to enter a model linearly or nonlinearly, and random coefficients at the 2nd level may only enter linearly. Beginning with LISREL 8.80 for Windows, such models can be fitted, giving users greater flexibility in model specification. An example with LISREL syntax is provided.     

心理测量理论的新进展：潜在分类模型

随着心理测量理论的发展,潜在分类模型越来越多地受到国外研究者的关注,它能够通过分析考生的作答过程来探讨其潜在能力倾向,改变了以总分来评判学生能力的传统评价模式。首先探讨了作为该类模型基础的规则空间模型,然后综述了近年来国外有关潜在分类模型的研究,最后对该模型的研究现状进行了评价并对未来发展进行了展望。     

心理测量理论的新进展:潜在分类模型

认知诊断模型是新一代心理测量理论——认知诊断理论的核心。它可分为潜在特质模型和潜在分类模型两大类。其中,潜在分类模型主要用于分析被试的作答过程从而探讨被试的潜在知识结构,克服了CCT和IRT的缺陷,开创了教育与心理测量领域新的里程碑。本文首先介绍作为该类模型基础的规则空间模型,然后集中探讨在此基础上发展起来的较新的潜在分类模型,最后对这类模型进行了评价和展望。     

Using Latent Class Analysis to Set Academic Performance Standards

This study investigated using latent class analysis to set performance standards for assessments comprised of multiple-choice and performance assessment items. Employing this procedure, it is possible to use a sample of student responses to accomplish four goals: (a) determine how well a specified latent structure fits student performance data; (b) determine which latent structure best represents the relationships in the data; (c) obtain estimates of item parameters for each latent class; and (d) identify to which class within that latent structure each response pattern most likely belongs. Comparisons with the Angoff and profile rating methods revealed that the approaches agreed with each other quite well, indicating that both empirical and test-based judgmental approaches may be used for setting performance standards for student achievement.     

Multilevel Cognitive Diagnosis Models for Assessing Changes in Latent Attributes

Cognitive diagnosis models (CDMs) have been developed to evaluate the mastery status of individuals with respect to a set of defined attributes or skills that are measured through testing. When individuals are repeatedly administered a cognitive diagnosis test, a new class of multilevel CDMs is required to assess the changes in their attributes and simultaneously estimate the model parameters from the different measurements. In this study, the most general CDM of the generalized deterministic input, noisy “and” gate (G‐DINA) model was extended to a multilevel higher order CDM by embedding a multilevel structure into higher order latent traits. A series of simulations based on diverse factors was conducted to assess the quality of the parameter estimation. The results demonstrate that the model parameters can be recovered fairly well and attribute mastery can be precisely estimated if the sample size is large and the test is sufficiently long. The range of the location parameters had opposing effects on the recovery of the item and person parameters. Ignoring the multilevel structure in the data by fitting a single‐level G‐DINA model decreased the attribute classification accuracy and the precision of latent trait estimation. The number of measurement occasions had a substantial impact on latent trait estimation. Satisfactory model and person parameter recoveries could be achieved even when assumptions of the measurement invariance of the model parameters over time were violated. A longitudinal basic ability assessment is outlined to demonstrate the application of the new models.     

Latent Curve Models and Latent Change Score Models Estimated in R

When using multiple imputation in the analysis of incomplete data, a prominent guideline suggests that more than 10 imputed data values are seldom needed. This article calls into question the optimism of this guideline and illustrates that important quantities (e.g., p values, confidence interval half-widths, and estimated fractions of missing information) suffer from substantial imprecision with a small number of imputations. Substantively, a researcher can draw categorically different conclusions about null hypothesis rejection, estimation precision, and missing information in distinct multiple imputation runs for the same data and analysis with few imputations. This article explores the factors associated with this imprecision, demonstrates that precision improves by increasing the number of imputations, and provides practical guidelines for choosing a reasonable number of imputations to reduce imprecision for each of these quantities.     

Predictions of Individual Change Recovered With Latent Class or Random Coefficient Growth Models

Popular longitudinal models allow for prediction of growth trajectories in alternative ways. In latent class growth models (LCGMs), person-level covariates predict membership in discrete latent classes that each holistically define an entire trajectory of change (e.g., a high-stable class vs. late-onset class vs. moderate-desisting class). In random coefficient growth models (RCGMs, also known as latent curve models), however, person-level covariates separately predict continuously distributed latent growth factors (e.g., an intercept vs. slope factor). This article first explains how complex and nonlinear interactions between predictors and time are recovered in different ways via LCGM versus RCGM specifications. Then a simulation comparison illustrates that, aside from some modest efficiency differences, such predictor relationships can be recovered approximately equally well by either model—regardless of which model generated the data. Our results also provide an empirical rationale for integrating findings about prediction of individual change across LCGMs and RCGMs in practice.     

探索认知诊断研究的新思路——追踪研究中的诊断分析

认知诊断模型是基于测量属性对测试对象进行的分类。本文旨在将近年越来越受研究者重视的追踪研究与通常仅作横断研究的认知诊断模型结合起来,根据现有文献探讨在重复测量中对被试进行测量属性诊断的可行性,从而实现从发展的角度对追踪监测个体属性的诊断,实现对其稳定性和可变性的解释。本文结合大量研究成果,重点融合非补偿性DINA模型和补偿性DINO模型,在潜在转换分析模型(LTA)的基础上进行分析与阐述。     

A Comparison of Approaches for Estimating Covariate Effects in Nonparametric Multilevel Latent Class Models

The inclusion of covariates improves the prediction of class memberships in latent class analysis (LCA). Several methods for examining covariate effects have been developed over the past decade; however, researchers have limited to the comparisons of the performance among these methods in cases of the single-level LCA. The present study investigated the performance of three different methods for examining covariate effects in a multilevel setting. We conducted a simulation to compare the performance of the three methods when level-1 and level-2 covariates were simultaneously incorporated into the nonparametric multilevel latent class model to predict latent class membership at each level. The simulation results revealed that the bias-adjusted three-step maximum likelihood method performed equally well as the one-step method when the sample sizes were sufficiently large and the latent classes were distinct from each other. However, the unadjusted three-step method significantly underestimated the level-1 covariate effect in most conditions.

     

Creating IRT-Based Parallel Test Forms Using the Genetic Algorithm Method

In educational measurement, the construction of parallel test forms is often a combinatorial optimization problem that involves the time-consuming selection of items to construct tests having approximately the same test information functions (TIFs) and constraints. This article proposes a novel method, genetic algorithm (GA), to construct parallel test forms effectively. The sum of squared errors of the generated TIFs produced by GA were compared with those of the Swanson and Stocking method, and the Wang and Ackerman method. Experimental results show that tests constructed using GA yielded lower error, and an average improvement ratio above 90%.     

A Typology of State Financial Aid Grant Programs Using Latent Class Analysis

Research in Higher Education - State financial aid grant programs are commonly categorized as either need-based, merit-based, or both, but their initial eligibility requirements include many more...     

Building Latent Class Growth Trees

Researchers use latent class growth (LCG) analysis to detect meaningful subpopulations that display different growth curves. However, especially when the number of classes required to obtain a good fit is large, interpretation of the encountered class-specific curves might not be straightforward. To overcome this problem, we propose an alternative way of performing LCG analysis, which we call LCG tree (LCGT) modeling. For this purpose, a recursive partitioning procedure similar to divisive hierarchical cluster analysis is used: Classes are split until a certain criterion indicates that the fit does not improve. The advantage of the LCGT approach compared to the standard LCG approach is that it gives a clear insight into how the latent classes are formed and how solutions with different numbers of classes relate. The practical use of the approach is illustrated using applications on drug use during adolescence and mood regulation during the day.     

An NCME Instructional Module on Latent DIF Analysis Using Mixture Item Response Models

The purpose of this ITEMS module is to provide an introduction to differential item functioning (DIF) analysis using mixture item response models. The mixture item response models for DIF analysis involve comparing item profiles across latent groups, instead of manifest groups. First, an overview of DIF analysis based on latent groups, called latent DIF analysis, is provided and its applications in the literature are surveyed. Then, the methodological issues pertaining to latent DIF analysis are described, including mixture item response models, parameter estimation, and latent DIF detection methods. Finally, recommended steps for latent DIF analysis are illustrated using empirical data.     

Analyzing Fairness Among Linguistic Minority Populations Using a Latent Class Differential Item Functioning Approach

ABSTRACT

Differential item functioning (DIF) analyses have been used as the primary method in large-scale assessments to examine fairness for subgroups. Currently, DIF analyses are conducted utilizing manifest methods using observed characteristics (gender and race/ethnicity) for grouping examinees. Homogeneity of item responses is assumed denoting that all examinees respond to test items using a similar approach. This assumption may not hold with all groups. In this study, we demonstrate the first application of the latent class (LC) approach to investigate DIF and its sources with heterogeneous (linguistic minority groups). We found at least three LCs within each linguistic group, suggesting the need to empirically evaluate this assumption in DIF analysis. We obtained larger proportions of DIF items with larger effect sizes when LCs within language groups versus the overall (majority/minority) language groups were examined. The illustrated approach could be used to improve the ways in which DIF analyses are typically conducted to enhance DIF detection accuracy and score-based inferences when analyzing DIF with heterogeneous populations.     

Latent Variable Models of Genotype-Environment Covariance

This study was concerned with the identification of genotype-environment covariance (CovGE) on measures of work-related variables. Two approaches to the estimation of CovGE were used, 1 based on a univariate behavior genetic model and the other based on a bivariate behavior genetic model. The sample consisted of 136 pairs of identical twins reared together, 175 pairs of fraternal twins reared together, 83 pairs of identical twins reared apart, and 182 pairs of fraternal twins reared apart, obtained from the Swedish Adoption/Twin Study of Aging (SATSA). Most of the CovGEs examined were significant, suggesting that (a) failure to specify CovGE in behavior genetic models will lead to bias in the estimates of the other parameters, and (b) CovGE is an important influence on individual difference in the workplace.