Application of Latent Trait Models to Identifying Substantively Interesting Raters

Historically, research focusing on rater characteristics and rating contexts that enable the assignment of accurate ratings and research focusing on statistical indicators of accurate ratings has been conducted by separate communities of researchers. This study demonstrates how existing latent trait modeling procedures can identify groups of raters who may be of substantive interest to those studying the experiential, cognitive, and contextual aspects of ratings. We employ two data sources in our demonstration—simulated data and data from a large‐scale state‐wide writing assessment. We apply latent trait models to these data to identify examples of rater leniency, centrality, inaccuracy, and differential dimensionality; and we investigate the association between rater training procedures and the manifestation of rater effects in the real data.     

Relating Latent Change Score and Continuous Time Models

The primary goal of this article is to demonstrate the close relationship between 2 classes of dynamic models in psychological research: latent change score models and continuous time models. The secondary goal is to point out some differences. We begin with a brief review of both approaches, before demonstrating how the 2 methods are mathematically and conceptually related. It will be shown that most commonly used latent change score models are related to continuous time models by the difference equation approximation to the differential equation. One way in which the 2 approaches differ is the treatment of time. Whereas there are theoretical and practical restrictions regarding observation time points and intervals in latent change score models, no such limitations exist in continuous time models. We illustrate our arguments with three simulated data sets using a univariate and bivariate model with equal and unequal time intervals. As a by-product of this comparison, we discuss the use of phantom and definition variables to account for varying time intervals in latent change score models. We end with a reanalysis of the Bradway–McArdle longitudinal study on intellectual abilities (used before by McArdle & Hamagami, 2004) by means of the proportional change score model and the dual change score model in discrete and continuous time.     

Using LISREL to Fit Nonlinear Latent Curve Models

Latent curve models offer a flexible approach to the study of longitudinal data when the form of change in a response is nonlinear. This article considers such models that are conditionally linear with regard to the random coefficients at the 2nd level. This framework allows fixed parameters to enter a model linearly or nonlinearly, and random coefficients at the 2nd level may only enter linearly. Beginning with LISREL 8.80 for Windows, such models can be fitted, giving users greater flexibility in model specification. An example with LISREL syntax is provided.     

Multilevel Cognitive Diagnosis Models for Assessing Changes in Latent Attributes

Cognitive diagnosis models (CDMs) have been developed to evaluate the mastery status of individuals with respect to a set of defined attributes or skills that are measured through testing. When individuals are repeatedly administered a cognitive diagnosis test, a new class of multilevel CDMs is required to assess the changes in their attributes and simultaneously estimate the model parameters from the different measurements. In this study, the most general CDM of the generalized deterministic input, noisy “and” gate (G‐DINA) model was extended to a multilevel higher order CDM by embedding a multilevel structure into higher order latent traits. A series of simulations based on diverse factors was conducted to assess the quality of the parameter estimation. The results demonstrate that the model parameters can be recovered fairly well and attribute mastery can be precisely estimated if the sample size is large and the test is sufficiently long. The range of the location parameters had opposing effects on the recovery of the item and person parameters. Ignoring the multilevel structure in the data by fitting a single‐level G‐DINA model decreased the attribute classification accuracy and the precision of latent trait estimation. The number of measurement occasions had a substantial impact on latent trait estimation. Satisfactory model and person parameter recoveries could be achieved even when assumptions of the measurement invariance of the model parameters over time were violated. A longitudinal basic ability assessment is outlined to demonstrate the application of the new models.     

Latent Curve Models and Latent Change Score Models Estimated in R

When using multiple imputation in the analysis of incomplete data, a prominent guideline suggests that more than 10 imputed data values are seldom needed. This article calls into question the optimism of this guideline and illustrates that important quantities (e.g., p values, confidence interval half-widths, and estimated fractions of missing information) suffer from substantial imprecision with a small number of imputations. Substantively, a researcher can draw categorically different conclusions about null hypothesis rejection, estimation precision, and missing information in distinct multiple imputation runs for the same data and analysis with few imputations. This article explores the factors associated with this imprecision, demonstrates that precision improves by increasing the number of imputations, and provides practical guidelines for choosing a reasonable number of imputations to reduce imprecision for each of these quantities.     

探索认知诊断研究的新思路——追踪研究中的诊断分析

认知诊断模型是基于测量属性对测试对象进行的分类。本文旨在将近年越来越受研究者重视的追踪研究与通常仅作横断研究的认知诊断模型结合起来,根据现有文献探讨在重复测量中对被试进行测量属性诊断的可行性,从而实现从发展的角度对追踪监测个体属性的诊断,实现对其稳定性和可变性的解释。本文结合大量研究成果,重点融合非补偿性DINA模型和补偿性DINO模型,在潜在转换分析模型(LTA)的基础上进行分析与阐述。     

Multidimensionality and IRT-Based Item lnvariance Indexes: The Effect of Between-Group Variation in Trait Correlation

Using a bidimensional two-parameter logistic model, the authors generated data for two groups on a 40-item test. The item parameters were the same for the two groups, but the correlation between the two traits varied between groups. The difference in the trait correlation was directly related to the number of items judged not to be invariant using traditional unidimensional IRT-based unsigned item invariance indexes; the higher trait correlation leads to higher discrimination parameter estimates when a unidimensional IRT model is fit to the multidimensional data. In the most extreme case, when r_{θ1 θ2}= Ofor one group and r _{θ1 θ2}= 1.0 for the other group, 33 out of 40 items were identified as not invariant. When using signed indexes, the effect was much smaller. The authors, therefore, suggest a cautious use of IRT-based item invariance indexes when data are potentially multidimensional and groups may vary in the strength of the correlations among traits.     

Estimating Non-Normal Latent Trait Distributions within Item Response Theory Using True and Estimated Item Parameters

Item response theory (IRT) procedures have been used extensively to study normal latent trait distributions and have been shown to perform well; however, less is known concerning the performance of IRT with non-normal latent trait distributions. This study investigated the degree of latent trait estimation error under normal and non-normal conditions using four latent trait estimation procedures and also evaluated whether the test composition, in terms of item difficulty level, reduces estimation error. Most importantly, both true and estimated item parameters were examined to disentangle the effects of latent trait estimation error from item parameter estimation error. Results revealed that non-normal latent trait distributions produced a considerably larger degree of latent trait estimation error than normal data. Estimated item parameters tended to have comparable precision to true item parameters, thus suggesting that increased latent trait estimation error results from latent trait estimation rather than item parameter estimation.     

Creating IRT-Based Parallel Test Forms Using the Genetic Algorithm Method

In educational measurement, the construction of parallel test forms is often a combinatorial optimization problem that involves the time-consuming selection of items to construct tests having approximately the same test information functions (TIFs) and constraints. This article proposes a novel method, genetic algorithm (GA), to construct parallel test forms effectively. The sum of squared errors of the generated TIFs produced by GA were compared with those of the Swanson and Stocking method, and the Wang and Ackerman method. Experimental results show that tests constructed using GA yielded lower error, and an average improvement ratio above 90%.     

Performance Modeling That Integrates Latent Trait and Class Theory

The qualitative characterization of individual performance that is central to modem psychological theory is not adequately modeled by traditional psychometric theory that assumes, among other things, unidimensionality. In the present study, data are presented that are more adequately modeled by HYBRID, a model that incorporates both latent trait and latent class components. The latent classes were defined by a cognitive analysis of the understanding that individuals have for a circumscribed domain. In addition to providing a better statistical fit, the analysis also improves the amount of diagnostic information available for a given individual.     

Using Restricted Latent Class Models to Map the Skill Structure of Achievement Items

This paper presents a new method for using certain restricted latent class models, referred to as binary skills models, to determine the skills required by a set o f test items. The method is applied to reading achievement data from a nationally representative sample o f fourth-grade students and offers useful perspectives on test structure and examinee ability, distinct from those provided by other methods o f analysis. Models fitted to small, overlapping sets o f items are integrated into a common skill map, and the nature o f each skill is then inferred from the characteristics o f the items for which it is required. The reading comprehension items examined conform closely to a unidimensional scale with six discrete skill levels that range from an inability to comprehend or match isolated words in a reading passage to the abilities required to integrate passage content with general knowledge and to recognize the main ideas o f the most difficult passages on the test.     

An NCME Instructional Module on Latent DIF Analysis Using Mixture Item Response Models

The purpose of this ITEMS module is to provide an introduction to differential item functioning (DIF) analysis using mixture item response models. The mixture item response models for DIF analysis involve comparing item profiles across latent groups, instead of manifest groups. First, an overview of DIF analysis based on latent groups, called latent DIF analysis, is provided and its applications in the literature are surveyed. Then, the methodological issues pertaining to latent DIF analysis are described, including mixture item response models, parameter estimation, and latent DIF detection methods. Finally, recommended steps for latent DIF analysis are illustrated using empirical data.     

HSK潜在维度的探索性分析——多维项目反应理论的应用

本文采用多维项目反应理论(MIRT)中的补偿型模型,探索性地分析HSK(初中等)阅读部分的潜在维度空间,为HSK的构想效度研究提供一个佐证。实验使用自编程序估计参数。在模型拟合检验时,应用聚类分析技术为被试分组。结果表明:三维的MIRT模型是阅读部分的最佳心理计量模型,其中维度1能较好地区分被试,是阅读部分的主要测量对象。维度1与维度2所代表的能力呈一定相反的变化趋势,维度3较独立于其他两个维度。     

Latent Variable Models of Genotype-Environment Covariance

This study was concerned with the identification of genotype-environment covariance (CovGE) on measures of work-related variables. Two approaches to the estimation of CovGE were used, 1 based on a univariate behavior genetic model and the other based on a bivariate behavior genetic model. The sample consisted of 136 pairs of identical twins reared together, 175 pairs of fraternal twins reared together, 83 pairs of identical twins reared apart, and 182 pairs of fraternal twins reared apart, obtained from the Swedish Adoption/Twin Study of Aging (SATSA). Most of the CovGEs examined were significant, suggesting that (a) failure to specify CovGE in behavior genetic models will lead to bias in the estimates of the other parameters, and (b) CovGE is an important influence on individual difference in the workplace.     

服装模特的表演技巧

服装表演是最接近于生活的舞台审美艺术,是在舞台上所呈现出的与现实生活有一定距离的审美活动.它要求模特不仅需要具备基本的素质条件,还要对不同类型、不同性质的服装具有敏感的认知能力,对不同音乐具有良好的配合能力,从而运用恰当的表演技巧,将服装的内涵表达到极致.     

Bayesian Analysis of Multivariate Latent Curve Models With Nonlinear Longitudinal Latent Effects

In longitudinal studies, investigators often measure multiple variables at multiple time points and are interested in investigating individual differences in patterns of change on those variables. Furthermore, in behavioral, social, psychological, and medical research, investigators often deal with latent variables that cannot be observed directly and should be measured by 2 or more manifest variables. Longitudinal latent variables occur when the corresponding manifest variables are measured at multiple time points. Our primary interests are in studying the dynamic change of longitudinal latent variables and exploring the possible interactive effect among the latent variables.

Much of the existing research in longitudinal studies focuses on studying change in a single observed variable at different time points. In this article, we propose a novel latent curve model (LCM) for studying the dynamic change of multivariate manifest and latent variables and their linear and interaction relationships. The proposed LCM has the following useful features: First, it can handle multivariate variables for exploring the dynamic change of their relationships, whereas conventional LCMs usually consider change in a univariate variable. Second, it accommodates both first- and second-order latent variables and their interactions to explore how changes in latent attributes interact to produce a joint effect on the growth of an outcome variable. Third, it accommodates both continuous and ordered categorical data, and missing data.