Developing local oral reading fluency cut scores for predicting high‐stakes test performance

This study evaluated the classification accuracy of a second grade oral reading fluency curriculum‐based measure (R‐CBM) in predicting third grade state test performance. It also compared the long‐term classification accuracy of local and publisher‐recommended R‐CBM cut scores. Participants were 266 students who were divided into a calibration sample (n = 170) and two cross‐validation samples (n = 46; n = 50), respectively. Using calibration sample data, local fall, winter, and spring R‐CBM cut scores for predicting students’ state test performance were developed using three methods: discriminant analysis (DA), logistic regression (LR), and receiver operating characteristic curve analysis (ROC). The classification accuracy of local and publisher‐recommended cut scores was evaluated across subsamples. Only DA and ROC produced cut scores that maintained adequate sensitivity (≥.70) across cohorts; however, LR and publisher‐recommended scores had higher levels of specificity and overall correct classification. Implications for developing local cut scores are discussed.     

The Partial Credit Model and Generalized Partial Credit Model as Constrained Nominal Response Models,With Applications in Mplus

The purpose of this article is to demonstrate constraining the nominal response model in Mplus software to calibrate data under the partial credit model (PCM) and generalized partial credit model (GPCM). Currently, many researchers are uncertain if the PCM and GPCM can be estimated within Mplus. Through model constraint commands in Mplus, we demonstrate that both models can be estimated in recent versions of this software. We present an example of this approach with data from 522 respondents on a subset of items from the Math Self-Efficacy Scale (Betz & Hackett, 1983). It is demonstrated that the presented model code is a viable way of estimating the models in Mplus.     

Models for Semiordered Data to Address Not Applicable Responses in Scale Measurement

The purpose of this study is to develop and evaluate unidimensional models that can handle semiordered data within scale items (i.e., items with multiple ordered response categories, and one additional nominal response category). We apply the models to scale data with not applicable (NA) responses to compare the model performance to conditions in which NA responses are treated as missing and ignored. We also conduct a small simulation study based on the operational study to evaluate the parameter recovery of the models under the operational conditions. Findings indicate that the proposed models show promise for (a) reducing standard errors of trait estimates for persons who select NA responses, (b) reducing nonresponse bias in trait estimates for persons who select NA responses, and (c) providing substantive information to practitioners about the nature of the relationship between NA selection and the trait of measurement.     

Measures of Effect Size for Comparative Studies: Applications, Interpretations, and Limitations

Although dissatisfaction with the limitations associated with tests for statistical significance has been growing for several decades, applied researchers have continued to rely almost exclusively on these indicators of effect when reporting their findings. To encourage an increased use of alternative measures of effect, the present paper discusses several measures of effect size that might be used in group comparison studies involving univariate and/or multivariate models. For the methods discussed, formulas are presented and data from an experimental study are used to demonstrate the application and interpretation of these indices. The paper concludes with some cautionary notes on the limitations associated with these measures of effect size. Copyright 2000 Academic Press.     

UNDERSTANDING DISPROPORTIONATE REPRESENTATION IN SPECIAL EDUCATION BY EXAMINING GROUP DIFFERENCES IN BEHAVIOR RATINGS

The aim of the current study was to examine mean‐group differences on behavior rating scales and variables that may predict such differences. Sixty‐five teachers completed the Clinical Assessment of Behavior–Teacher Form (CAB‐T) for a sample of 982 students. Four outcome variables from the CAB‐T were assessed. Hierarchical linear modeling was used to analyze variance components across three levels; examine mean‐group differences across outcome variables for student gender, race/ethnicity, and free or reduced‐price lunch status; and examine whether teacher variables predicted teacher‐specific differences in ratings. Results revealed that a significant amount of variance was attributable to teacher‐ and school‐level variables. Several mean‐group differences emerged, and some teacher‐specific differences in ratings across groups were predicted by teacher self‐efficacy for behavior management and teacher age, but not for teacher race/ethnicity, gender, or years of experience.     

Mediated Effects with the Parallel Process Latent Growth Model: An Evaluation of Methods for Testing Mediation in the Presence of Nonnormal Data

This Monte Carlo simulation study investigated the impact of nonnormality on estimating and testing mediated effects with the parallel process latent growth model and 3 popular methods for testing the mediated effect (i.e., Sobel’s test, the asymmetric confidence limits, and the bias-corrected bootstrap). It was found that nonnormality had little effect on the estimates of the mediated effect, standard errors, empirical Type I error, and power rates in most conditions. In terms of empirical Type I error and power rates, the bias-corrected bootstrap performed best. Sobel’s test produced very conservative Type I error rates when the estimated mediated effect and standard error had a relationship, but when the relationship was weak or did not exist, the Type I error was closer to the nominal .05 value.     

Authoritative schools: A test of a model to resolve the school effectiveness debate

School effectiveness research has fueled debate on the importance of a press for academic excellence versus communal values. Research on parenting styles offers a theoretical framework that may resolve the debate. We hypothesized that dimensions of parenting styles—demandingness (academic press) and responsiveness (communal values)—predict students’ mathematics achievement, engagement, and locus of control. HLM analyses of NELS: 88 data on 19,435 eighth-graders partially supported the hypothesis: Students’ perceptions of school responsiveness predicted their engagement and internal control. In addition, students in responsive schools had smaller differences in mathematics achievement and internal control attributable to SES, suggesting that responsive schools may increase equity. We offer suggestions for further investigation of the model in hope of resolving the debate.     

Additive Models for Multitrait–Multimethod Data With a Multiplicative Trait–Method Relationship: A Simulation Study

Confirmatory factor analysis (CFA) is widely used for analyzing multitrait-multimethod (MTMM) data. But there is no consensus about whether multiplicative or additive trait-method effect of its parameterization, most appropriately represents the underlying structure of MTMM data. Given the popularity of two additive CFA models, the CT-CM model and the CT-CU model, this simulation investigates their performance for multiplicative MTMM data. Results showed that the CT-CM model had much lower convergence and proper solution rates than the CT-CU model. Although both models had adequate fit for the converged solutions, all parameter estimates from the CT-CM model were unacceptably biased. The CT-CU model worked well in most conditions and was quite robust to the multiplicative data when the matrix size was 3T3M.     

Applying the Liu-Agresti Estimator of the Cumulative Common Odds Ratio to DIF Detection in Polytomous Items

Liu and Agresti (1996) proposed a Mantel and Haenszel-type (1959) estimator of a common odds ratio for several 2 × J tables, where the J columns are ordinal levels of a response variable. This article applies the Liu-Agresti estimator to the case of assessing differential item functioning (DIF) in items having an ordinal response variable. A simulation study was conducted to investigate the accuracy of the Liu-Agresti estimator in relation to other statistical DIF detection procedures. The results of the simulation study indicate that the Liu-Agresti estimator is a viable alternative to other DIF detection statistics.     

A Generalized DIF Effect Variance Estimator for Measuring Unsigned Differential Test Functioning in Mixed Format Tests

One approach to measuring unsigned differential test functioning is to estimate the variance of the differential item functioning (DIF) effect across the items of the test. This article proposes two estimators of the DIF effect variance for tests containing dichotomous and polytomous items. The proposed estimators are direct extensions of the noniterative estimators developed by Camilli and Penfield (1997) for tests composed of dichotomous items. A small simulation study is reported in which the statistical properties of the generalized variance estimators are assessed, and guidelines are proposed for interpreting values of DIF effect variance estimators.