Estimating Classification Consistency and Accuracy for Cognitive Diagnostic Assessment

This article introduces procedures for the computation and asymptotic statistical inference for classification consistency and accuracy indices specifically designed for cognitive diagnostic assessments. The new classification indices can be used as important indicators of the reliability and validity of classification results produced by cognitive diagnostic assessments. For tests with known or previously calibrated item parameters, the sampling distributions of the two new indices are shown to be asymptotically normal. To illustrate the computations of the new indices, we apply them to the real diagnostic data from a fraction subtraction test (Tatsuoka). We also use simulated data to evaluate their performances and distributional properties.