A Comparison of Tests for Equality of Two or More Independent Alpha Coefficients

This article extends the Bonett (2003a) approach to testing the equality of alpha coefficients from two independent samples to the case of m ≥ 2 independent samples. The extended Fisher-Bonett test and its competitor, the Hakstian-Whalen (1976) test, are illustrated with numerical examples of both hypothesis testing and power calculation. Computer simulations are used to compare the performance of the two tests and the Feldt (1969) test (for m = 2) in terms of power and Type I error control. It is shown that the Fisher-Bonett test is just as effective as its competitors in controlling Type I error, is comparable to them in power, and is equally robust against heterogeneity of error variance.     

Estimation of the Reliability of Differences Under Revised Reliabilities of Component Scores

Projecting the changes in the reliability of a difference score (d =× - Y ) as a consequence of changes in the reliabilities of X and Y does not represent a straightforward application of the Spearman-Brown formula. Formulas are developed for estimating the changes in the reliability of X-Y under two possible assumptions: (a ) × and Y have equal variances both before and after their reliabilities are altered, and (b ) × and Y have unequal variances before and after × and Y are modified. The second of these situations, which includes the first as a special case, is probably the more common .     

SCHOOL PSYCHOLOGISTS’ ETHICAL STRAIN AND RUMINATION: INDIVIDUAL PROFILES AND THEIR ASSOCIATIONS WITH WEEKLY WELL‐BEING

We investigated school psychologists’ experiences of ethical strain (the frequency of ethical dilemmas at work and the stress caused by these dilemmas) and dilemma‐related rumination outside working hours. Individual latent profiles were estimated at the study baseline based on these three dimensions. The psychologists’ weekly well‐being (vigor, exhaustion, and sleep quality) was compared against their profile during the following three working weeks. The sample included 133 school psychologists, among whom four groups were identified: Low ruminators (39%), an Intermediate group (39%), High ruminators (20%), and Atypical outliers (2%). High ruminators fared least well in terms of weekly well‐being. Of all the groups, they reported the lowest levels of vigor, the highest levels of exhaustion, and the lowest sleep quality. The study contributes to understanding how psychologists differ in their experiences of ethical strain, and highlights the role of ruminating: mental detachment from ethical demands is especially important for school psychologists’ well‐being.     

An Analysis of Knowledge Structure, Diversity and Diagnostic Abilities Among Pre-Service Science Teachers Within the Domain of Oxidation and Reduction Chemistry

The purpose of this study was to investigate the development of preservice science teachers’ knowledge structures in the domain of oxidation and reduction chemistry. Knowledge structures were elicited through video-recorded semi-structured interviews before and after the unit of instruction, and analyzed using a visual flow map representation. Paralleling these interviews, the preservice teachers were tasked with diagnosing middle school students’ scientific understandings. Data analyzed quantitatively and qualitatively showed large variation in knowledge structure complexity across the preservice teachers, strong correlations between measures of knowledge structure and diversity (as defined by the Shannon Wiener diversity index), and the development of more balanced knowledge structure representations. For most preservice teachers, their diagnostic scores of the middle school students showed a small increase.     

Confidence Intervals for the Proportion of Mastery in Criterion-Referenced Measurement

A relatively simple method is developed to obtain confidence intervals for a student's proportion of domain mastery in criterion-referenced or mastery measurement situations. The method uses the binomial distribution as a model for the student's scores under hypothetically repeated assessments. Though the use of this model is not a new idea, the method of implementation has not been proposed previously. The technique makes use of widely available F tables and hence does not require elaborate computer equipment or proprietary computer programs.     

The estimation of the IRT reliability coefficient and its lower and upper bounds, with comparisons to CTT reliability statistics

The primary purpose of this study is to investigate the mathematical characteristics of the test reliability coefficient ρ _XX′ as a function of item response theory (IRT) parameters and present the lower and upper bounds of the coefficient. Another purpose is to examine relative performances of the IRT reliability statistics and two classical test theory (CTT) reliability statistics (Cronbach’s alpha and Feldt–Gilmer congeneric coefficients) under various testing conditions that result from manipulating large-scale real data. For the first purpose, two alternative ways of exactly quantifying ρ _XX′ are compared in terms of computational efficiency and statistical usefulness. In addition, the lower and upper bounds for ρ _XX′ are presented in line with the assumptions of essential tau-equivalence and congeneric similarity, respectively. Empirical studies conducted for the second purpose showed across all testing conditions that (1) the IRT reliability coefficient was higher than the CTT reliability statistics; (2) the IRT reliability coefficient was closer to the Feldt–Gilmer coefficient than to the Cronbach’s alpha coefficient; and (3) the alpha coefficient was close to the lower bound of IRT reliability. Some advantages of the IRT approach to estimating test-score reliability over the CTT approaches are discussed in the end.     

A Theory-Based Comparison of the Reliabilities of Fixed-Length and Trials-to-Criterion Scoring of Physical Education Skills Tests

Abstract

The reliabilities of two types of measurement plans were compared across six hypothetical distributions of true scores or abilities. The measurement plans consisted of a fixed-length plan (FL), where the number of trials for all examinees is fixed in advance, and the trials-to-criterion plan (TTC), where the number of successful trials is fixed, and examinees continue until this criterion is reached. The comparisons revealed that for most hypothetical distributions considered, the FL plan produced higher test reliabilities. In certain cases of negative skewness, however, the TTC plan was superior. Two formulae were presented for the estimation of the reliability of a TTC test.     

Estimation of Measurement Error Variance at Specific Score Levels

An improved method is derived for estimating conditional measurement error variances, that is, error variances specific to individual examinees or specific to each point on the raw score scale of the test. The method involves partitioning the test into short parallel parts, computing for each examinee the unbiased estimate of the variance of part-test scores, and multiplying this variance by a constant dictated by classical test theory. Empirical data are used to corroborate the principal theoretical deductions.     

Defensive responding and leverpress topography compatibility: Effects of shock intensity,S-S interval,and lever position

The degree of compatibility between defensive responding and leverpress topography was investigated. Rats were trained to leverpress on a Sidman avoidance schedule for one 60-min session. Measures of defensive responding (jumping and rearing) were recorded during initial training. Shock intensity (.5, 1.0, and 2.0 mA) and S-S interval (1, 3, and 5 sec) were varied in Experiment 1. S-S interval had no effect on defensive responding or on performance. Rate of jumping was an increasing function of intensity only at lower intensities. Both jumping and rearing were significantly correlated with leverpressing. Shock intensity (1.0 and 2.0 mA) and lever position (3.02 and 16.83 cm) were varied in Experiment 2. Response rate was an increasing function of intensity at the high lever position but not at the low position. Correlations between rearing and leverpressing, and between jumping and leverpressing, were not significant for either lever position. Bolles’ (1970) theory of species-specific defense reactions was invoked to account for initial facilitated performance.