ISSUES IN ITEM BANKING

An item bank is defined as a relatively large collection of easily accessible test questions. A wide variety of item bank schemes that meet this relatively unrestricted definition is illustrated. Advantages and disadvantages of item banking and the conditions under which item banks have the most potential value are identified. An extensive list of questions to be asked in designing item banking systems is provided. The following five questions were singled out for further discussion: How many items should be in the bank? Should users develop their own item collections or use the collections of others? How should the items be classified? Should items be calibrated? Will each test have different items or will the same test be administered to all?     

PRINTED COMMENTS WITH ITEM ANALYSES

This article suggests that item analysis computer programs include, as an option, a printing of English comments alongside of the item statistics. The particular comments that appear for each item depend upon that item's statistics. These comments should provide clear directions to the classroom teacher for the improvement of the items, and they should be flexible to reflect the differing decisions required by norm-referenced items and criterion-referenced items.     

AN ITERATIVE ITEM BIAS DETECTION METHOD

Two strategies for assessing item bias are discussed: methods that compare (transformed) item difficulties unconditional on ability level and methods that compare the probabilities of correct response conditional on ability level. In the present study, the logit model was used to compare the probabilities of correct response to an item by members of two groups, these probabilities being conditional on the observed score. Here the observed score serves as an indicator of ability level. The logit model was iteratively applied: In the Tth iteration, the T items with the highest value of the bias statistic are excluded from the test, and the observed score indicator of ability for the (T + 1)th iteration is computed from the remaining items. This method was applied to simulated data. The results suggest that the iterative logit method is a substantial improvement on the noniterative one, and that the iterative method is very efficient in detecting biased and unbiased items.     

EFFECTS OF VIOLATING ITEM CONSTRUCTION PRINCIPLES1

Two forms of a social studies achievement test were constructed with half the items for each form containing a cue, grammar, or length fault. Faults were found to make the items easier, which was supported by confidence intervals for the differences. However, validity coefficients with achievement and intelligence criteria, as well as the reliability coefficients, were virtually unchanged. The results agreed with those of Dunn and Goldstein (1959), even though the methodology differed. A suggested measure of test-wiseness for groups is presented.     

TIME ALLOCATION AND PATTERNS OF ITEM RESPONSE

A measure of student patterns of item response is usually interpreted as a measure of carelessness or guessing. However, when aggregated to the class level, it was shown that the index identifies classes that have a poor match between test content and instructional coverage. In particular, Sato's (1980) caution index identified classes whose pattern of time allocation across content areas was atypical. In addition, it was found that the mean caution index for a class can best be interpreted knowing the within-class standard deviation of the index.     

ITEM ANALYSIS FOR TEACHER-MADE MASTERY TESTS1

Various item selection techniques are compared on resultant criterionreferenced reliability and validity. Techniques compared include three nominal criterion-referenced methods, a traditional point biserial selection, teacher selection, and random selection. Eighteen volunteer junior and senior high school teachers supplied behavioral objectives and item pools ranging from 26 to 40 items. Each teacher obtained reponses from four classes. Pairs of tests of various length were developed by each item selection method. Estimates of test reliability and validity were obtained using responses independent of the test construction sample. Resultant reliability and validity estimates were compared across item selection techniques. Two of the criterion-referenced item selection methods resulted in consistently higher observed validity. However, the small magnitude of improvement over teacher or random selection raises a question as to whether the benefit warrants the necessary extra effort on the part of the classroom teacher.     

CONSTRUCTING AN ITEM BANK USING PARTIAL CREDIT SCORING

A method for banking test items scored in several ordered response categories is described. Each item is seen as an ordered sequence of steps, and test forms are equated using the estimated difficulties of the steps in their shared items. Procedures for analyzing the internal consistency of individual links and for analyzing the coherence of an entire linking structure are described. The methodology is used to link six forms of a mathematics problem solving test.     

ITEM BANKS: WHAT, WHY, HOW

In this paper, we discuss curricular implications of item banking and its practical value to teachers and students, and list a variety of working banks with their sources. We also review the psychometric basis of item banking, outline a family of computer programs for accomplishing banking, and give the equations necessary to build a bank. We conclude with a discussion of item quality control and examples of items that misfit because of miskeying, guessing, or carelessness.     

VALIDITY OF APPROXIMATION TECHNIQUES FOR DETECTING ITEM BIAS

The purpose of this research was to recommend an item bias procedure when the number of minority examinees is too small to use preferred three-parameter IRT methods. The chi-square, Angoff delta-plot, andpseudo-IRT indices were compared with both real and simulated data. For the real test data a criterion of known bias had been established by cross-validated IRT-3 results. The findings from the Math Test and the simulated test were consistent. The pseudo-IRT approach was best (measured by both correlations and percent agreement) in delecting criterion bias. The chi-square was close in accuracy to the pseudo-IRT index. The Angoff delta-plot method was found to be inadequate on both heuristic and empirical grounds. In extreme cases it even identified items as biased against whites that were simulated to be biased against blacks. However, a modified Angoff index, where p-value differences were regressed on item point biserials (and the residualized values used as the index), was nearly as good as the chi-square in identifying known bias. A final caution was offered regarding the use of item bias techniques. The statistical flags should never be used mechanically to discard items; rather they should be used to inspect items for possible differences in meaning.     

THE PLACE AND VALUE OF ITEM BANKING1

School climate, defined here as the type of mobility system reflected in the school's selection procedures, was shown to interact with ethnic group membership and locus of control (after SES factors were controlled) in affecting student achievement in Israeli schools. Although achievement tended to be highest for all in a competitive and non‐selective environment, the achievement of the socially higher status group was found to be more sensitive to changes in the school atmosphere than that of the lower status group. Students revealing a strong internal locus of control appeared to be less affected by changes in the environment than others.