ESTIMATING THE RELIABILITY OF MULTIPLE TRUE-FALSE TESTS

This study was designed to examine the level of dependence within multiple true-false (MTF) test item clusters by computing sets of item intercorrelations with data from a test composed of both MTF and multiple choice (MC) items. It was posited that internal analysis reliability estimates for MTF tests would be spurious due to elevated MTF within-cluster intercorrelations. Results showed that, on the average, MTF within-cluster dependence was no greater than that found between MTF items from different clusters, between MC items, or between MC and MTF items. But item for item, there was greater dependence between items within the same cluster than between items of different clusters.     

OPTIMAL NUMBER OF OPTIONS: AN INVESTIGATION OF THE ASSUMPTION OF PROPORTIONALITY

One assumption common to all models for determining the optimal number of options per item (e. g., Lord, 1977) is that total testing time is proportional to the number of items and the number of options per item. Therefore, under this assumption given a fixed testing time, the test can be shortened or lengthened by deleting or adding a proportional number of options. The present study examines the validity of this assumption in three tests which were administered with 2, 3, 4, and 5 options per item. The number of items attempted in the first 10 and 15 minutes of the testing session and the time needed to complete the tests were recorded. Thus, the rate of performance for both fixed time and fixed test length was analyzed. A strong and consistently negative relationship between rate of performance and the number of options was detected in all tests. Thus, the empirical results did not support the assumption of proportionality. Furthermore, the data indicated that the method by which options are deleted can play a role in this context. A more realistic assumption of generalized proportionality, proposed by Grier (1976), was supported by the results from a Mathematical Reasoning test, but was only partially supported for a Vocabulary and a Reading Comprehension test.     

FURTHER RESULTS ON THE STANDARD ERRORS OF ESTIMATE ASSOCIATED WITH ITEM-EXAMINEE SAMPLING PROCEDURES

Defining one observation as the score received by one examinee on one item, the results of this investigation suggest that, for a given test length, item-examinee sampling procedures having the same number of observation have, for all practical purposes, the same standard error in estimating μ but different standard errors in estimating σ. Additionally, the variance of the item difficulty indices (proportion answering the item correctly) was found to be a significant factor in accounting for differences in standard errors of estimating μ between normative distributions differing primarily in degree of skewness.     

A NOTE ON ALLOCATING ITEMS TO SUBTESTS IN MULTIPLE MATRIX SAMPLING AND APPROXIMATING STANDARD ERRORS OF ESTIMATE WITH THE JACKKNIFE

Investigated empirically through post mortem item-examinee sampling were the relative merits of two alternative procedures for allocating items to subtests in multiple matrix sampling and the feasibility of using the jackknife in approximating standard errors of estimate. The results indicate clearly that a partially balanced incomplete block design is preferable to random sampling in allocating items to subtests. The jackknife was found to better approximate standard errors of estimate in the latter item allocation procedure than in the former.     

Exploring Satisfying Experiences in Museums

This article presents results from an ongoing research project that investigates the experiences visitors find satisfying in museums. Using a list constructed from interviews with visitors and surveys, data were obtained from visitors in nine Smithsonian museums. Analysis of the results showed that experiences can be classified into four categories: Object experiences, Cognitive experiences, Introspective experiences, and Social experiences. The article points out that the type of most satisfying experience differs according to the characteristics of museums, exhibitions, and visitors. It also proposes an interpretation for these data, and suggests some possible applications.