WHICH EXAMINEES ARE MOST FAVOURED BY THE USE OF MULTIPLE CHOICE TESTS?

Scores were obtained from 198 ninth grade students on achievement motivation, test anxiety, testwiseness, and risktaking. Tests in mathematics and vocabulary were constructed in free response and multiple choice form, and administered to the subjects in that order, with an interval of 5 weeks between administrations. Partial correlations were computed between scores on the multiple choice tests and achievement motivation, test anxiety, testwiseness, and risktaking, with free response scores partialled out. The partial correlations were corrected for the unreliability in the free response scores, and tested for significance. All partials involving achievement motivation and test anxiety were nonsignificant, as were all partials based on mathematics scores. The partial correlations of vocabulary scores with testwiseness and risktaking were significant without exception. It was concluded that the use of multiple choice tests can favour certain examinees those who are highly testwise and willing to take risks in the test situation. It was noted that the extent to which these examinees were favoured was dependent on the nature of the test, and that a verbal test seemed more susceptible than a numerical test.     

ITEM BEHAVIOR ASSOCIATED WITH CHANGES IN THE NUMBER OF ALTERNATIVES IN MULTIPLE CHOICE ITEMS1

Language reading examinations in French and Spanish were administered to students in order to compare the behavior of “natural” four-choice items with “natural” five-choice items rescored as four-choice items after removing the least popular incorrect alternative. No significant differences in the regression systems of these items were found. However, “natural” four-choice items were significantly less reliable than “natural” five-choice items.     

A NEW APPROACH TO THE PROBLEM OF CHANGING INITIAL RESPONSES TO MULTIPLE CHOICE QUESTIONS

The hypothesis that it is unwise to change answers to multiple choice questions was tested using the technique of multiple regression analysis. The net number of correct answers as a result of changing responses was regressed against final grade in the course, numeric score on the exam, percent of total answers changed for all questions and for analytical questions, sex of the student, and scope of the exam.
The results show that there are gains to be made by changing responses. The variables which proved to be significant indicated that students who did well on the test changed a large percentage of answers, and that those who were taking a final exam tended to gain more. Final grades, sex of the student, and analytical questions had no significant impact on gains from changing responses. On the basis of the results gathered, the authors reject the hypothesis that changing responses is unwise.     

ESTIMATING THE RELIABILITY OF MULTIPLE TRUE-FALSE TESTS

This study was designed to examine the level of dependence within multiple true-false (MTF) test item clusters by computing sets of item intercorrelations with data from a test composed of both MTF and multiple choice (MC) items. It was posited that internal analysis reliability estimates for MTF tests would be spurious due to elevated MTF within-cluster intercorrelations. Results showed that, on the average, MTF within-cluster dependence was no greater than that found between MTF items from different clusters, between MC items, or between MC and MTF items. But item for item, there was greater dependence between items within the same cluster than between items of different clusters.     

EFFECTS OF DIFFERENT SAMPLES ON ITEM AND TEST CHARACTERISTICS OF CRITERION-REFERENCED TESTS

Although many have rejected classical test construction and analysis procedures for criterion-referenced tests, the present study was concerned with the possibility that classical procedures are both applicable and appropriate when samples of both mastery and nonmastery examinees are employed. A rationale for using these samples was presented, and empirical evidence was gathered which supported the practice of combining samples to increase the variance of test scores and thereby permit the proper estimate of reliability and item validities.     

THE ISSUE OF ITEM AND TEST VARIANCE FOR CRITERION-REFERENCED TESTS1

It has been argued that item variance and test variance are not necessary characteristics for criterion-referenced tests, although they are necessary for normreferenced tests. This position is in error because it considers sample statistics as the criteria for evaluating items and tests. Within a particular sample, an item or test may have no variance, but in the population of observations for which the test was designed, calibrated, and evaluated, both items and tests must have variance.     

THE EFFECT OF DIFFERENTIAL OPTION WEIGHTING ON MULTIPLE-CHOICE OBJECTIVE TESTS1

The purpose of this study was to determine in what way Guttman weighting affected the internal consistency and intercorrelation of the suhtests of the Scholastic Aptitude Test. The tests were first scored with Guttman weights and then with conventional correction-for-guessing weights. The internal consistency of the tests increased markedly when Guttman weights were used. The correlation of the two verbal subtests increased somewhat when Guttman weights were used, but the correlation of the two mathematics subtests as well as the intercorrelation of all verbal and mathematics subtests decreased. Differences in the factor structure of the Guttman- and conventionally-weighted subtests were used to explain the result.     

AN EXAMINATION OF THE CONTEXT EFFECT IN ITEM SAMPLING

Item sampling and/or multiple matrix sampling techniques have been recommended for a variety of purposes. For some of these purposes, it must be assumed that examinee performance on a set of items is unaffected by the conditions under which the items are taken (i.e., no context effect exists). In this paper factors that may lead to a context effect among high school students are discussed. The net effect of such factors on examinee scores for an English test and a mathematics test is investigated empirically. For the English test there was little support for the existence of a context effect, However, a definite context effect was found for the mathematics test.     

THE EFFECT OF DIFFERENTIAL WEIGHTING OF INDIVIDUAL ITEM RESPONSES ON THE PREDICTIVE VALIDITY AND RELIABILITY OF AN APTITUDE TEST

An empirical investigation of the effect of choice weight scoring on predictive validity and reliability. Choice weight scoring refers to the procedure whereby different weights may be assigned to all the options of an item. Four groups of subjects were included in the experiment. Weights derived from each group were used to score tests for another group in order to assess the cross-validity of the weighted scoring. In no case did the increments in reliability and validity due to the weighted scoring exceed .03.     

THE ISSUE OF ITEM AND TEST VARIANCE FOR CRITERION-REFERENCED TESTS: A REPLY

It is a necessary condition that items and tests have variance and discrimination in the range of interest (population of observations) for which they are calibrated and selected. The basis for selection of the calibration sample determines the kind of scale which will be developed, A random sample from a population of individuals leads to a norm-referenced scale, and a sample representative of abilities of a range of a characteristic leads to a criterion-referenced scale.     

THE ISSUE OF ITEM AND TEST VARIANCE FOR CRITERION-REFERENCED TESTS: A CLARIFICATION

This note contends that item or score variability is an unnecessary characteristic of criterion-referenced tests as they have been traditionally conceived, namely, as measures of well defined classes of examinee behaviors.     

活动积温及其不等效性研究

本文从农业生产的角度出发,提出等效积温的概念,从理论上分析了活动积温与等效积温的关系和海拔高度对气温等效性的影响.以等效积温取代活动积温更能反映气温与农业景观的关系.     

THE EFFECTS OF VIOLATIONS OF UNIDIMENSIONALITY ON THE ESTIMATION OF ITEM AND ABILITY PARAMETERS AND ON ITEM RESPONSE THEORY EQUATING OF THE GRE VERBAL SCALE

One of the major assumptions of item response theory (IRT)models is that performance on a set of items is unidimensional, that is, the probability of successful performance by examinees on a set of items can be modeled by a mathematical model that has only one ability parameter. In practice, this strong assumption is likely to be violated. An important pragmatic question to consider is: What are the consequences of these violations? In this research, evidence is provided of violations of unidimensionality on the verbal scale of the GRE Aptitude Test, and the impact of these violations on IRT equating is examined. Previous factor analytic research on the GRE Aptitude Test suggested that two verbal dimensions, discrete verbal (analogies, antonyms, and sentence completions)and reading comprehension, existed. Consequently, the present research involved two separate calibrations (homogeneous) of discrete verbal items and reading comprehension items as well as a single calibration (heterogeneous) of all verbal item types. Thus, each verbal item was calibrated twice and each examinee obtained three ability estimates: reading comprehension, discrete verbal, and all verbal. The comparability of ability estimates based on homogeneous calibrations (reading comprehension or discrete verbal) to each other and to the all-verbal ability estimates was examined. The effects of homogeneity of item calibration pool on estimates of item discrimination were also examined. Then the comparability of IRT equatings based on homogeneous and heterogeneous calibrations was assessed. The effects of calibration homogeneity on ability parameter estimates and discrimination parameter estimates are consistent with the existence of two highly correlated verbal dimensions. IRT equating results indicate that although violations of unidimensionality may have an impact on equating, the effect may not be substantial.     

MULTIPLE PROCESSING STRATEGIES AND THE CONSTRUCT VALIDITY OF VERBAL REASONING TESTS

This study examines the influence of processing strategies, and the associated metacomponents that determine when to apply them, on the construct validity of a verbal reasoning test. Three strategies for solving verbal analogy items were examined: a rule-oriented strategy, an association strategy, and a partial rule strategy. Construct validity was studied in two separate stages: construct representation and nomothetic span. For construct representation, evidence was obtained that all three strategies, and their related metacomponents, are associated with performance on analogy items. For nomothetic span, the current study found that all three strategies contribute to individual differences in verbal reasoning and to the predictive validity of the test. The results of this study also point to the utility of metacomponents as constructs for describing and understanding test performance. Implications of the results for test development and theories of aptitude are elaborated.