AN INTERPRETATION OF LIVINGSTON'S RELIABILITY COEFFICIENT FOR CRITERION-REFERENCED TESTS

An alternative interpretation of Livingston's reliability coefficient is based on the notion of the relation of the size of the reliability coefficient to the range of talent. It is shown that the (generally) larger Livingston coefficient does not imply a smaller standard error of measurement and consequently does not imply a more dependable determination of whether or not a true score falls below (or exceeds) a given criterion value.     

CRITERION-REFERENCED APPLICATIONS OF CLASSICAL TEST THEORY 1,2

A reliability coefficient for criterion-referenced tests is developed from the assumptions of classical test theory. This coefficient is based on deviations of scores from the criterion score, rather than from the mean. The coefficient is shown to have several of the important properties of the conventional normreferenced reliability coefficient, including its interpretation as a ratio of variances and as a correlation between parallel forms, its relationship to test length, its estimation from a single form of a test, and its use in correcting for attenuation due to measurement error. Norm-referenced measurement is considered as a special case of criterion-referenced measurement.     

THE ISSUE OF ITEM AND TEST VARIANCE FOR CRITERION-REFERENCED TESTS1

It has been argued that item variance and test variance are not necessary characteristics for criterion-referenced tests, although they are necessary for normreferenced tests. This position is in error because it considers sample statistics as the criteria for evaluating items and tests. Within a particular sample, an item or test may have no variance, but in the population of observations for which the test was designed, calibrated, and evaluated, both items and tests must have variance.     

THE ISSUE OF ITEM AND TEST VARIANCE FOR CRITERION-REFERENCED TESTS: A CLARIFICATION

This note contends that item or score variability is an unnecessary characteristic of criterion-referenced tests as they have been traditionally conceived, namely, as measures of well defined classes of examinee behaviors.     

THE ISSUE OF ITEM AND TEST VARIANCE FOR CRITERION-REFERENCED TESTS: A REPLY

It is a necessary condition that items and tests have variance and discrimination in the range of interest (population of observations) for which they are calibrated and selected. The basis for selection of the calibration sample determines the kind of scale which will be developed, A random sample from a population of individuals leads to a norm-referenced scale, and a sample representative of abilities of a range of a characteristic leads to a criterion-referenced scale.     

EFFECTS OF DIFFERENT SAMPLES ON ITEM AND TEST CHARACTERISTICS OF CRITERION-REFERENCED TESTS

Although many have rejected classical test construction and analysis procedures for criterion-referenced tests, the present study was concerned with the possibility that classical procedures are both applicable and appropriate when samples of both mastery and nonmastery examinees are employed. A rationale for using these samples was presented, and empirical evidence was gathered which supported the practice of combining samples to increase the variance of test scores and thereby permit the proper estimate of reliability and item validities.     

THE ROLE OF RELIABILITY IN CRITERION-REFERENCED TESTS

In discussion of the properties of criterion-referenced tests, it is often assumed that traditional reliability indices, particularly those based on internal consistency, are not relevant. However, if the measurement errors involved in using an individual's observed score on a criterion-referenced test to estimate his or her universe scores on a domain of items are compared to errors of an a priori procedure that assigns the same universe score (the mean observed test score) to all persons, the test-based procedure is found to improve the accuracy of universe score estimates only if the test reliability is above 0.5. This suggests that criterion-referenced tests with low reliabilities generally will have limited use in estimating universe scores on domains of items.     

IMPLICATIONS OF CRITERION-REFERENCED MEASUREMENT1,2

During the past several years measurement and instructional specialists have distinguished between norm-referenced and criterion-referenced approaches to measurement. More traditional, a norm-referenced measure is used to identify an individual's performance in relation to the performance of others on the same measure. A criterion-referenced test is used to identify an individual's status with respect to an established standard of performance. This discussion examines the implications of these two approaches to measurement, particularly criterion-referenced measurement, with respect to variability, item construction, reliability, validity, item analysis, reporting, and interpretation.     

RELIABILITY OF CRITERION-REFERENCED TESTS: A DECISION-THEORETIC FORMULATION

It has been suggested that the primary purpose for criterion-referenced testing in objective-based instructional programs is to classify examinees into mastery states or categories on the objectives included in the test. We have proposed that the reliability of the criterion-referenced test scores be defined in terms of the consistency of the decision-making process across repeated administrations of the test. Specifically, reliability is defined as a measure of agreement over and above that which can be expected by chance between the decisions made about examinee mastery states in repeated test administrations for each objective measured by the criterion-referenced test.     

TOWARD A SEMIOTIC THEORY OF CHOICE AND OF LEARNING

A bstract .  In this essay, Andrew Stables and Stephen Gough explore some of the implications for educational policy and practice of a view of living (and, therefore, of learning) as semiotic engagement. Such a view, Stables and Gough argue, has the potential to displace or circumvent essentially Cartesian models currently dominant within learning theory (cognitivism and responses to it) and within neoclassical economics (rational choice and responses to it). It thus enables synergies between theories of learning and of economic behavior, allowing for greater consistency in thinking about (but not necessarily prescribing for) both educational policy and provision, on the one hand, and curriculum and pedagogy, on the other. In addition, the authors claim that giving semiotics a foundational role in educational thinking provides a basis for the broader development of liberal political thought within a postmodern cultural context.     

为建设中国特色的马克思主义文艺学努力——董学文的文艺理论研究与批评述论

董学文教授长期为建设中国特色的马克思主义文艺学努力,取得了丰硕成果。本文就他对马克思主义经典作家文艺思想的研究,对错误文艺思潮及不健康文艺作品的辨析,对文艺学当代形态的探索等三个方面,做了简要评述。     

TOWARD A THEORY OF COMPUTER ESSAY GRADING

A principal components analysis was conducted to determine whether the measures of essays made by the computer could be grouped into factors. Six factors (fluency, spelling, diction, sentence structure, punctuation, and paragraphing) were identified. Factor scores were computed for the original essays, and two sets of papers were identified for each factor: papers with high scores and papers with low scores. The selected papers had average scores for all other factors. The high and low papers were then compared to determine what attributes of interest to humans were being reflected by the factors. The attributes found were: quantity of thought, spelling, range of vocabulary and word choice, structure of sentences, emphasis through subordination, and paragraph organization.