THE EFFECT OF DIFFERENTIAL WEIGHTING OF INDIVIDUAL ITEM RESPONSES ON THE PREDICTIVE VALIDITY AND RELIABILITY OF AN APTITUDE TEST

An empirical investigation of the effect of choice weight scoring on predictive validity and reliability. Choice weight scoring refers to the procedure whereby different weights may be assigned to all the options of an item. Four groups of subjects were included in the experiment. Weights derived from each group were used to score tests for another group in order to assess the cross-validity of the weighted scoring. In no case did the increments in reliability and validity due to the weighted scoring exceed .03.     

EFFECTS OF COACHING ON GRE APTITUDE TEST SCORES

Test preparation activities were determined for a large representative sample of Graduate Record Examination (GRE) Aptitude Test takers. About 3% of these examinees had attended formal coaching programs for one or more sections of the test.
After adjusting for differences in the background characteristics of coached and uncoached students, effects on test scores were related to the length and the type of programs offered. The effects on GRE verbal ability scores were not significantly related to the amount of coaching examinees received, and quantitative coaching effects increased slightly but not significantly with additional coaching. Effects on analytical ability scores, on the other hand, were related significantly to the length of coaching programs, through improved performance on two analytical item types, which have since been deleted from the test.
Overall, the data suggest that, when compared with the two highly susceptible item types that have been removed from the GRE Aptitude Test, the test item types in the current version of the test (now called the GRE General Test) appear to show relatively little susceptibility to formal coaching experiences of the kinds considered here.     

ON DETECTING APTITUDE EFFECTS IN UNDERGRADUATE ACADEMIC ACHIEVEMENT SCORES

The widespread occurrence in undergraduate academic achievement studies of such measures as Grade Point Average and degree class is re‐examined in the light of evidence to indicate that these constructs may not be unitary. Performance data for a sample of (N = 88) University undergraduates when analysed in terms of a broadly perceived Apritude‐Treatment‐Interaction system, are used to illustrate both the manifold nature of student academic achievement as a dependent variable and the critically active role of the data analyst when operating within this system. By employing a particular set of aptitude predictors drawn from the personality domain, varying patterns of interrelationship with performance can be delineated as a function of the level and kind of articulation applied to the measures under inspection.     

RELIABILITY AND STRUCTURAL VALIDITY OF THE TEACHER RATING SCALES OF EARLY ACADEMIC COMPETENCE

Currently, there are few strengths‐based preschool rating scales that sample a wide array of behaviors believed to be essential for early academic success. The purpose of this study was to assess the factor structure of a new measure of early academic competence for at‐risk preschool populations. The Teacher Rating Scales of Early Academic Competence (TRS‐EAC) includes two broad scales (Early Academic Skills and Early Academic Enablers) and was completed by 60 teachers for 440 children enrolled in Head Start and public preschool classrooms. Evidence from two exploratory factor analyses supported a five‐factor solution for the Early Academic Skills Scale (Creative Thinking, Critical Thinking Skills, Numeracy, Early Literacy, and Comprehension) and a five‐factor solution for the Early Academic Enablers Scale (Approaches to Learning, Social and Emotional Competence, Fine Motor Skills, Gross Motor Skills, and Communication). TRS‐EAC scores also demonstrated good to excellent reliability and were related to children's performance on direct measures of early academic skills.     

THE EFFECT OF SELECTED POOR ITEM-WRITING PRACTICES ON TEST DIFFICULTY, RELIABILITY AND VALIDITY

Violations of four selected principles of writing multiple choice items were introduced into an undergraduate political science examination. Three of the four poor practices had no overall effect on test difficulty. A significant (α= .05) interaction effect between the poor practices and course achievement occurred for one of the four practices, with the poorer students generally gaining most from the poorly written items. KR 20 values were significantly lower for sets of items with the same flaws than for "good" versions of the items in three of four comparisons. The reductions in reliability were equivalent to those expected to result from shortening the test by 13 to 56 percent. Concurrent validity (correlation of experimental test scores with final examination scores) was significantly lower in two of four cases. The reductions in validity were equivalent to those expected to result from shortening the test by 56 to 83 percent.     

ITEM-RACE INTERACTION ON A TEST OF SCHOLASTIC APTITUDE1

Several samples of black and white students were drawn from the 1970 PSAT administration in Georgia and studied for item x race interaction on both the verbal and mathematical sections of the test. When subsamples of candidates were drawn from their respective racial groups, matched on mathematical for the study of verbal items and matched on verbal for the study of mathematical items, there was an observable decrease in the size of the item x race interaction, suggesting that one factor contributing to that interaction was simply the difference in performance levels on the test shown by the two races. Further analyses demonstrated a moderate item x group interaction for blacks native to different cities and a moderate item x group interaction for blacks native to areas of different population density.     

THE EFFECT OF DIFFERENTIAL OPTION WEIGHTING ON MULTIPLE-CHOICE OBJECTIVE TESTS1

The purpose of this study was to determine in what way Guttman weighting affected the internal consistency and intercorrelation of the suhtests of the Scholastic Aptitude Test. The tests were first scored with Guttman weights and then with conventional correction-for-guessing weights. The internal consistency of the tests increased markedly when Guttman weights were used. The correlation of the two verbal subtests increased somewhat when Guttman weights were used, but the correlation of the two mathematics subtests as well as the intercorrelation of all verbal and mathematics subtests decreased. Differences in the factor structure of the Guttman- and conventionally-weighted subtests were used to explain the result.     

高校学术失范问题的探索性实证研究

近年来,伴随社会转型,一度被视为社会最后一片净土的学术界也面临公共信任危机,传统的学术伦理与效率取向的学术制度安排之间的冲突,引发了一系列关于学术人活动和行为指向扭曲的争论。这种带有学术失范意义的现象,实际上反映了我国目前学术体制运行过程中所存在的功能性障碍。本文试图通过调查研究,对我国高校学术失范程度加以分析,并探索其中的制度性原因。研究结果表明,学术失范作为一种客观事实确实存在,它的产生与制度设计中普遍主义规则、传统研究取向和传统学术工作价值的失落相关,更与制度本身程序公正性缺失和非学术侵扰特别是行政权力过度介入存在高度关联。     

A COMPARISON OF THE RELIABILITY AND VALIDITY OF TWO METHODS FOR ASSESSING PARTIAL KNOWLEDGE ON A MULTIPLE-CHOICE TEST

Differential weighting of response alternatives and confidence testing have been proposed as ways to assess partial knowledge on multiple-choice tests. 211 students in an educational measurement course took their midterm examination under one of three procedures. Results from those students administered the test under conventional directions provided a baseline for comparing, in terms of reliability and validity, the results from students who took the test under the differential weighting of response alternatives or the confidence testing instructions. Reliability was estimated by the split-half technique. Validity was estimated by correlating midterm test scores with scores on a final examination. This investigation provides some support for the contention that validity can be improved using more sophisticated testing techniques. Suggestions for the conduct of more definitive studies were offered.     

MATCHED PAIR TRUE-FALSE SCORING: EFFECT ON RELIABILITY AND VALIDITY1

The matched pair technique for writing and scoring true-false items was designed to compensate for the acquiescence response set of primary grade children. The claim that this technique increases reliability to an appreciable extent over traditional true-false scoring was investigated by comparing alpha internal consistency coefficients computed for the matched pair true-false, traditional true-false, and three other scoring schemes. Both the total sample coefficients and individual classroom coefficients were computed from the standardization sample of a primary grade economics achievement test (Primary Test of Economic Understanding). Classroom reliability coefficients computed from the matched pair scores were found to be higher than those from scores computed by the other methods. Total sample coefficients obtained from four of the five methods were nearly equal. Evidence of the effects of each scoring technique on concurrent validity is also presented. Contrary to expectations, the correlations of traditional and matched pair scores with Iowa Test of Basic Skills (ITBS) subtests (when adjusted for differing reliabilities) were approximately equal.     

THE EFFECTS OF IMMEDIATE FEEDBACK ON RETENTION IN AN ACADEMIC SETTING

The present study was to determine the effects of immediate feedback on retention. The sample included 186 students enrolled in eight sections of an introductory psychology course. Four sections were randomly assigned to the experiment group, and four were designated the control group. Experimental treatment was accomplished through the use of a self‐scoring test‐answer sheet. A quasi‐experimental, nonequivalent control group research design was used to allow testing of a null hypothesis. Analysis of variance revealed no significant difference in retention rates between the experimental and control groups.