JOINT CONFIRMATORY FACTOR ANALYSIS OF THE WOODCOCK‐JOHNSON TESTS OF COGNITIVE ABILITIES,THIRD EDITION,AND THE STANFORD‐BINET INTELLIGENCE SCALES,FIFTH EDITION,WITH A PRESCHOOL POPULATION

This study examined the underlying constructs measured by the Woodcock‐Johnson Tests of Cognitive Abilities, Third Edition (WJ‐III COG) and the Stanford‐Binet Intelligence Scales, Fifth Edition (SB5), based on the Cattell‐Horn‐Carrol (CHC) theory of cognitive abilities. This study reports the results of the first joint confirmatory factor analysis of the WJ‐III COG and SB5 with an independently collected preschool‐aged sample. The WJ‐III COG and SB5 were administered to 200 preschool‐aged children of 4 to 5 with no known disorders or disabilities. Confirmatory factor analyses using maximum likelihood estimation were conducted to evaluate three models of increasing complexity and two alternative models to determine which model best describe the underlying constructs measured by the WJ‐III COG and the SB5. Though none of the models displayed a good fit to the data, results showed that the underlying construct of the two tests was best represented by a Three‐Stratum alternative CHC model in which the Gf factor and subtests were omitted. Exploratory factor analysis was conducted to provide further insights into the actual latent structure underlying the data. Implications of findings to guide school‐based practitioners in using cross‐battery assessment with preschool children were addressed.     

THE UTILITY OF MAZE ACCURATE RESPONSE RATE IN ASSESSING READING COMPREHENSION IN UPPER ELEMENTARY AND MIDDLE SCHOOL STUDENTS

This study examined the predictive utility of five formative reading measures: words correct per minute, number of comprehension questions correct, reading comprehension rate, number of maze correct responses, and maze accurate response rate (MARR). Broad Reading cluster scores obtained via the Woodcock–Johnson III (WJ III) Tests of Achievement served as the criterion variable. Examiners administered all reading measures to 98 students enrolled in upper elementary (UES) and middle school (MS) grades (grades 4 through 8). Results indicated significant intercorrelations within all predictor variables for the combined and UES samples; fewer intercorrelations existed at the MS level. MARR exhibited the highest correlation with WJ III Broad Reading across all correlational analyses. Additional regression analyses indicated that the MARR technique best predicted WJ III Broad Reading cluster scores across all groups analyzed. Limitations of the current study and suggestions for future research are discussed.     

An examination of the CTONI utilizing Gc‐Gf theory: A comparison of the CTONI and WJ‐III

The present investigation examined the validity of the Comprehensive Test of Nonverbal Intelligence (CTONI) with the Woodcock‐Johnson III Tests of Cognitive Abilities (WJ‐III COG) by administering these instruments in counterbalanced order to 60 college students. Results indicated that the mean CTONI NIQ score was not significantly different from the mean WJ‐III COG General Intellectual Ability (GIA) score. However, mean score differences were found between the CTONI NIQ and the WJ‐III COG Verbal Ability, Thinking Ability, Comprehension‐Knowledge, and Fluid Reasoning cluster scores. Although the correlations between the CTONI and the WJ‐III COG cluster scores were generally of small magnitude, the CTONI Geometric Nonverbal IQ composite score demonstrated construct specificity, whereas the CTONI Pictorial Nonverbal IQ scale did not. The implications of the findings are discussed and practitioners are asked to use caution when using this instrument to assess the fluid reasoning abilities of college students. © 2007 Wiley Periodicals, Inc.     

Assessing giftedness in children: Comparing the accuracy of three shortened measures of Intelligence to the Stanford–Binet Intelligence Scales,Fifth Edition

This study examined the accuracy of three shortened measures of intelligence: the Woodcock–Johnson Tests of Cognitive Ability, Third Edition Brief Intellectual Ability (WJ III COG BIA) score; the Stanford–Binet Intelligence Scale, Fifth Edition Abbreviated IQ (SB5 ABIQ); and the Kaufman Brief Intelligence Test IQ Composite (K‐BIT) in predicting giftedness as assessed by the SB5 Full Scale IQ score (SB5 FSIQ). Participants were 202 third grade students who were individually administered the SB5, WJ III, and K‐BIT. Four scores (SB5 ABIQ, K‐BIT, WJ III COG BIA, and SB5 FSIQ) were extracted. Correlational analyses revealed positive, significant relationships among the four scores. Discriminant function analyses, conducted to examine the accuracy of the three shortened measures of intelligence in predicting giftedness, revealed that overall, the WJ III score was the most accurate, and the K‐BIT score was the least accurate in identifying giftedness. © 2008 Wiley Periodicals, Inc.     

Examiner errors on the WRAT-R

A random sample of 55 WRAT-R protocols, completed by nine practitioners for a metropolitan school district in the South, was analyzed for examiner errors. All nine practitioners made errors, which occurred on 95% of the protocols and averaged 3.0 errors per protocol. The most frequent errors included failures to obtain a correct ceiling or basal, and failures to record examinees' responses. Correction of the examiner errors resulted in changes in 11 standard scores, and 3 additional changes in grade equivalent scores. These results indicate that WRAT-R administration and scoring are not as objective as assumed by the test developers, and that examiner errors on this test can adversely affect diagnostic decisions.     

Common WISC-III examiner errors: Evidence from graduate students in training

Sixty WISC-III protocols, administered by graduate students in training, were examined to obtain preliminary data on the frequency and types of administration and scoring errors that examinees commit. Results were compared with previous studies that have evaluated examiner errors on the Wechsler scales. In general, the present results were consistent with those of previous studies that have illustrated that a large number of scoring errors are committed by graduate students as well as by other professional groups. The majority of errors committed by participants in this study were general errors. That is, errors were not specific to a particular subtest. The five most frequent errors included failure to query, failure to record responses verbatim, reporting Full Scale IQ incorrectly, reporting Verbal IQ incorrectly, and adding individual subtest scores incorrectly. However, the traditional difficult to score Verbal subtests were not as troublesome for examiners in this study as they were for examiners in previous studies. In addition, significant decreases in the mean number of errors per protocol and in the number of most frequently occurring errors per protocol were noted. © 1998 John Wiley & Sons, Inc.     

Error correction in text

Previous research has shown that writers and editors of all ages and abilities have trouble correcting errors in texts. In this study, we were interested in discovering whether people do not correct these errors mainly because (1) they do not have the knowledge to correct them, or because (2) even though they do have the knowledge to correct the errors, they do not use it. The first case would point to aknowledge deficit, or a deficit at the cognitive level; the second case would point to aprocessing deficit, or a deficit at the metacognitive level. The study compared the number and type of implanted errors corrected by high school and college subjects working on two different texts under three different conditions. We found that, for both ages, the biggest stumbling block in correcting errors was not the knowledge of how to correct them, but rather a failure to detect them: They did not use their available knowledge to find the errors. This processing deficit may be the result of a dearth of available error-finding strategies, or knowledge may not be activated because of lack of motivation or because of a failure to perceive the nature of the task.     

The relative contributions of the Cattell‐Horn‐Carroll cognitive abilities in explaining writing achievement during childhood and adolescence

This study examined the relative contributions of measures of Cattell‐Horn‐Carroll (CHC) cognitive abilities in explaining writing achievement. Drawing from samples that covered the age range of 7 to 18 years, simultaneous multiple regression was used to regress scores from the Woodcock‐Johnson III (WJ III; Woodcock, McGrew, & Mather, 2001) that represent CHC broad and narrow abilities onto the WJ III Basic Writing Skills and Written Expression cluster scores. At most age levels, Comprehension‐Knowledge demonstrated moderate to strong effects on both writing clusters, Processing Speed demonstrated moderate effects on Basic Writing Skills and moderate to strong effects on Written Expression, and Short‐Term Memory demonstrated moderate effects. At the youngest age levels, Long‐Term Retrieval demonstrated moderate to strong effects on Basic Writing Skills and moderate effects on Written Expression. Auditory Processing, and Phonemic Awareness demonstrated moderate effects on only Written Expression at the youngest age levels and at some of the oldest age levels. Fluid Reasoning demonstrated moderate effects on both writing clusters only during some of the oldest age levels. Visual‐Spatial Thinking primarily demonstrated negligible effects. The results provide insights into the cognitive abilities most important for understanding the writing skills of children during the school‐age years. © 2008 Wiley Periodicals, Inc.     

Joint confirmatory factor analysis of the differential ability scales and the Woodcock‐Johnson Tests of Cognitive Abilities–Third Edition

This study examined the underlying constructs measured by the Differential Ability Scales (DAS; C.D. Elliott, 1990a) as they relate to the Cattell‐Horn‐Carroll (CHC) Theory (K.S. McGrew, 1997) of cognitive abilities. The DAS and Woodcock‐Johnson Tests of Cognitive Abilities (WJ‐III COG; R.W. Woodcock, K.S. McGrew, & N. Mather, 2001) were administered to 131 children in grades 3 through 5 who took part in a concurrent validity study included in the Woodcock‐Johnson Tests of Cognitive Abilities, Third Edition, technical manual (K.S. McGrew & R.W. Woodcock, 2001). Confirmatory factor analyses using maximum likelihood estimation were conducted with the AMOS 5.0 (J.L. Arbuckle, 2001) statistical program to evaluate three models of increasing complexity, to compare how well each fit the data set, and to identify the one that best described the underlying constructs measured by the DAS. Results suggested that the synthesized Three‐Stratum CHC Model provided the most parsimonious representation among the three models tested. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 119–138, 2007.     

A preliminary investigation of the concurrent validity of reading comprehension rate: A direct,dynamic measure of reading comprehension

Reading comprehension rate (RCR) is a direct measure of reading skills that may be useful in formatively evaluating students reading beyond the fourth‐grade level. To investigate the concurrent validity of RCR, we correlated RCR, reading comprehension level (RCL), and words correct per minute (WC/M) with the Broad Reading Cluster Scores of the Woodcock‐Johnson III Tests of Achievement (WJ‐III ACH) across 88 students in 4th, 5th, and 10th grades. Results showed that aloud‐RCR was significantly correlated with the WJ‐III ACH scores for 4th‐grade (r = .90; n = 22), 5th‐grade (r = .87; n = 29), and 10th‐grade (r = .65; n = 37) students. Regression analysis specified a one‐predictor model for 4th‐grade students (aloud‐RCR), a two‐predictor model for 5th‐grade students (WC/M and aloud‐RCR), and a one‐predictor model for 10th‐grade students (WC/M). Discussion focuses on directions for future research and applied issues related to RCR probe passage development. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 373–388, 2007.     

明代殿试考官概述及其特点

据台湾学生书局印行的《明代登科录汇编》中殿试录的记载,对明代殿试考官的人数结构;读卷官、提调官、监试官、受卷官、弥封官、掌卷官、巡绰官和供给官等八种考官的人数、官衔、科举功名出生及所占比例做了比较深入细致的分析。最后对殿试考官职位的高品级、各种考官来源上的京官化、考官荣誉头衔的高品级化、考官科举出身的标准化等四个特点进行了分析。     

Relations between measures of Cattell‐Horn‐Carroll (CHC) cognitive abilities and mathematics achievement across the school‐age years

Cognitive clusters from the Woodcock‐Johnson III (WJ III) Tests of Cognitive Abilities that measure select Cattell‐Horn‐Carroll broad and narrow cognitive abilities were shown to be significantly related to mathematics achievement in a large, nationally representative sample of children and adolescents. Multiple regression analyses were used to predict performance on the Math Calculation Skills and Math Reasoning clusters from the WJ III Tests of Achievement for 14 age groups ranging in age from 6 to 19 years. Comprehension‐Knowledge (Gc) demonstrated moderate relations with Math Calculation Skills after the early school‐age years and moderate to strong relations with Math Reasoning. Fluid Reasoning (Gf), Short‐term Memory (Gsm), and Working Memory generally demonstrated moderate relations with the mathematics clusters. Processing Speed (Gs) demonstrated moderate relations with Math Reasoning during the elementary school years and moderate to strong relations with Math Calculation Skills. During the earliest ages of the analysis, Long‐term Retrieval (Glr) demonstrated moderate relations with the mathematics clusters, and Auditory Processing (Ga) demonstrated moderate relations with Math Calculation Skills. Visual‐Spatial Thinking (Gv) generally demonstrated nonsignificant relations with the mathematics clusters. © 2003 Wiley Periodicals, Inc. Psychol Schs 40: 155–171, 2003.     

Automated Test Assembly for Cognitive Diagnosis Models Using a Genetic Algorithm

Much recent psychometric literature has focused on cognitive diagnosis models (CDMs), a promising class of instruments used to measure the strengths and weaknesses of examinees. This article introduces a genetic algorithm to perform automated test assembly alongside CDMs. The algorithm is flexible in that it can be applied whether the goal is to minimize the average number of classification errors, minimize the maximum error rate across all attributes being measured, hit a target set of error rates, or optimize any other prescribed objective function. Under multiple simulation conditions, the algorithm compared favorably with a standard method of automated test assembly, successfully finding solutions that were appropriate for each stated goal.     

Phonological and semantic factors in the object-naming errors of skilled and less-skilled readers

Children who read poorly have difficulty naming objects, and their errors usually bear a semantic or a phonetic resemblance to the correct words. Excessive semantic and phonetic naming errors could both be due to underlying phonological deficiencies in poor readers. When children cannot name an object because its name is not represented well in long-term memory or cannot be processed well, semantic information as well as partially available phonological information may be used in selecting an alternative response. This hypothesis was tested by looking for the joint influence of semantics and phonology in the naming errors of third-grade children. The same children were asked to name a set of pictured objects, repeat the object names after being spoken by the examiner, and recognize the objects from their spoken names. A separate group of children produced associative responses to the same pictures. First, it was found that, compared with skilled readers, less-skilled readers who named objects without any time pressure had a deficit that could not be attributed to repetition difficulty or limited vocabulary. Second, the naming errors showed a semantic relationship to the correct words that was as strong as that of the associative responses. Third, the naming errors also showed a phonetic relationship to the correct words, whereas the associative responses did not. Finding a joint semantic and phonetic effect in the naming errors of children suggests that the errors may be attributable to phonological deficiencies.     

ENGLISH LANGUAGE PROFICIENCY AND TEST PERFORMANCE: AN EVALUATION OF BILINGUAL STUDENTS WITH THE WOODCOCK‐JOHNSON III TESTS OF COGNITIVE ABILITIES

In this article, we report the findings of an exploratory empirical study that investigated the relationship between English Language Proficiency (ELP) on performance on the Woodcock‐Johnson Tests of Cognitive Abilities‐Third Edition (WJ III) when administered in English to bilingual students of varying levels of ELP. Sixty‐one second‐grade students, identified as Limited English Proficient, were recruited from a suburban public school district and were given the WJ III in addition to their annual state standardized assessment of ELP. The findings of this study provide evidence to support a linear, inverse relationship between ELP and performance on tests that require higher levels of English language development and mainstream cultural knowledge. The implications of the findings of the present study suggest that practitioners must consider an examinee's level of developmental language proficiency and cultural knowledge acquisition as continuous variables when determining the impact of such factors on test performance and evaluation regarding whether scores obtained from tests administered in English are indeed valid for interpretation.     

Effects of "On-Line" Test Feedback on the Seriousness of Subsequent Errors

The present study examined the effect of three modes of feedback on the seriousness of error types committed on a post-test. The measure of seriousness of error types, or wrong rules, indicates to what extent the wrong rules deviate from the right rule. An on-line, free-response, six-item pre-test covering addition of signed numbers was administered to 263 eighth graders. Upon completion of the pre-test, the students were randomly assigned to receive one of the following kinds of feedback to responses to a second six-item test: (a) information about whether or not the response was correct, (b) the correct answer, or (c) the correct rule for solving the problem. The effect of the feedback mode on the seriousness of errors committed on a third six-item test, the post-test, was found to be differential and dependent upon the seriousness of errors committed on the pre-test.     

Teacher heterogeneity,value-added and education policy

This study examines the theoretical and practical implications of ranking teachers with a one-dimensional value-added metric when teacher effectiveness varies across subjects or student types. We create a theoretical framework which suggests specific tests of the standard teacher input homogeneity assumption. Using North Carolina data we show that value-added fails to empirically meet these tests and document that this leads to a large number of teacher misrankings. Thus, critics of potential value-added teacher personnel policies are correct that such policies will terminate many of the wrong teachers. However, we derive the conditions under which such policies will improve student test scores and find that they will almost certainly be met. We then demonstrate that value-added information can also be used to improve student test scores by matching teachers to students or subjects according to their comparative advantage. These matching gains likely exceed those of a feasible, value-added based firing policy.     

Computer spellcheckers and dyslexics—a performance survey

Word-processors are widely used by dyslexics but the spellcheckers are often not appropriate to their needs. This paper examines a sample of around 3000 words of running text containing almost 600 errors typical of those made by dyslexics. It first describes the types of error and the ways in which they differ from the more conventional misspellings that spellcheckers are designed to deal with. It then presents the results of running four current spellcheckers over this sample and finds that the correct word was suggested for only around half of the errors. There were two types of failure: errors that were flagged but not corrected and errors that were not flagged at all. The former often bore a phonetic resemblance to the intended word and in many cases looked correctable. The latter were caused by misspellings that resulted in another valid word. Spellcheckers need to use syntactic and contextual information to improve performance in this area. This would also enable them to make more appropriate suggestions for the errors that are flagged.     

The exchangeability of brief intelligence tests for children with intellectual giftedness: Illuminating error variance components’ influence on IQs

This study examined the exchangeability of total scores (i.e., intelligent quotients [IQs]) from three brief intelligence tests. Tests were administered to 36 children with intellectual giftedness, scored live by one set of primary examiners and later scored by a secondary examiner. For each student, six IQs were calculated, and all 216 values were submitted to a generalizability theory analysis. Despite strong convergent validity and reliability evidence supporting brief IQs, the resulting dependability coefficient was only .80, which indicates relatively low exchangeability across tests and examiners. Although error variance components representing the effects of the examiner, examiner‐by‐examinee interaction, the examiner‐by‐test interaction, and the test contributed little to IQ variability, the component representing the test‐by‐examinee interaction contributed about one‐third of the variance in IQs. These findings hold implications for selecting and interpreting brief intelligence tests and general testing for intellectual giftedness.     

Identifying students' errors in administering the WAIS-R

This study was conducted to investigate specific problems caused by the traditional method of teaching students to administer the Wechsler Adult Intelligence Scale-Revised (WAIS-R). Analysis of 180 protocols provided by 26 graduate students revealed an average of 8.8 mistakes per protocol. When these errors were corrected, 81% of the Full Scale IQs were changed. The types of examiner errors most frequently made by students were identified, as were the subtests and items on which the most numerous mistakes occurred. Students' performance showed no improvement over five practice administrations, but did improve over eight. Even so, students continued to exhibit a high rate of errors. These findings suggest that traditional methods of teaching assessment specialists need to be examined.