Achievement testing with the Wechsler Quicktest: an examination of its psychometric properties and applied utility with a Greek–Cypriot sample

The study aimed to field‐test a Greek version of the Wechsler Quicktest and to examine its psychometric properties. The Quicktest was individually administered to 208 students, aged 5–14 years, along with a reading test. Based on the Rasch analysis, data for the Quicktest subtests showed acceptable fit to the model. Also, correlations were found between the Quicktest subtests. Moreover, the Quicktest was significantly correlated to school grades, teachers’ predictions, scores on the Learning Disability Evaluation Scale (LDES) and Trigka’s Reading Test. A Spearman analysis indicated significant correlations between scores on the Quicktest subtests and the grade level of students. Also, a regression analysis indicated that scores on the Quicktest can predict the scores on the other measures. Results supported the Quicktest’s criterion‐related validity and construct validity and, also, its utility as part of a systematic diagnostic procedure for learning disabilities.     

Using generalizability theory to examine the accuracy and validity of large-scale ESL writing assessment

Using generalizability (G-) theory, this study examined the accuracy and validity of the writing scores assigned to secondary school ESL students in the provincial English examinations in Canada. The major research question that guided this study was: Are there any differences between the accuracy and construct validity of the analytic scores assigned to ESL students and to NE students for the provincial English writing examination across three years? A series of G-studies and decision (D-) studies for three years were conducted to examine accuracy and validity issues. Results showed that differences in score accuracy did exist between ESL and NE students when initial (pre-adjudication) scores were used. The observed G-coefficients for ESL students were significantly lower than those for NE students in all three years, indicating that there were less accuracy and increased errors associated with the writing scores assigned to ESL students. Further, there were significantly less convergent validity in one year and less discriminant validity in all three years of the writing scores assigned to ESL students than to NE students. These findings raise a potential question about the presence of bias in the assessment of ESL students’ writing if initial scores were used.     

Validity of the comprehensive receptive and expressive vocabulary test in assessment of children with speech and learning problems

The current researchers investigated construct, predictive, and differential validity for the Comprehensive Receptive and Expressive Vocabulary Test (CREVT). Participants were 243 public school students, ages 5.5 to 17.25 years. They represented four primary disabilities: Learning Disability (n = 115), Learning Disability with Speech Impairment (n = 29), Mental Retardation (n = 40), and Speech Impairment (n = 59). Adequate construct validity for the CREVT was documented, using the Wechsler Intelligence Test for Children–III as a criterion. Also, the CREVT significantly predicted the scores on the Wide Range Achievement Test–3. Lastly, the CREVT effectively differentiated between students with disabilities. These findings suggest that the CREVT may be helpful in identifying the presence of learning problems. © 2002 Wiley Periodicals, Inc. Psychol Schs 39: 613–619, 2002.     

Predicting College Performance of American Indians: A Large‐Sample Examination of the SAT

Extensive research has examined the validity and fairness of standardized tests in academic admissions. However, due to their underrepresentation in higher education, American Indians have gained much less attention in this research. In the present study, we examined for American Indian students (1) group differences on SAT scores, (2) the predictive and incremental validity of SAT over high school grades, (3) the effect of socioeconomic status on SAT validity, (4) differential prediction in the use of SAT scores, and (5) potential omitted variables that could explain differential prediction for American Indian students. Results provided evidence of predictive and incremental validity of SAT scores, and the validity of SAT scores was largely independent of socioeconomic status. Overprediction was found when using SAT scores to predict college performance and it was reduced when including high school grades as an additional predictor. This study provides substantial evidence of the validity and fairness of SAT scores for American Indians.     

Predictive and construct validity of the developing cognitive abilities test: Relations with the iowa tests of basic skills

This study investigated the predictive and construct validity of the Developing Cognitive Abilities Test in a heterogeneous sample of 863 sixth grade students. Level H of the DCAT was administered during the student's sixth grade year and selected subtests of the Iowa Tests of Basic Skills were administered eight months later during their seventh grade year. Results showed that correlations between the DCAT and Iowa Tests of Basic Skills ranged from .50 (DCAT Spatial‐ITBS Language Usage) to .74 (DCAT Total‐ITBS Mathematics Problem Solving) with a median r= .635. Correlations also supported the construct (convergent) validity of the DCAT when compared to the ITBS with the DCAT Verbal subtest correlating significantly higher with the ITBS Vocabulary, Reading, and Language Usage than either the DCAT Quantitative or Spatial subtests which are not as verbally oriented. The DCAT Quantitative subtest was associated with the ITBS Mathematics Problem Solving to a greater extent than either the DCAT Verbal or Spatial subtests. © 2000 John Wiley & Sons, Inc.     

SCHOLASTIC APTITUDE,MATRICULATION GRADES,AND GROUP INTERVIEW AS PREDICTORS OF PERFORMANCE IN NURSING SCHOOL

This study examines the predictive validity of three commonly used nursing school admission indices, that is, scholastic aptitude test scores, matriculation grades, and evaluations of performance in a group interview situation, in a sample of 321 Israeli nursing school students. Grade point average, supervisor evaluation of clinical internship, and scores on a government certification exam served as primary indices of criterion performance. Whereas composite aptitude test scores correlated moder ately with both grade point average and certification exam scores, matriculation grades correlated negligibly with all three criterion measures. Group interview ratings correlated moderately with clinical performance, but negligibly with the remaining criteria. Aptitude test scores were not found to be biased predictors of criterion performance by ethnicity or social background. The implications of these findings for the selection of nursing school candidates in Israel are discussed.     

Evidence of bias in predictive validity on the Kaufman assessment battery for children in samples of Anglo and Mexican American children

Possible bias in the differential predictive validity of the Kaufman Assessment Battery (K-ABC) was investigated with 76 Anglo and 90 Mexican American fifth- and sixth-grade boys and girls. All children were English-speaking and from similar socioeconomic backgrounds. The criterion variable was the Comprehensive Tests of Basic Skills (CTBS; Language, Reading, Mathematics, and Total Scores). Several statistical techniques were used to investigate test bias (examination of predictive validity coefficients; two methods of examining homogeneity of slopes of the regression lines). The results showed considerable evidence of bias in differential predictive validity, indicating that the global cognitive score of the K-ABC (the Mental Processing Composite) was less effective in predicting CTBS scores for the Mexican American group than for the Anglo group.     

Criterion‐related validity of two curriculum‐based measures of mathematical skill in relation to reading comprehension in secondary students

This study analyzed the relationship between benchmark scores from two curriculum‐based measurement probes in mathematics (M‐CBM) and student performance on a state‐mandated high‐stakes test. Participants were 298 students enrolled in grades 7 and 8 in a rural southeastern school. Specifically, we calculated the criterion‐related and predictive validity of benchmark scores from CBM probes measuring math computation and math reasoning skills. Results of this study suggest that math reasoning probes have strong concurrent and predictive validity. The study also provides evidence that calculation skills, while important, do not have strong predictive strength at the secondary level when a state math assessment is the criterion. When reading comprehension skill is taken into account, math reasoning scores explained the greatest amount of variance in the criterion measure. Computation scores explained less than 5% of the variance in the high‐stakes test, suggesting that it may have limitations as a universal screening measure for secondary students.     

Peer ethnicity and achievement: a meta-analysis into the compositional effect

This study reports a meta-analysis on the effects of ethnic minority share in school on achievement test scores. Best evidence from the studies that have appeared thus far on this topic shows that these compositional effects appear small in general, but may be larger when the ethnic minority group is African Americans in the USA than when the minority group consists of immigrants. A high share of students from an ethnic minority group seems to affect the achievement of students belonging to the same ethnic group more than the achievement of students belonging to the ethnic majority or to other ethnic minority groups. Effects of the share of immigrants on test scores of ethnic majority students even seem to be close to zero. Several robustness checks confirm our results. The review concludes with a discussion of implications for research and policy practice.     

SAT Validity for Linguistic Minorities at the University of California, Santa Barbara

The validity of the SAT as an admissions criterion for Latinos and Asian Americans who are not native English speakers was examined. The analyses, based on 1997 and 1998 UCSB freshmen, focused on the effectiveness of SAT scores and high school grade-point average (HSGPA) in predicting college freshman grade-point average (FGPA). When regression equations were estimated based on all students combined, some systematic prediction errors occurred. For language minorities, using only high school grades as a predictor led to predicted FGPAs that tended to exceed actual FGPAs, particularly for Latinos. Including SAT scores in the equation notably reduced prediction bias. Further analyses showed that, while HSGPA had the highest correlation with FGPA for most groups, SAT verbal score was the strongest predictor of FGPA for language minorities in 1998. An overriding conclusion is that combining data across language groups can obscure important test validity information.     

Relationships between age and raw score increases on the Kaufman-assessment battery for children

Raw scores on the 16 K-ABC subtests and the total raw scores on the sequential and simultaneous processing scales and the achievement scale were correlated with age in months for two separate samples, each subdivided by race and sex: the K-ABC standardization sample (N = 2000) and an additional group of blacks and whites tested during the development of the K-ABC sociocultural norms (N = 615). Within each sample, the highest and lowest correlations from all race/sex groups were contrasted across all K-ABC subtests and scales. All correlations between age and raw scores were statistically significant (p ⩽ .05). No significant differences occurred in the magnitude of these relationships as a function of race/sex grouping, supporting the construct validity of the K-ABC as a developmental measure of children's aptitude and achievement for blacks, whites, Hispanics, males, and females.     

School Psychologists' Sense of Self-Efficacy for Consultation

The Consultation Self-Efficacy Scale (CSES) was developed and validated to assess school psychologists' perceptions of self-efficacy for engaging in school-based consultation. A pilot study with graduate students (N = 92) indicated high internal consistency (α = .99) and provided evidence of discriminant validity, as a group with more consultation experience had significantly higher total self-efficacy scores (M = 512.25) than a moderately experienced group (M = 437.35). Subsequently, a sample of 347 practicing school psychologists completed a revised CSES, consistently endorsing moderate to high levels of consultation self-efficacy (M = 404.08, SD = 51.73). Although consultation self-efficacy was hypothesized to be a multidimensional construct, exploratory factor analysis indicated a single-factor structure. Construct validity of the instrument was supported by significant correlations between school psychologists' consultation self-efficacy ratings and perceptions of their ability to respond to hypothetical consultation referral problems (r = .69, p < .01), and regression analysis found that years of experience and time spent consulting with teachers predicted psychologists' self-efficacy scores.     

Are scholastic aptitude tests in Israel biased towards Arab college student candidates?

This study sets out to examine empirically the cross-cultural validity of the test bias contention as applied to scholastic aptitude testing in the Israeli scene. The analyses were based on the test scores of 1017 Arab and 1778 Jewish student applicants to a major Israeli campus, who were administered standardized scholastic aptitude tests as part of routine precollege admissions procedures. The psychometric properties of four subtests appearing on both the Arabic and Hebrew versions of the University admissions aptitude test battery were compared for Jewish and Arab student candidate subgroups, via a variety of internal (e.g., factor structure, reliability, standard error of measurement, discrimination indices, etc.) as well as external (e.g., predictive validity, standard error of estimate, etc.) criteria. A comparison of the reliability indices, by culture, shows aptitude tests scores to be somewhat less reliable measures for Arab compared to Jewish student candidates. Also, scholastic aptitude test scores reveal significant, but slight, intercept bias, tending to overpredict the scholastic achievement of Arab student candidates. On the whole, however, the data were consistent with the results of previous research carried out in the American cultural scene, reporting negligible differences in construct or predictive test validity across varying cultural groups and the findings appear to be more consistent with the psychometric than with the cultural bias position.     

Factors influencing academic success and retention following a 1st-year post-secondary success course

Research has found that grades are the most valid instruments for predicting educational success. Why grades have better predictive validity than, for example, standardized tests is not yet fully understood. One possible explanation is that grades reflect not only subject-specific knowledge and skills but also individual differences in other aspects. The purpose was to investigate the relative importance of knowledge and skills and other aspects encapsulated in grades for the predictive validity of compulsory school grades for educational success in upper secondary school. Structural equation modelling was used. Participants were 9th-grade students from 3 birth cohorts, each comprising full populations of approximately 100,000 students. The results showed that the subject-specific factors and an additional common grade factor contributed to the predictive validity. Effects of gender and parents' education were found in the common grade factor, with girls and students with a lower educational background being advantaged.     

The Effect of Coaching on the Predictive Validity of Scholastic Aptitude Tests

The present study was designed to examine whether coaching affects the predictive validity and fairness of scholastic aptitude tests. Two randomly allocated groups, coached and uncoached, were compared, and the results revealed that although coaching enhanced scores on the Israeli Psychometric Entrance Test by about 25% of a standard deviation, it did not affect predictive validity and did not create a prediction bias. These results refute claims that coaching reduces predictive validity and creates a bias against the uncoached examinees in predicting the criterion. The results are consistent with the idea that score improvement due to coaching does not result strictly from learning specific skills that are irrelevant to the criterion.     

Do preservice teacher education candidates’ implicit biases of ethnic differences and mindset toward academic ability change over time?

Teachers' implicit biases about ethnic differences in student achievement and teachers' mindsets have been associated with significant differences in their students’ achievement. In two studies (N = 313; N = 57) with preservice teachers undertaking a three-year teacher education programme aimed at promoting social justice, we found that third-year students showed significantly less implicit ethnic achievement bias and reduced fixed mindsets compared to the first-year preservice teachers. Students from the ethnic minority were found to have the least bias, but still associated student achievement more with the ethnic majority group. It is concluded that more is needed to reduce implicit biases and develop a growth mindset among our preservice teachers.     

An analysis of the factor structure of the WISC-R and the effect of color on the coding subtest

The Wechsler Intelligence Scale for Children-Revised was administered to 76 subjects from 8–0 to 16–11 years. All subjects were students from rural school districts in southwest Minnesota, and were referred to a consulting school psychologist for various learning problems. The subtest scores for each subject were factor analyzed, using a principal-factor method. Three factors were found consistent with previous research, although there were mild positive loadings of all subtests except Coding on the two primary factors, suggesting that the test items measure a consistent ability such as Spearman's g. In the second aspect of the study, each subject's results on the old and new color formats of the Coding B subtest were compared. No order effects were found, and there was a highly significant difference in raw score and scaled score favoring the color format. This suggests the possibility of a color enhancement effect, at least with students having learning problems. Caution is advised for interpreting coding scores as an index of distractibility as recommended in the literature, if the subject has color-deficient vision.     

Accuracy of teachers’ tracking decisions: short- and long-term effects of accountability

Bias in teachers’ judgment formation and decision making has long been acknowledged. More specifically, studies have repeatedly demonstrated discrepancies between teacher ratings of minority and majority students with similar academic profiles. Studies have also demonstrated that increasing accountability reduced bias. Little is known, however, about the effect of accountability and bias on the accuracy of decisions. This study investigated the short- and long-term effect of accountability priming on the accuracy of transition decisions. It considered both the extent to which teacher decision accuracy differed for minority and majority students with similar academic profiles (accuracy bias) and differences in levels of confidence for accurate versus erroneous decisions (metacognitive judgment bias). In a longitudinal experimental design, we presented 38 primary school teachers with 9 student vignettes at 3 points in time (baseline, post priming, and 6-month follow-up), varying students’ ethnic background, and asked them to make a school tracking decision for each student. We measured decision accuracy as well as teachers’ level of confidence for each decision. Accuracy and confidence levels were combined to provide two indices of metacognitive judgment accuracy. Results confirmed the hypothesis that accuracy of decisions would improve as a result of increased level of accountability. More specifically, we found that teachers made more accurate decisions after priming, whereby ethnic background differences disappeared. In addition, teachers’ metacognitions varied, whereby after priming decision accuracy was better matched with teachers’ confidence levels. Although accuracy levels were still higher at follow-up than at pre-test, the ethnic bias recurred. This study shows that increased levels of accountability are associated with not only increased decision accuracy but also reduced metacognitive judgment bias, especially in regard to minority students. It also demonstrates accountability may be an effective way of reducing systematic errors in decision making. Findings are discussed in terms of theory and current changes in educational practice.     

The literacy achievement of Norwegian minority students

The article presents the literacy achievement of Norwegian minority students, their reading habits, and their enjoyment of reading based on the Programme for International Student Assessment (PISA) 2000 study. Aspects of their family background and attitudes towards school are related to literacy achievement results. A comparison between Denmark, Sweden, Norway and Germany shows that the achievement gap between majority and minority students is larger in Denmark and Germany than in Norway and Sweden. A more detailed presentation of the Norwegian reading results shows that 35% of the Norwegian minority students perform at a level indicating that they are able to read in a technical sense, but they are unlikely to be able to use reading as an independent tool in acquiring knowledge and skills. The minority students' responses to questions about socio‐economic family background, reading habits, learning strategies and school motivation give a complex picture of their situation in Norwegian schools. The results indicate that there is some potential for equalising differences between minority students and majority students.     

Validating the Watson Glaser Critical Thinking Appraisal

This study validated the Watson Glaser Critical Thinking Appraisal (WGCTA) on a sample of 273 private university students in Lebanon. For that purpose, evidence for construct validation was investigated through identifying the test’s factor structure and subscale total correlations, in addition to differences in scores by gender, different levels, and streams using a series of ANOVA tests. Evidence for the test’s reliability and concurrent validity was also collected. Moreover, students in five courses that emphasize critical thinking were pre–post tested using the WGCTA. Exploratory factor analysis revealed a unidimensional factor while results showed no significant differences in scores on the WGCTA between males and females or between students at different levels except for graduate students in the Business stream. Furthermore, results revealed significant differences for the senior students between different streams. Finally, results showed no statistically significant changes in the mean scores of students who were pre- and post-tested except for nursing students. The test’s significant correlations with various criterion measures provided evidence of its convergent and divergent validities. The results were discussed in terms of the nature of the construct itself, and the effects of training and effective instructional strategies on growth of critical thinking. Recommendations for future research were proposed.