ASSESSING THE INCREMENTAL VALUE OF KABC‐II LURIA MODEL SCORES IN PREDICTING ACHIEVEMENT: WHAT DO THEY TELL US BEYOND THE MPI?

The current study examined the incremental validity of the Luria interpretive scheme for the Kaufman Assessment Battery for Children‐Second Edition (KABC‐II) for predicting scores on the Kaufman Test of Educational Achievement‐Second Edition (KTEA‐II). All participants were children and adolescents (N = 2,025) drawn from the nationally representative KABC‐II/KTEA‐II linked standardization sample. Consistent with previous studies, the full scale Mental Processing Index (MPI) score accounted for clinically significant portions of KTEA‐II score variance in all of the regression models that were assessed. In contrast, the Luria factor scores collectively failed to provide meaningful incremental predictive variance after controlling for the effects of the MPI. Individually, the factor scores consistently accounted for trivial portions of achievement variance. Potential implications of these results for the correct interpretation of the KABC‐II within clinical practice are discussed.     

DIFFERENTIAL WEIGHTING BY JUDGED DEGREE1OF CORRECTNESS

Two conventional scores and a weighted score on a group test of general intelligence were compared for reliability and predictive validity. One conventional score consisted of the number of correct answers an examinee gave in responding to 69 multiple-choice questions; the other was the formula score obtained by subtracting from the number of correct answers a fraction of the number of wrong answers. A weighted score was obtained by assigning weights to all the response alternatives of all the questions and adding the weights associated with the responses, both correct and incorrect, made by the examinee. The weights were derived from degree-of-correctness judgments of the set of response alternatives to each question. Reliability was estimated using a split-half procedure; predictive validity was estimated from the correlation between test scores and mean school achievement. Both conventional scores were found to be significantly less reliable but significantly more valid than the weighted scores. (The formula scores were neither significantly less reliable nor significantly more valid than number-correct scores.)     

Students classified as LD and the college foreign language requirement: a quantitative analysis

This study was conducted to determine whether students classified as learning disabled (LD) who were permitted to substitute courses for the college foreign language (FL) requirement at one university would display significant cognitive and academic achievement differences when grouped by level of discrepancy between IQ and achievement, by discrepancy between achievement according to different measures, and by level of performance on phonological-orthographic processing measures, on the Modern Language Aptitude Test (MLAT), and in FL courses. Results showed that there were no differences among students with different levels of discrepancy (i.e., < 1.0 SD, 1.0-1.49 SD, and > 1.50 SD) on MLAT and American College Testing (ACT) scores, graduating grade point average (GPA) or college FL GPA. Results also showed that among students who scored below versus at or above the 25th percentile on phonological-orthographic processing measures, there were no differences on measures of IQ, ACT, MLAT, and GPA, as well as most measures of academic achievement. Implications for the use of the LD label to grant FL course substitutions or waivers, use of the MLAT in the diagnostic and course substitution/waiver process, and the validity and reliability of traditional criteria for the classification as LD are discussed.     

IMPROVING THE IDENTIFICATION OF ANXIOUS ELEMENTARY SCHOOL CHILDREN THROUGH THE USE OF AN ADJUSTED ANXIETY SCALE

Anxiety scores (Test Anxiety Scale for Children) from 165 sixth graders were adjusted for defensiveness (Lie Scale for Children) by an equally weighted summation of the two scores. Construct validity of the adjusted anxiety score was markedly superior to that of the uncorrected score, as indicated by an increase in correlation with achievement on a programmed instruction unit of(in one case) -.33 to -.52.     

Effects of Improving the Reliability of the GPA on Prediction Generally and on Comparative Predictions for Gender and Race Particularly

The reliability of a method of adjusting grade point averages for differences in departmental grading standards was examined, as were the effects of such adjustments on the predictive validity of high school grades, SAT scores, and achievement test scores. The index of differential grading standards for all on-time graduates of the Dartmouth College class of 1986 was quite reliable, and its use in adjusting grade averages increased predictive validity, reduced its erosion over years, reduced the apparent underprediction of women, and improved predictions for blacks. Differential group enrollment in courses in the science division seems to account for much of the effect of adjustment on grades. Improvement in the reliability of the criterial grade averages also was shown to have similar effects on gender and race prediction in another data set     

How valid are student perceptions of teaching quality across education systems?

In an international context, teaching quality is often assessed via student surveys. Yet, there is little empirical evidence for the validity of students' perceptions of teaching quality across education systems. Following an argumentative validation approach, we evaluate the reliability, stability, and international comparability of students' perceptions of teaching quality as well as their predictivity on student outcomes. Based on longitudinal data from six education systems our main findings include that student perceptions of teaching quality are 1) reliable on the class level, 2) stable between two measurements, and 3) metric invariant across education systems, yet 4) their effects on achievement, interest, and self-efficacy vary between the aspects of teaching quality and education systems. Our study provides first substantial empirical findings for the validity of student perceptions across education systems and points towards culture-specific differences in the effectiveness of teaching.     

Validity of alternative approaches for the identification of learning disabilities: operationalizing unexpected underachievement

This article reviews the validity of models based on (a) aptitude-achievement discrepancies, (b) low achievement, (c) intraindividual differences, and (d) response to instruction for the classification and identification of learning disabilities (LD). Models based on aptitude-achievement discrepancies and intraindividual differences showed little evidence of discriminant validity. Low achievement models had stronger discriminant validity but do not adequately assess the most significant component of the LD construct, unexpected underachievement. All three of these status models have limited reliability because of their reliance on a measurement at a single time point. Models that incorporate response to instruction have stronger reliability and validity but cannot represent the sole criterion for LD identification. Hybrid models combining low achievement and response to instruction most clearly capture the LD construct and have the most direct relation to instruction. The assessment of students for LD must reflect a stronger underlying classification that takes into account relations with other developmental disorders as well as the reliability and validity of the underlying classification and resultant identification system.     

Evaluation of Procedure-Based Scoring for Hands-On Science Assessment

This article evaluates a procedure-based scoring system for a performance assessment (an observed paper towels investigation) and a notebook surrogate completed by fifth-grade students varying in hands-on science experience. Results suggested interrater reliability of scores for observed performance and notebooks was adequate (>.80) with the reliability of the former higher. In contrast, interrater agreement on procedures was higher for observed hands-on performance (.92) than for notebooks (.66). Moreover, for the notebooks, the reliability of scores and agreement on procedures varied by student experience, but this was not so for observed performance. Both the observed-performance and notebook measures correlated less with traditional ability than did a multiple-choice science achievement test. The correlation between the two performance assessments and the multiple-choice test was only moderate (mean = .46), suggesting that different aspects of science achievement have been measured. Finally, the correlation between the observed-performance scores and the notebook scores was .83, suggesting that notebooks may provide a reasonable, albeit less reliable, surrogate for the observed hands-on performance of students.     

Predicting reading achievement for Kindergarten boys and girls

This study assessed the validity of the Kindergarten Teacher Rating Scale (KTRS) in predicting reading achievement for male and female students. The KTRS was a significant predictor of reading achievement for both boys and girls; differential predictive validity for boys and girls was not found. The KTRS explained about 30% of the variance in reading achievement both at the end of the 1st grade and the beginning of 2nd grade. The proportion of variance in reading achievement explained by variance in KTRS scores was significantly greater than the proportion of variance in reading achievement explained by variance in reading readiness scores. There were no significant differences in the mean KTRS scores for male and female students.     

An analysis of difference score reliabilities on three measures with a sample of low-achieving youngsters

A common practice in the field of learning disabilities is analysis of ability-achievement discrepancies. The reliability of discrepancy scores is an important statistic in such decision making. In this study, selected ability and achievement devices were administered to a sample of low achievers (N = 99), and the reliability of various difference scores was analyzed. In all cases, the reliabilities of difference scores were moderately high. Reliabilities of differences for devices normed on the same population and differences for devices normed on different populations were comparable. These results are discussed in light of current psychometric practices.     

Validity and reliability of the classroom participation questionnaire with deaf and hard of hearing students in public schools

The Classroom Participation Questionnaire (CPQ) was administered to 136 deaf or hard of hearing (D/HH) students attending general education classrooms in Grades 4-10. The CPQ is a student-rated measure that yields scores for Understanding Teachers, Understanding Students, Positive Affect, and Negative Affect. Validity and reliability of a long (28-item) and a short (16-item) form are reported. We provide evidence of (a) internal structure validity through an examination of the relationships between the subscales and an analysis of interitem reliability within each scale, (b) reliability over time by examining the scores of students over a 3-year period, and (c) external structure validity through an examination of the relationships of the CPQ with measures of teacher-rated academic competence and Stanford achievement scores. The results suggest that both the long and short form of the CPQ can be used to assess participation of D/HH students in general education classrooms.     

Exploring the moderating role of well-being on the adaptive link between self-assessment practices and learning achievement

Self-assessment practices can positively impact student learning and achievement. However, less is known about psychological moderators that can influence the impact that self-assessment practice may have on achievement. Integrating the dual processing self-regulation model and the self-assessment cycle framework, this study examined the role of students’ positive well-being in the link between self-assessment practices and domain-specific achievement scores. Hierarchical multiple regression analyses using data from 765 secondary school students revealed that self-assessment practices positively predicted achievement scores. Results also showed that positive well-being moderated the relationship between seeking internal feedback and self-reflection and achievement scores, in that the positive link between such self-assessment practices and achievement was stronger for students who had low or moderate well-being. These findings reinforce the impact of self-assessment on learning and add to the ecological validity of the dual processing self-regulation model. The discussion focuses on how these findings contribute to our understanding of how well-being could influence the adaptive role of internal and cognitive self-assessment practices in student learning. Limitations and future research directions are also discussed.     

Relationships between age and raw score increases on the Kaufman-assessment battery for children

Raw scores on the 16 K-ABC subtests and the total raw scores on the sequential and simultaneous processing scales and the achievement scale were correlated with age in months for two separate samples, each subdivided by race and sex: the K-ABC standardization sample (N = 2000) and an additional group of blacks and whites tested during the development of the K-ABC sociocultural norms (N = 615). Within each sample, the highest and lowest correlations from all race/sex groups were contrasted across all K-ABC subtests and scales. All correlations between age and raw scores were statistically significant (p ⩽ .05). No significant differences occurred in the magnitude of these relationships as a function of race/sex grouping, supporting the construct validity of the K-ABC as a developmental measure of children's aptitude and achievement for blacks, whites, Hispanics, males, and females.     

Validity of the Differential Aptitude Test for the Assessment of Immigrant Children

This paper addresses the construct as well as the criterion validity of the Differential Aptitude Test (DAT) for the assessment of secondary school minority group students ( N = 111) as compared to majority group students ( N = 318) in The Netherlands. Comparison of the test dimensions with the structural equation modelling program EQS showed that construct validity was good for both groups. With one exception, the subtests of the DAT measured the cognitive abilities of minority and majority group students equally well. The estimate of g as computed with the DAT showed strong predictive validity with little bias for various school subjects and achievement tests for mathematics and Dutch. Although some criteria revealed prediction bias to the disadvantage of the minority group, these differences concerned very small changes in R 2 . Conversely, the predictive value decreased substantially when an estimate of g was used excluding subtests that measure aspects of crystallised intelligence. Spearman's hypothesis tested with DAT subtest scores and criterion scores showed that g explained most of the group differences. Professional test users can safely draw conclusions from the DAT regardless of the students' ethnicity.     

A Note on the Problem Solving Style Questionnaire: An alternative to Kolb's Learning Style Inventory?

Given the psychometric limitations of existing measures of Kolb's experiential learning model (ELM), two new scales of learning styles have been developed. The validity of these scales has been supported in samples of undergraduate and MBA students in the USA. This paper provides evidence of some psychometric properties of scores yielded by these scales using samples of undergraduate students in the UK. Only limited support is found for the internal consistency reliability and construct validity of scores produced by the scales. However, an item attrition exercise identifies a two‐factor solution providing an acceptable fit to the data. The scales are reported as being positively correlated with academic performance and prior academic achievement. Despite the mixed evidence, we suggest further development of the scales is warranted to create a psychometrically sound measure of the ELM.     

Development of the nonverbal immediacy scale (NIS): Measures of self‐and other‐perceived nonverbal immediacy

In recent years nonverbal immediacy has received considerable attention from researchers concerned with instructional communication, interpersonal communication, and organizational communication. Unfortunately, the instruments used to measure nonverbal immediacy in these contexts sometimes have been problematic in terms of their reliability estimates. This research attempted to overcome this problem, or failing that, to identify the cause(s) of the reduced reliability. The research resulted in a scale with high reliability when used as either a self‐report or an other‐report measure. It was also found to be equally reliable across the contexts of instructional, interpersonal, and organizational communication. Content validity of the scale is good and an initial test of predictive validity produced a high validity correlation. Unexpected sex differences were observed in the results and these are discussed in this report.     

Examining students’ affective commitment toward country: a case study of a Singapore primary school

The purpose of this study was to examine students’ affective commitment toward Singapore. Affective commitment refers to the sense of attachment to the nation state. The sample was taken from 286 students in a primary school. In the first section of the paper, we described the design of a Likert-type Affective Commitment to Country questionnaire. Factor analyses (principal component analysis and confirmatory factor analysis) showed evidence of construct validity for the 10-item scale, and an overall Cronbach alpha reliability coefficient of 0.91. In the second section, we reported the statistics related to the students’ affective commitment scores. Overall, a positive affective commitment toward the country was found. Results of our t-test analyses revealed that no statistically significant difference was found between boys and girls for each of the questionnaire items. However, students who had higher academic achievement reported significantly higher scores than their lower ability counterparts with regard to six items of the questionnaire. Suggestions for future research are discussed.     

The Impact of Internet Virtual Physics Laboratory Instruction on the Achievement in Physics, Science Process Skills and Computer Attitudes of 10th-Grade Students

The purpose of this study was to investigate and compare the impact of Internet Virtual Physics Laboratory (IVPL) instruction with traditional laboratory instruction in physics academic achievement, performance of science process skills, and computer attitudes of tenth grade students. One-hundred and fifty students from four classes at one private senior high school in Taoyuan Country, Taiwan, R.O.C. were sampled. All four classes contained 75 students who were equally divided into an experimental group and a control group. The pre-test results indicated that the students’ entry-level physics academic achievement, science process skills, and computer attitudes were equal for both groups. On the post-test, the experimental group achieved significantly higher mean scores in physics academic achievement and science process skills. There was no significant difference in computer attitudes between the groups. We concluded that the IVPL had potential to help tenth graders improve their physics academic achievement and science process skills.     

MATCHED PAIR TRUE-FALSE SCORING: EFFECT ON RELIABILITY AND VALIDITY1

The matched pair technique for writing and scoring true-false items was designed to compensate for the acquiescence response set of primary grade children. The claim that this technique increases reliability to an appreciable extent over traditional true-false scoring was investigated by comparing alpha internal consistency coefficients computed for the matched pair true-false, traditional true-false, and three other scoring schemes. Both the total sample coefficients and individual classroom coefficients were computed from the standardization sample of a primary grade economics achievement test (Primary Test of Economic Understanding). Classroom reliability coefficients computed from the matched pair scores were found to be higher than those from scores computed by the other methods. Total sample coefficients obtained from four of the five methods were nearly equal. Evidence of the effects of each scoring technique on concurrent validity is also presented. Contrary to expectations, the correlations of traditional and matched pair scores with Iowa Test of Basic Skills (ITBS) subtests (when adjusted for differing reliabilities) were approximately equal.     

Cultural adaptation,translation and validation of the ISPCAN Child Abuse Screening Tool – Retrospective Version (ICAST-R) for young adults in Sri Lanka

The aim of the present study was to adapt and validate the ISPCAN child abuse screening tool-retrospective version (ICAST-R) in Sri Lanka with a view to investigating the experiences of physical, sexual and emotional abuse during childhood.The adaptation was performed using qualitative research methods with young adults, parents, teachers, and a multidisciplinary group of experts. The translation to Sinhala (the local Sri Lankan dialect) was carried out by a nominal group technique. A multidisciplinary team of experts assessed the Sinhala ICAST-R (SICAST-R) for its content validity. Moreover, acceptability, reliability and construct validity were determined by conducting a validation study among 200 schooling young adults. The principal component analysis (PCA) technique was used to assess the construct validity. Response rates for each item were taken as evidence of acceptability. The internal consistency was assessed by Cronbach’s alpha, and test-retest reliability after two weeks was assessed using Cohen's kappa coefficient.The adaptation of ICAST-R included the introduction of an objective manner by which to measure severity of abuse and the inclusion of a set of questions regarding help-seeking behavior following physical and emotional abusive experiences.The SICAST-R showed adequate content validity and high acceptability, with response rates ranging from 90.3% to 99.5%. The minimum Cohen’s kappa coefficient was 0.76, indicating good test-retest reliability. The internal consistency (Cronbach’s alpha) for the total tool was 0.708, with the three constructs being 0.398, 0.844 and 0.637 for physical, sexual and emotional abuse, respectively. The PCA demonstrated good reproducibility for sexual and emotional abuse with the hypothesized structure.Overall, the SICAST-R showed adequate validity for the assessment of experiences of physical, sexual and emotional abuse during childhood among Sri Lankan young adults.