The use of standard scores in diagnosing learning disabilities: A critique

This paper examines the limitations of standard scores of achievement tests commonly used in diagnosing learning disabilities. The consideration of these limitations is an important factor in attempting to decide whether a marked discrepancy exists between ability and achievemen, a requirement for the diagnosis of learning disabilities under Public Law 94–142. The phrase “standard score scale” is ambiguous because it can refer to both status score scales and developmental score scales. Unfortunately, many school psychologists seem unaware of the distinction between these two types of standard scores and the ramifications of this distinction. Many standardized achievement tests commonly used in the assessment of learning disabilities use status standard scores despite their severe limitations (noncomparability across grade levels and subjects, and failure to reflect changes in variability across grade levels). While developmental standard scores are to be preferred over status standard scores in diagnosing learning disabled children, their value is significantly lowered because they require greater growth for below-average students than for average or above-average students. Moreover, developmental scores are nonequal interval and they assume that subject matter is normally distributed within age or grade groups. Although we recommend the use of developmental standard scores over status standard scores, we urge that they be interpreted cautiously.     

THE CHOICE OF SCALE FOR EDUCATIONAL MEASUREMENT: AN IRT PERSPECTIVE

Two methods of constructing equal-interval scales for educational achievement are discussed: Thurstone's absolute scaling method and Item Response Theory (IRT). Alternative criteria for choosing a scale are contrasted. It is argued that clearer criteria are needed for judging the appropriateness and usefulness of alternative scaling procedures, and more information is needed about the qualities of the different scales that are available. In answer to this second need, some examples are presented of how IRT can be used to examine the properties of scales: It is demonstrated that for observed score scales in common use (i.e., any scores that are influenced by measurement error), (a) systematic errors can be introduced when comparing growth at selected percentiles, and (b) normalizing observed scores will not necessarily produce a scale that is linearly related to an underlying normally distributed true trait.     

Age and developmental level as antecedents of science learning

Over the past decade, developmental theory has occupied a central role in science education instructional theory and empirical research. The purpose of the present study is to quantitatively synthesize studies relating age (or grade) and developmental level to science learning among grade 6-12 students over the 1963-1978 period. Twenty-seven studies were reviewed. Annual increments observed in measures of developmental level were consistent with current theory, and annual increments in cognitive achievement were relatively constant over the grade 4-9 interval. Measures of student ability were found to be better predictors of cognitive achievement than developmental measures; age and grade level were weakly related to developmental level and cognitive achievement, showing only significant correlations across grade levels.     

The interpretation of norm-based scores from individualized testing using the Iowa tests of basic skills

School teachers and administrators are often faced with the dilemma of deciding what level of an achievement test to assign a child whose developmental rate is atypical of his peers. It is not attractive to mismatch the developmental and achievement test level; however, alternative procedures often call for extra testing time. The ITBS has an out-of-level option which allows for a developmental/achievement test level match that does not require additional testing. Since the procedures used by ITBS to assign grade equivalent scores do not take grade level into account, questions have been raised about the interpretation of grade equivalent scores achieved from out-of-level testing. This research addresses the question of the comparability of equal scores on the same test from children in different grades. The results indicate that the scores are comparable and support the assignment of ITBS levels that match the child's developmental level.     

INFORMANT EFFECTS ON BEHAVIORAL AND ACADEMIC ASSOCIATIONS: A LATENT VARIABLE LONGITUDINAL EXAMINATION

Discrepancies among informants’ ratings of a given child's behavior complicate the study of linkages between child behavior and academic achievement. In the current study, we examined the potential moderating effect of informant type on associations between behavior and two types of achievement in a longitudinal growth model that captured children's development from 54 months of age through fifth grade. Latent internalizing and externalizing behavioral constructs, as separately measured by mothers and teachers, were modeled as time‐varying predictors of achievements to capture changes that occur as children progress through different developmental stages. Behavioral ratings obtained by both informants explained largely equivalent levels of reading achievement variance, and teachers’ ratings of child behavior explained more variance in analytic type achievements than did those of mothers.     

Overview of the Scaling Methodology Used in the National Assessment

The National Assessment of Educational Progress (NAEP) uses item response theory (IRT)–based scaling methods to summarize the information in complex data sets. Scale scores are presented as tools for illuminating patterns in the data and for exploiting regularities across patterns of responses to tasks requiring similar skills. In this way, the dominant features of the data are captured. Discussed are the necessity of global scores or more detailed subscores, the creation of developmental scales spanning different age levels, and the use of scale anchoring as a way of interpreting the scales.     

A Comparative Study of the Effects of Recency of Instruction on the Stability of IRT and Conventional Item Parameter Estimates

A potential concern for individuals interested in using item response theory (IRT) with achievement test data is that such tests have been specifically designed to measure content areas related to course curriculum and students taking the tests at different points in their coursework may not constitute samples from the same population. In this study, data were obtained from three administrations of two forms of a Biology achievement test. Data from the newer of the two forms were collected at a spring administration, made up of high school sophomores just completing the Biology course, and at a fall administration, made up mostly of seniors who completed their instruction in the course from 6–18 months prior to the test administration. Data from the older form, already on scale, were collected at only a fall administration, where the sample was comparable to the newer form fall sample. IRT and conventional item difficulty parameter estimates for the common items across the two forms were compared for each of the two form/sample combinations. In addition, conventional and IRT score equatings were performed between the new and old forms for each o f the form sample combinations. Widely disparate results were obtained between the equatings based on the two form/sample combinations. Conclusions are drawn about the use o f both classical test theory and IRT in situations such as that studied, and implications o f the results for achievement test validity are also discussed     

Monitoring Student Achievement for Accountability: The Demonstration of a Model

Abstract

This study demonstrated a procedural model that can be applied by any school to assess, guide, and account for the progress of its students as well as to analyze its own effectiveness. The model uses equivalent achievement tests to monitor student achievement in subject areas at grade levels, between grade levels, and across subgroups of students. Multiple regression analyses of test scores between grades identify factors associated with achievement Using sixth and eighth grade Comprehensive Tests of Basic Skills scores in a matched longitudinal sample of 208 students, the study found small differences in average achievement between boys and girls. Differences between corresponding sixth and eighth grade test means were higher in mathematics than in language. From the sixth grade to the eighth, there was a widening gap in average achievement between high and low I.Q. groups. In multiple regressions of eighth grade test scores on sixth grade measures, I.Q., study skills, and reading were prevalent in the regression equations, but clusters of measures associated with achievement differed between high and low’ LQ. groups. The results of the study have implications for developing and evaluating the achievement of students with varying mental abilities.     

A Comparison of Developmental Scales Based on Thurstone Methods and Item Response Theory

A developmental scale for the North Carolina End-of-Grade Mathematics Tests was created using a subset of identical test forms administered to adjacent grade levels. Thurstone scaling and item response theory (IRT) techniques were employed to analyze the changes in grade distributions across these linked forms.Three variations of Thurstone scaling were examined, one based on Thurstone's 1925 procedure and two based on Thurstone's 1938 procedure. The IRT scaling was implemented using both B i M ain and M ultilog .All methods indicated that average mathematics performance improved from Grade 3 to Grade 8, with similar results for the two IRT analyses and one version of Thurstone's 1938 method.The standard deviations of the IRT scales did not show a consistent pattern across grades, whereas those produced by Thurstone's 1925 procedure generally decreased; one version of the 1938 method exhibited slightly increasing variation with increasing grade level, while the other version displayed inconsistent trends.     

WISC and WISC-R correlates of academic achievement

WISC and WISC-R test results were correlated with achievement test scores and school grades of 36 children who had completed two years of school. Global intelligence estimates from both scales correlated at significant levels with all achievement test measures. Individual subtests from the two scales were unevenly correlated with grades in specific school subjects over both school years. Data suggest that while the two scales may be grossly equivalent as global predictors of school achievement, the individual subtests from the two scales may not correlate equivalently with specific external criteria such as school grades.     

Conflicting Growth Expectations Cannot Both Be Real: A Rejoinder to Yen

Different scaling procedures applied to the same empirical data can result in conflicting growth patterns. To retain standing in the norm group, some developmental scales require low-achieving students to gain more annually than high-achieving students. Thus, for low-achieving students, the growth patterns exhibited by grade equivalent scales may be more reasonable than those for developmental standard score scales.     

Predicting reading achievement for Kindergarten boys and girls

This study assessed the validity of the Kindergarten Teacher Rating Scale (KTRS) in predicting reading achievement for male and female students. The KTRS was a significant predictor of reading achievement for both boys and girls; differential predictive validity for boys and girls was not found. The KTRS explained about 30% of the variance in reading achievement both at the end of the 1st grade and the beginning of 2nd grade. The proportion of variance in reading achievement explained by variance in KTRS scores was significantly greater than the proportion of variance in reading achievement explained by variance in reading readiness scores. There were no significant differences in the mean KTRS scores for male and female students.     

Sociocultural Differences in Examinees' Attitudes Toward Scholastic Ability Exams

The major aim of the present study was threefold: (a) to compare the test attitudes and perceptions o f examinees of varying sociocultural group membership toward verbal and nonverbal standardized ability tests; (b) to determine the degree of covariation between test attitudes and test scores; and (c) to delineate the properties and potential applications of a test attitude or feedback inventory specifically designed to assess examinees' perceptions of key situational variables in the test context. The feedback inventory was administered to a sample of 259 seventh grade students in Israel immediately following standardized group scholastic ability testing procedures. On the whole, few meaningful group differences in test attitudes were observed by social class, ethnicity, or sex. However, a nonverbal test was generally rated more favorably than a verbal test, among varying sociocultural and sex subgroups. Considered together, test attitude scales share a meaningful proportion o f variance with the test score on both verbal and nonverbal tests. However, in view o f the negligible ethnic and social class differences in test attitudes and the nonsignificant interaction between test attitudes and background variables, the data provide little support for the situational bias claim     

IRT Ability Estimates from Customized Achievement Tests Without Representative Content Sampling

This study examines the effects of using item response theory (IRT) ability estimates based on customized tests that were formed by selecting specific content areas from a nationally standardized achievement test. Subsets of items were selected from four different subtests of the Iowa Tests of Basic Skills (Hieronymus, Hoover, & Lindquist, 1985) on the basis of (a) selected content areas (content-customized tests) and (b) a representative sampling of content areas (representative-customized tests). For three of the four tests examined, ability estimates and estimated national percentile ranks based on the content-customized tests in school samples tended to be systematically higher than those based on the full tests. The results of the study suggested that for certain populations, IRT ability estimates and corresponding normative scores on content-customized versions of standardized achievement tests cannot be expected to be equivalent to scores based on the full-length tests.     

Do `Skills Beget Skills'? Evidence on the effect of kindergarten entrance age on the evolution of cognitive and non-cognitive skill gaps in childhood

We use exogenous variation in the skills that children have at the beginning of kindergarten to measure the extent to which “skills beget skills” in this context. Children who are relatively older when they begin kindergarten score higher on measures of cognitive and non-cognitive achievement at the beginning of kindergarten. Their scores on cognitive assessments grow faster during kindergarten and first grade. However, after first grade the scores of younger entrants catch up. We find no evidence that the growth in non-cognitive measures differs between older and younger entrants. Finally, we provide evidence suggesting that schools are not the cause of the younger students’ faster growth after first grade.     

AN INVESTIGATION OF RELATIONS AMONG ACADEMIC ENABLERS AND READING OUTCOMES

The current study examined the link between academic enablers and different types of reading achievement measures. Academic enablers are skills and behaviors that support, or enable, students to perform well academically, such as engagement, interpersonal skills, motivation, and study skills. The sample in this study consisted of 61 third‐, fourth‐, and fifth‐grade students (54% male). Academic enablers were rated by classroom teachers via the Academic Competence Evaluation Scales (ACES; DiPerna & Elliott, 2000 ). Four different measures of reading achievement were included: classroom grades, global ratings of reading skills, standardized test scores, and Reading CBM scores. Results indicated that academic enablers were significantly related to each type of reading outcome. Academic enablers accounted for the greatest amount of variance for classroom grades (45%) and the least amount of variance in standardized test scores (11%). Results suggest that academic enablers are an important part of academic success in reading, particularly classroom grades, but when considering the variance accounted for by academic enablers, they alone are not likely to improve Reading CBM scores or standardized test scores.     

Predicting First‐Grade Math Achievement from Developmental Number Sense Trajectories

Number sense development was tracked from the beginning of kindergarten through the middle of first grade, over six time points. Children (n= 277) were then assessed on general math achievement at the end of first grade. Number sense performance in kindergarten, as well as number sense growth, accounted for 66 percent of the variance in first‐grade math achievement. Background characteristics of income status, gender, age, and reading ability did not add explanatory variance over and above growth in number sense. Even at the beginning of kindergarten, number sense was highly correlated with end of first‐grade math achievement (r= 0.70). Clarifying the observed slope effect, general growth mixture modeling showed that children who started kindergarten with low number sense but made moderate gains by the middle of kindergarten had higher first‐grade math achievement than children who started out with similarly low number sense with flat growth. The majority of children in the low/flat growth class were from low‐income families. The findings indicate that screening early number sense development is useful for identifying children who will face later math difficulties or disabilities.     

Limitations of Standard Scores in Individual Achievement Testing

This paper considers two major problems related to the Identification of learning disabilities with individually administered achievement tests: the appropriateness of standard versus developmental scores for determining severity of discrepancy and the limitations of existing developmental score scales. The paper also examines the characteristics of the developmental score scales of individualized achievement tests commonly used to evaluate learning disabilities.     

Oral Language and Reading in Bilingual Children

This article examines the question: Do lexical, syntactic, fluency, and discourse measures of oral language collected under narrative conditions predict reading achievement both within and across languages for bilingual children? More than 1,500 Spanish–English bilingual children attending kindergarten–third grade participated. Oral narratives were collected in each language along with measures of Passage Comprehension and Word Reading Efficiency. Results indicate that measures of oral language in Spanish predict reading scores in Spanish and that measures of oral language skill in English predict reading scores in English. Cross‐language comparisons revealed that English oral language measures predicted Spanish reading scores and Spanish oral language measures predicted English reading scores beyond the variance accounted for by grade. Results indicate that Spanish and English oral language skills contribute to reading within and across languages.