Criterion‐related validity of two curriculum‐based measures of mathematical skill in relation to reading comprehension in secondary students

This study analyzed the relationship between benchmark scores from two curriculum‐based measurement probes in mathematics (M‐CBM) and student performance on a state‐mandated high‐stakes test. Participants were 298 students enrolled in grades 7 and 8 in a rural southeastern school. Specifically, we calculated the criterion‐related and predictive validity of benchmark scores from CBM probes measuring math computation and math reasoning skills. Results of this study suggest that math reasoning probes have strong concurrent and predictive validity. The study also provides evidence that calculation skills, while important, do not have strong predictive strength at the secondary level when a state math assessment is the criterion. When reading comprehension skill is taken into account, math reasoning scores explained the greatest amount of variance in the criterion measure. Computation scores explained less than 5% of the variance in the high‐stakes test, suggesting that it may have limitations as a universal screening measure for secondary students.     

CONSISTENCY AND MAGNITUDE OF DIFFERENCES IN READING CURRICULUM‐BASED MEASUREMENT SLOPES IN BENCHMARK VERSUS STRATEGIC MONITORING

Differences in oral reading curriculum‐based measurement (R‐CBM) slopes based on two commonly used progress monitoring practices in field‐based data were compared in this study. Semester‐specific R‐CBM slopes were calculated for 150 Grade 1 and 2 students who completed benchmark (i.e., 3 R‐CBM probes collected 3 times per year) and strategic (i.e., one R‐CBM probe collected monthly) assessments. Slopes based on two adjacent benchmark assessments were positively correlated with slopes based on three monthly strategic assessments in the spring semester of Grade 1 but not in either Grade 2 semester, and significant differences were found between the slopes in all semesters. Consistent with another study showing that slopes are overestimated when single probes are administered per occasion, slopes were larger when based on strategic versus benchmark data in the current study, and the average discrepancies between slopes were greater‐than‐expected growth rates in all semesters. The current findings, based on field‐based data, illustrate the impact of variations in commonly used progress monitoring procedures on the precision of calculated slope estimates.     

Developing local oral reading fluency cut scores for predicting high‐stakes test performance

This study evaluated the classification accuracy of a second grade oral reading fluency curriculum‐based measure (R‐CBM) in predicting third grade state test performance. It also compared the long‐term classification accuracy of local and publisher‐recommended R‐CBM cut scores. Participants were 266 students who were divided into a calibration sample (n = 170) and two cross‐validation samples (n = 46; n = 50), respectively. Using calibration sample data, local fall, winter, and spring R‐CBM cut scores for predicting students’ state test performance were developed using three methods: discriminant analysis (DA), logistic regression (LR), and receiver operating characteristic curve analysis (ROC). The classification accuracy of local and publisher‐recommended cut scores was evaluated across subsamples. Only DA and ROC produced cut scores that maintained adequate sensitivity (≥.70) across cohorts; however, LR and publisher‐recommended scores had higher levels of specificity and overall correct classification. Implications for developing local cut scores are discussed.     

An Evaluation of the Linearity of Curriculum‐Based Measurement of Oral Reading (CBM‐R) Progress Monitoring Data: Idiographic Considerations

Educators use curriculum‐based measurement of oral reading (CBM‐R) to measure student response to instruction. Current decision‐making frameworks assume students demonstrate linear growth across a school year. However, growth appears nonlinear for at least a subgroup of students. We assessed the degree to which grade two (n = 800) and grade three (n = 800) students receiving intensive interventions experienced discontinuous growth. We also explored when discontinuous growth tended to occur, and whether students improved or regressed afterward. Results indicate linear patterns were adequate for most students (80 percent). Students who showed discontinuous growth early tended to improve afterward. Conversely, students who showed discontinuous growth later tended to plateau. Findings suggest multilevel models may obscure variability in growth patterns. Practice and research implications are discussed.     

Relationship of reading fluency assessment data with state accountability test scores: A longitudinal comparison of grade levels

The No Child Left Behind Act of 2001 mandated statewide accountability testing and focused the accountability conversation on reading. Therefore, the current study examined the relationship between curriculum‐based measurement for reading (R‐CBM) and state accountability test scores, potential grade differences in relationship magnitude, and differences in relationship magnitude among R‐CBM and Maze as they compare to state test scores. Data for 5,472 students in Grades 3, 5, 7, and 8 were correlated and resulted in corrected coefficients that ranged from .51 (eighth graders) to .71 (third graders) for R‐CBM and .49 (eighth graders) to .54 (seventh graders) for Maze. The coefficients between R‐CBM and state test scores were significantly larger for third and fifth graders than those for eighth graders. No significant differences in magnitude were found between the correlation coefficients for state test scores to R‐CBM and to Maze among seventh or eighth graders. Potential implications and suggestions for future research are included. © 2006 Wiley Periodicals, Inc. Psychol Schs 43: 527–535, 2006.     

FORMATIVE,INFORMATIVE, AND SUMMATIVE ASSESSMENT: THE RELATIONSHIP AMONG CURRICULUM‐BASED MEASUREMENT OF READING,CLASSROOM ENGAGEMENT,AND READING PERFORMANCE

This study explored the relationships among formative curriculum‐based measures of reading (CBM‐R), student engagement as an extra‐academic indicator of student motivation, and summative performance on a high‐stakes reading assessment. A diverse sample of third‐, fourth‐, and fifth‐grade students and their teachers responded to questionnaires about student engagement in academic tasks. These questionnaires were collected about the same time as fall CBM‐R oral reading fluency and maze screening tasks. Results indicated that fall student and teacher reports of engagement and a composite score of reading competence derived from CBM‐R screening tests uniquely predicted student performance on year‐end standardized reading tests. Profile analyses indicated that student engagement was associated with better reading performance among students with low competence, suggesting that engagement may be particularly important for increasing student performance for struggling readers. Implications for interventions targeting both student motivation, as well as reading skill development, are discussed.     

USING CURRICULUM‐BASED MEASUREMENT TO EXAMINE SUMMER LEARNING LOSS

Summer loss of reading is a potential factor in maintaining, and potentially widening, the achievement gap. This study used curriculum‐based measurement of reading (R‐CBM) to investigate the effect of the summer on reading. For this study, 317 students in Grades 2 to 5 were assessed in the spring and fall using Dynamic Indicators of Basic Early Literacy Skills Oral Reading Fluency (ORF) measure. Repeated measures analysis of variance modeled overall change in ORF scores by grade, family income, ethnic minority status, English language learner status, and special education (SPED) status. Students in Grades 2 and 3 evidenced overall summer loss, whereas students in Grades 4 and 5 did not exhibit summer loss. In addition, students in Grade 2 showed differential loss based on family income and SPED status. These results support the broadening application of R‐CBM and add to the summer loss literature. Findings are discussed relative to prevention and intervention.     

CBM maze-scores as indicators of reading level and growth for seventh-grade students

The technical adequacy of CBM maze-scores as indicators of reading level and growth for seventh-grade secondary-school students was examined. Participants were 452 Dutch students who completed weekly maze measures over a period of 23 weeks. Criterion measures were school level, dyslexia status, scores and growth on a standardized reading test. Results supported the technical adequacy of maze scores as indicators of reading level and growth. Alternate-form reliability coefficients were significant and intermediate to high. Mean maze scores showed significant increase over time, students’ growth trajectories differed, and students’ initial performance levels (intercepts) and growth rates (slopes) were not correlated. Maze reading level and growth were related to reading level and/or growth on criterion measures. A nonlinear model provided a better fit for the data than a linear model. Implications for use of CBM maze-scores for data-based decision-making are discussed.     

General education teacher judgments of their low‐performing students' short‐term reading progress

This study investigated the accuracy of classroom teachers' judgments of the reading progress of their low‐performing students. Participants were 36 second grade teachers and students in their lowest reading groups (n = 150). Student progress was monitored weekly using reading‐curriculum‐based measurement (R‐CBM) procedures. After 6 weeks, teachers were asked to rate their students' progress. Expert judges later reviewed the teachers' R‐CBM graphs and rated the individual and group progress based on the graphs. Teacher ratings did not correlate with expert ratings or the R‐CBM slope estimates. Expert ratings correlated highly with slope estimates. Teachers' estimates of progress were significantly higher than expert judges' ratings, indicating that teachers may overestimate student progress. Implications for practice and future research are discussed. © 2008 Wiley Periodicals, Inc.     

Monitoring Reading Growth: Goal Setting,Measurement Frequency,and Methods of Evaluation

This study examined effects of goal setting, frequency of progress monitoring, and method of evaluating progress on the number of instructional change prompts generated by reading‐curriculum‐based measurements (CBM). Participants were 31 high‐incidence special education students in grades 2–6 selected from 8 schools. Following baseline measurements, researchers administered grade‐level CBM passages every 2 weeks for 8 weeks. Comparisons included 3 goal levels (0.5, 1.0, or 1.5 word growth per week), 3 monitoring/evaluation schedules (every 2, 4, or 8 weeks), and 2 methods for evaluating progress (points below the aim line or slope). Overall, more ambitious goals, less frequent monitoring, and using slope to evaluate progress generated more instructional change prompts. Results favored less frequent monitoring and using slope to evaluate progress.     

EMPIRICAL SYNTHESIS OF THE EFFECT OF STANDARD ERROR OF MEASUREMENT ON DECISIONS MADE WITHIN BRIEF EXPERIMENTAL ANALYSES OF READING FLUENCY

Intervention researchers often use curriculum‐based measurement of reading fluency (CBM‐R) with a brief experimental analysis (BEA) to identify an effective intervention for individual students. The current study synthesized data from 22 studies that used CBM‐R data within a BEA by computing the standard error of measure (SEM) for the median data point from the baseline and intervention data. The median CBM‐R score from the intervention that the authors of each study identified as most effective fell within the SEM (68% confidence interval) of the baseline data approximately 30% of the time, but the ranges for the two author‐identified most effective interventions overlapped over 75% of the time. Extended analyses were consistent with the BEA results for approximately three‐fourths of the instances after considering the SEM of the baseline and intervention phases. Using matched passages did not improve the overlap of the ranges, but there was less overlap when the study used three data points per condition. Results emphasize the importance of considering SEM of CBM‐R data when comparing interventions within a BEA. Further implications for practice and future research are included.     

AN INVESTIGATION OF RELATIONS AMONG ACADEMIC ENABLERS AND READING OUTCOMES

The current study examined the link between academic enablers and different types of reading achievement measures. Academic enablers are skills and behaviors that support, or enable, students to perform well academically, such as engagement, interpersonal skills, motivation, and study skills. The sample in this study consisted of 61 third‐, fourth‐, and fifth‐grade students (54% male). Academic enablers were rated by classroom teachers via the Academic Competence Evaluation Scales (ACES; DiPerna & Elliott, 2000 ). Four different measures of reading achievement were included: classroom grades, global ratings of reading skills, standardized test scores, and Reading CBM scores. Results indicated that academic enablers were significantly related to each type of reading outcome. Academic enablers accounted for the greatest amount of variance for classroom grades (45%) and the least amount of variance in standardized test scores (11%). Results suggest that academic enablers are an important part of academic success in reading, particularly classroom grades, but when considering the variance accounted for by academic enablers, they alone are not likely to improve Reading CBM scores or standardized test scores.     

Identifying low‐progress readers: Comparing teacher judgment with a curriculum‐based measurement procedure

Identifying low‐progress readers easily and quickly is an essential prerequisite for effective literacy intervention in schools. In this study, teacher judgment of reading performance is compared with a curriculum‐based measurement procedure. This study involved 33 teachers and their Year 3 to Year 5 classes. Twelve students were randomly selected from each class and their teachers were asked to rank them based on their judgments of student reading performance. All students were also assessed on a Passage Reading Test (PRT) based on the principles of curriculum‐based measurement. The obtained oral reading fluency measures for the students were ranked for each class and compared with teacher judgment rankings. The results indicated that only one‐half of the teachers identified the same poorest reader as did the curriculum‐based PRT. Moreover, only 15% of the teachers identified the same 3 lowest performing readers as the PRT. These findings suggest that over‐reliance on teacher judgment for identifying low‐progress readers may be misplaced and that curriculum‐based PRTs may provide a more objective and quick alternative procedure.     

Cross‐language relationship between Spanish and English oral reading fluency among Spanish‐speaking English language learners in bilingual education classrooms

This study examined whether oral reading fluency in a child's first language (Spanish) as assessed by Curriculum‐Based Measurement (CBM) was related to oral reading fluency in a second language (English) and whether Spanish oral reading fluency probes administered in the fall were predictive of English oral reading fluency outcomes for spring of the same academic year. A total of 68 bilingual education students across grades 1 through 5 were assessed in Spanish and English during the fall, winter, and spring. Results showed that reading in Spanish and English across grades and time periods correlated moderately high with the exception of fourth grade. In addition, Spanish oral reading fluency at the beginning of the year significantly predicted English reading outcomes at the end of the year. These findings suggest that CBM can be a valuable tool for evaluating the relationship between oral reading fluency in both the first and second language. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 795–806, 2007.     

Within-year oral reading fluency with CBM: a comparison of models

This study examined the type of growth model that best fit within-year growth in oral reading fluency and between-student differences in growth. Participants were 2,465 students in grades 3–5. Hierarchical linear modeling (HLM) analyses modeled curriculum-based measurement (CBM) oral reading fluency benchmark measures in fall, winter, and spring with grade level and student characteristics (including special education and Limited English Proficiency status) as covariates. Results indicated that a discontinuous growth model fit the data better than a linear growth model, with greater growth in the fall than in the spring. Oral reading fluency growth rates also differed by grade and student characteristics. Implications for school practice and research are discussed.     

Curriculum-based measurement yearly growth rates: An examination of English Language Learners and native English speakers

The use of curriculum-based measurement (CBM) is supported by several decades of research regarding their technical adequacy, practical utility, and use with diverse populations. Questions remain regarding the measurement of growth using tri-annual reading CBM (R-CBM) assessment. Existing research on annual rates of growth is inconclusive with some studies suggesting that the most growth occurs from the fall to the winter and others finding growth accelerates from winter to spring. With a sample of students in third through fifth grades, consisting of a high percentage of English Language Learners (ELLs) and students exited from bilingual instruction, this study found that Non-ELL students demonstrated semester differences in R-CBM growth consistent with prior research in third and fourth grades, but not in fifth grade. However, this pattern was not always consistent for ELLs and students recently exited from bilingual instruction, suggesting that conclusions regarding semester differences in R-CBM growth rates should not be consistently extended to ELLs.     

Parent involvement and reading: Using curriculum‐based measurement to assess the effects of paired reading

This study examined the effectiveness of a paired reading intervention using curriculum‐based measurement (CBM) probes to document reading progress for one fourth‐ and two third‐grade students. These students were considered below‐average readers. Parents and children used the paired reading method in their homes for 10 minutes, four times a week. Children and parents were also provided with reading material at their instructional level. Results showed that the children's reading accuracy and rates increased from baseline to follow‐up on CBM measures and on pre/posttesting. In addition, students and parents gave favorable treatment acceptability ratings for paired reading. Implications for future research are outlined. © 2003 Wiley Periodicals, Inc. Psychol Schs 40: 613–626, 2003.     

Can students run their own interventions?: A self‐administered math fluency intervention

Manipulating the ratio of known to unknown items has been shown to improve student on‐task behavior and increase the desirability of schoolwork. Although many intervention protocols manipulate ratios of known to unknown items, these frequently require extensive adult cuing. School psychologists recommending such interventions may face resistance from teachers who find the work to be too time intensive. Self‐administered interventions whereby the students act as primary interventionists may alleviate this concern. A modified multiple baseline design across a group of 11 students identified by their teachers as having difficulty mastering multiplication facts was used to evaluate a self‐administered folding‐in technique. As hypothesized, for the overall sample, stronger effects were observed on math curriculum‐based measures (CBM) than on a control reading task (Maze CBM). Individually, 7 of the 11 students demonstrated higher levels of growth during the intervention phase than during the baseline phase. No student demonstrated higher Maze CBM growth rates during the intervention phase relative to the baseline phase. In contrast, of the 4 students for whom a link between the intervention and the baseline could not be established, 3 demonstrated higher levels of performance on math CBMs following the intervention phase. Results are discussed in light of a response‐to‐intervention model. © 2012 Wiley Periodicals, Inc.     

Sources of variance in curriculum‐based measures of silent reading

Curriculum‐Based Measurement silent reading (CBM‐SR) items have been found to be reliable and valid for measuring reading comprehension skills This generalizability study reports the findings from administration of three CBM‐SR passages to fifth through eighth grade students in one school district. Using Repeated Measures Analyses of Variance (RMANOVA) procedures, the statistical probability of performance on the CBM‐SR task as a differential indicator of reading comprehension skill was found to be significant among students in different grade levels and between students who did and did not receive special education services. Follow‐up analyses were conducted using generalizability theory to estimate the amount of variance in CBM‐SR scores from individual score differences, grade levels, and special education status. The results indicated that on two of the passages, variability in CBM‐SR scores came primarily from grade level differences in scores on the tasks, while on the third passage, the differences were most attributable to individual differences in scores, regardless of grade level or special education services. Implications for the use of CBM‐SR items for routine assessment of students' reading skills are discussed. © 2003 Wiley Periodicals, Inc. Psychol Schs 40: 363–377, 2003.     

Progress Monitoring in Social Studies Using Vocabulary Matching Curriculum‐Based Measurement

Two hundred and two (n = 202) sixth‐grade students in social studies were administered a weekly vocabulary‐matching curriculum‐based measure (CBM) for 35 weeks. Students were also administered the Scholastic Reading Inventory (SRI), along with the annual state high‐stakes test in Communication Arts. CBM scores were analyzed with respect to alternate form reliability, validity with criterion measures, and student growth over time. Results suggest that the vocabulary‐matching CBM is reliable and valid with the SRI but not with the state test. Students showed an overall linear trend of growth, but this growth was flat in the middle of the semester. Implications for research and practice are discussed.