A preliminary investigation of the concurrent validity of reading comprehension rate: A direct,dynamic measure of reading comprehension

Reading comprehension rate (RCR) is a direct measure of reading skills that may be useful in formatively evaluating students reading beyond the fourth‐grade level. To investigate the concurrent validity of RCR, we correlated RCR, reading comprehension level (RCL), and words correct per minute (WC/M) with the Broad Reading Cluster Scores of the Woodcock‐Johnson III Tests of Achievement (WJ‐III ACH) across 88 students in 4th, 5th, and 10th grades. Results showed that aloud‐RCR was significantly correlated with the WJ‐III ACH scores for 4th‐grade (r = .90; n = 22), 5th‐grade (r = .87; n = 29), and 10th‐grade (r = .65; n = 37) students. Regression analysis specified a one‐predictor model for 4th‐grade students (aloud‐RCR), a two‐predictor model for 5th‐grade students (WC/M and aloud‐RCR), and a one‐predictor model for 10th‐grade students (WC/M). Discussion focuses on directions for future research and applied issues related to RCR probe passage development. © 2007 Wiley Periodicals, Inc. Psychol Schs 44: 373–388, 2007.     

The Development of Scientific Thinking in Elementary School: A Comprehensive Inventory

The development of scientific thinking was assessed in 1,581 second, third, and fourth graders (8‐, 9‐, 10‐year‐olds) based on a conceptual model that posits developmental progression from naïve to more advanced conceptions. Using a 66‐item scale, five components of scientific thinking were addressed, including experimental design, data interpretation, and understanding the nature of science. Unidimensional and multidimensional item response theory analyses supported the instrument's reliability and validity and suggested that the multiple components of scientific thinking form a unitary construct, independent of verbal or reasoning skills. A partial credit model gave evidence for a hierarchical developmental progression. Across each grade transition, advanced conceptions increased while naïve conceptions decreased. Independent effects of intelligence, schooling, and parental education on scientific thinking are discussed.     

Universal Design and Multimethod Approaches to Item Review

Test items undergo multiple iterations of review before states and vendors deem them acceptable to be placed in a live statewide assessment. This article reviews three approaches that can add validity evidence to states' item review processes. The first process is a structured sensitivity review process that focuses on universal design considerations for items. The second method is a series of statistical analyses intended to increase the limited amount of information that can be derived from analyses on low-incidence populations (such as students who are blind, deaf, or have cognitive disabilities). Finally, think aloud methods are described as a method for understanding why particular items might be problematic for students .     

Characteristics of character usage in Chinese Web searching

The use of non-English Web search engines has been prevalent. Given the popularity of Chinese Web searching and the unique characteristics of Chinese language, it is imperative to conduct studies with focuses on the analysis of Chinese Web search queries. In this paper, we report our research on the character usage of Chinese search logs from a Web search engine in Hong Kong. By examining the distribution of search query terms, we found that users tended to use more diversified terms and that the usage of characters in search queries was quite different from the character usage of general online information in Chinese. After studying the Zipf distribution of n-grams with different values of n, we found that the curve of unigram is the most curved one of all while the bigram curve follows the Zipf distribution best, and that the curves of n-grams with larger n (n = 3–6) had similar structures with β-values in the range of 0.66–0.86. The distribution of combined n-grams was also studied. All the analyses are performed on the data both before and after the removal of function terms and incomplete terms and similar findings are revealed. We believe the findings from this study have provided some insights into further research in non-English Web searching and will assist in the design of more effective Chinese Web search engines.     

Evaluating the Power of Latent Growth Curve Models to Detect Individual Differences in Change

We evaluated the statistical power of single-indicator latent growth curve models to detect individual differences in change (variances of latent slopes) as a function of sample size, number of longitudinal measurement occasions, and growth curve reliability. We recommend the 2 degree-of-freedom generalized test assessing loss of fit when both slope-related random effects, the slope variance and intercept-slope covariance, are fixed to 0. Statistical power to detect individual differences in change is low to moderate unless the residual error variance is low, sample size is large, and there are more than four measurement occasions. The generalized test has greater power than a specific test isolating the hypothesis of zero slope variance, except when the true slope variance is close to 0, and has uniformly superior power to a Wald test based on the estimated slope variance.     

A comparative judgement approach to the large-scale assessment of primary writing in England

ABSTRACT

Writing assessment is a key feature of most education systems, yet there are limitations with traditional methods of assessing writing involving rubrics. In contrast, comparative judgement appears to overcome the reliability issues that beset the assessment of performance assessment tasks. The approach presented here extends previous work on comparative judgement by directly involving teachers in a large number of schools in the judging of young pupils’ writing. To ensure quality control the process incorporated a process of ‘anchoring’ that ensured that teachers could not artificially inflate their own pupils’ scores. The approach was used to assess the writing of 55,599 primary pupils in England in 2017–2018. Overall, the results showed that a comparative judgement approach to writing incorporating anchoring shows promise in providing a fair and robust large-scale method to assess writing.