Predictive validity of the K-ABC for exceptional preschoolers

This 5-year follow-up study examined the predictive validity of the Kaufman Assessment Battery for Children (K-ABC) for 39 children identified during preschool as exhibiting language impairment (n = 10), behavior control deficits (n = 13), or normal language and behavioral development (n = 16). Mean age at follow-up was 9.75 years. The results generally supported the predictive validity of the K-ABC (p<.001). However, analyses by group indicated that the relationship between baseline K-ABC and follow-up K-ABC, language, and achievement measures for the language-impaired subjects was weak (p>.05). Although these results must be interpreted cautiously because of the small sample size, they suggest that for young children likely to be referred as at risk for future learning problems, the K-ABC may not be useful for prediction of later cognitive skills. Implications for practitioners are discussed.     

The child sexual behavior inventory: Reliability and validity in a Dutch normative and clinical sample

BackgroundThe Child Sexual Behavior Inventory (CSBI) is one of the few instruments available to assess sexual behavior in children. Because of cross-cultural differences in the definition of normal and atypical sexual behaviors, the CSBI is not generalizable to other countries.ObjectiveThe current study examined psychometric properties of the Dutch version of the CSBI.Participants and settingThe study represents 3206 children from a normative sample, and 932 children from clinical samples.MethodsParents ratings on the CSBI, and questionnaires for psychosocial problems were obtained to assess psychometric properties.ResultsResults showed that Dutch parents interpret more CSBI items as developmentally appropriate than USA parents. Reliability coefficients indicated internal consistency (α = .39–.82), test-retest (r = .86, p < .000) and interrater reliability (Cohen's d =0.47, p >.05). Positive correlations between the CSBI total scale and the subscales demonstrated content validity (r = .32–.96, p < .01). Differences between normative and clinical samples on the CSBI total (ηp² = .01–.65, p < .000), DRSB (ηp² = .00–.03, p = .00–.38) and SASI scales (ηp² =.00–.18, p < .01) indicated construct validity. Within the clinical samples, highest associations were found between the CSBI score and convergent measures (r = .64, p < .000). Lowest associations were found between the CSBI scores and divergent measures r = .14–.54, p = n.s.−<.000.ConclusionsCross-cultural differences in the perception of developmental appropriate sexual behavior underscore the importance of country-specific normative data. Good psychometric properties of the Dutch version of the CSBI were supported by this study.     

Reliability and validity evidence of scores on the Achievement Goal Tendencies Questionnaire in a sample of Spanish students of compulsory secondary education

This study examined the reliability and validity evidence drawn from the scores of the Spanish version of the Achievement Goal Tendencies Questionnaire (AGTQ) using a sample of 2,022 (51.1% boys) Spanish students from grades 7 to 10. Confirmatory factor analysis replicated the correlated three‐factor structure of the AGTQ in this sample: Learning Goals (LG), Social Reinforcement Goals (SRG), and Performance Goals (PG). The AGTQ yields relatively reliable scores for Spanish students. Logistic regression analyses were used to provide additional validity evidence of AGTQ scores on academic success (general, Spanish, and mathematics). Results indicated that general academic success was 15% and 5% more likely every time that scores on LG and PG scales increased 1 point, respectively, although general academic success was 9% less likely every time that scores on SRG scale increased 1 point. These results were similar for Spanish and mathematics. However, the score on the LG scale did not forecast success in both subjects. © 2009 Wiley Periodicals, Inc.     

Reliability and validity of a Turkish version of the DELES

The primary aim of this study was to examine the reliability and validity of a Turkish version of the Distance Education Learning Environment Survey (DELES) in post-secondary distance education. The second aim was to investigate empirically the conceptualisation of the distance education learning environment as a singular latent construct, within a second-order factorial structure that consists of six first-order factors. The DELES is a 34-item instrument for assessing social-psychological perceptions among distance education students. The tested hypotheses concerned key antecedents and consequences of the factorial structure of DELES in the Turkish context using confirmatory factor analyses. Principal component analysis of the Turkish data revealed six main factors of interest, which corresponded to the hypothesised constructs in the original DELES. The data suggested good reliability and validity for the Turkish DELES (DELES-TR) relative to the original version. Further, second-order analyses were conducted with two complementary samples to demonstrate the validity of six dimensions of the DELES-TR and their contributions to an overall factorial structure of social-psychological learning environment in distance education. Initial support for the second-order factorial validity of DELES-TR was found. Results indicate that a dimensional model of DELES-TR with the six first-order factors provided a superior fit to the data than first-order DELES-TR with six factors. Additionally, an investigation was conducted regarding associations between the DELES-TR’s psychosocial scales and a student satisfaction scale. This study is unique in its focus on the learning environment in distance education in the Turkish setting.     

Reliability and validity of a mathematics performance assessment

This study evaluated the reliability and validity of a performance assessment designed to measure students' thinking and reasoning skills in mathematics. The QUASAR Cognitive Assessment Instrument (QCA1) was administered to over 1.700 sixth and seventh grade students of various ethnic backgrounds in six schools that are participating in the QUASAR project. The consistency of students' responses across tasks and the validity for inferences drawn from the scores on the assessment to the more broadly-defined construct domain were examined. The intertask consistency and the dimensionality of the assessment was assessed through the use of polychoric correlations and confirmatory factor analysis, and the generalizability of the derived scores was examined through the use of generalizability theory. The results from the confirmatory factor analysis indicate that a one-factor model fits the data for each of the four QCAI forms. The major findings from the generalizability studies (person x task and person x rater x task) indicate that, for each of the four forms, the person x task variance component accounts for the largest percentage of the total variability and the percentage of variance accounted for by the variance components that include the rater effect is negligible. The variance components that-include the rater effect were negligible. The generalizability and dependability coefficients for the person x task decision studies (n_t, = 9) range from .71-.84. These results indicate that the use of nine tasks may not be adequate for generalizing to the larger domain of mathematics for individual student level scores. The QUASAR project, however, is interested in assessing mathematics achievement at the program level not the student level; therefore, these coefficients are not alarmingly low.     

Long‐term outcome of oral language and phonological awareness intervention with socially disadvantaged preschoolers: the impact on language and literacy

Early intervention aims to prevent poor literacy outcomes associated with social disadvantage. This study examined whether the short‐term positive effect of a preschool classroom‐based oral language and phonological awareness (PA) programme was maintained and transferred to literacy 2 years later. The vocabulary knowledge, grammatical skill, auditory comprehension and reading comprehension of 54 6–7‐year‐old Australian children who attended school in a low‐socioeconomic area were measured. Children's PA abilities were also assessed and are reported elsewhere. There were no significant differences between children who had received intervention in preschool and those who had not, with the entire cohort performing below the average range of the general population. The findings indicated that while generating short‐term positive effects, intervention in preschool did not enhance socially disadvantaged children's language and literacy achievement in the long term.     

Identifying at‐risk preschoolers: The discriminant validity of the differential ability scales

The discriminant validity of the Upper Preschool Level of the Differential Ability Scales (DAS) was studied using 32 at‐risk preschoolers (females = 14, males = 18) with a mean age of 57.28 months (SD = 4.43) and 30 normal preschoolers (females = 13, males = 17) with a mean age of 56.40 months (SD = 6.46). The results indicated that the DAS was an excellent measure to use when trying to differentiate between at‐risk and normal preschoolers. The DAS could reliably identify, better than chance, whether a child was at‐risk or normal. Specifically, the General Conceptual Ability score was found to be approximately 77% accurate in correctly classifying at‐risk from normal preschoolers. In addition, the use of the DAS in an abbreviated format by selecting specific core and/or diagnostic subtests during the preschool screening process was strongly supported. © 1999 John Wiley & Sons, Inc.     

英语口语测试的信度与效度分析

英语口语考试的信度和效度受口试形式、评分标准和考官素质等多方面因素的影响。提高英语口试的效度和信度,需坚持英语口试形式与内容的统一,设计出科学、客观并具有可操作性的评分标准。高信度与效度的英语口语测试对教学具有积极的反拨作用。     

Reliability and validity of the Learning Styles Questionnaire

The reliability and predictive validity of a short form of Honey and Mumford's Learning Styles Questionnaire were examined. Students' learning preferences were measured by self-report ratings on their degree of likings for 19 teaching and learning activities commonly used in higher and professional education. Subjects were 381 second-year undergraduates enrolled in accountancy, engineering and communication programmes at the Hong Kong Polytechnic. The alpha coefficients for the four learning styles scales were quite low, ranging from 0.311 for the Pragmatist scale to 0.421 for the Reflector scale. Factor analysis of the items did not reveal any coherent factor structure congruent with the underlying constructs. However, significant though weak correlations were found between the learning styles scores and the learning preferences of the students as hypothesised.     

Reliability and validity of the WRAT with Mexican-American children

The reliability and validity of the WRAT were investigated with 191 Mexican-American children. Internal consistency reliability coefficients for the WRAT were found to be high and comparable to those reported in the WRAT manual. Correlations between the WRAT subtests and those of the MAT all were significant and suggested moderate to high relationship between these two measures. It was concluded that the WRAT meets minimum requirements of reliability and validity with Mexican-American children.     

Reliability and validity of the adult reading history questionnaire

The reliability and validity of a revised version of Finucci's (1982) Reading History Questionnaire was examined in two adult samples. One sample included 84 adults from an ongoing study of familial dyslexia, and a second sample was composed of parents of 107 children from a longitudinal study of reading development. Internal consistency was demonstrated by Cronbach's alphas of .94 and .92 in the two samples. Test-retest reliability was demonstrated by significant correlations (.87 and .84 in the two samples) over several years between an earlier and revised form of the questionnaire. Validity was demonstrated via (a) correlations between the questionnaire score and reading measures (rs = .57-.70), (b) the results of a discriminant function analysis that used questionnaire scores to predict reading disability diagnosis, and (c) the finding that the questionnaire had substantial incremental validity in predicting reading skill in a hierarchical regression analysis that first entered IQ and SES. These results indicated that the questionnaire is both reliable and valid.     

Reliability and validity evidence for the EARLI literacy probes

The purpose of this study was to examine the reliability and validity of scores from six early literacy probes developed for use with preschool‐age children. The literacy probes were administered to 84 preschoolers at three time points over a 6‐month period. Also, a criterion measure of early literacy skills was administered to a subset of students. Results indicated that the probes demonstrate adequate reliability for screening decisions, moderate to strong concurrent relationships with a criterion measure, and developmentally appropriate changes in children's early literacy skills over time. These results add to the growing body of evidence supporting the technical adequacy of general outcome measurement for use in monitoring the progress of preschoolers' academic skill development. © 2009 Wiley Periodicals, Inc.     

Reliability and concurrent validity of the DUSO affectivity assessment device

Forms M and L of The DUSO Affectivity Assessment Device were administered to 23 first-grade children. Alternate form reliability was significant at the .02 level of confidence. Forty-five first-grade children were administered Form L of The DUSO AAD and one week later the California Test of Personality. Concurrent validity correlations were significant for the total group and for subgroups differentiated by sex.     

Reliability and concurrent validity of light's retention scale

Light's Retention Scale was completed for 123 candidates to repeat the first grade. These same first-grade children were tested with a number of measures of maturity, including achievement tests, measures of intellectual functioning, visual-motor integration, self-concept, and physical maturity. Results indicated that Light's total score is not sufficiently reliable and has little concurrent validity for making retention decisions. The total score should never be used.     

Reliability and validity of the Math Essential Skill Screener—Elementary Version (MESS-E)

The Math Essential Skill Screener–Elementary Version (MESS-E) is a screener devised to identify primary grade students at risk for math difficulties. Item analysis, interitem consistency, test–retest reliability, decision efficiency, and construct validity of the MESS-E were studied using four independent samples of boys and girls grades 1–3 (aged 6–8). Item analysis revealed median item difficulty of .64 and median item discrimination of .75. Interitem consistency was .92 (n = 171) and .94 (n = 711), while 30-day test–retest reliability was .86 (n = 125). Exploratory factor analysis indicated a one-factor solution accounting for 37% of observed variance. LISREL 7 confirmatory factor analysis procedures determined that the one-factor model fit the standardization sample data poorly (goodness-of-fit index = .729, χ² to df ratio = 9.91). The MESS-E yielded concurrent validity coefficients (n = 171) of .74 with the Woodcock–Johnson: Tests of Achievement–Revised (WJ-R) Math Cluster, .80 with the Wide-Range Achievement Test–Revised (WRAT-R) Arithmetic subtest and .73 with the KeyMath-R Operations Area standard scores. A diagnostic efficiency study yielded a total predictive value (TPV) of .93, sensitivity = .98, specificity = .88, positive predictive power (PPP) = .89, negative predictive power (NPP) = .98, and incremental validity = 39%. The MESS-E displayed a slight tendency to overidentify children potentially at risk for math difficulties. © 1998 John Wiley & Sons, Inc.     

Reliability and concurrent validity of a measure of critical thinking skills in biology

The present study investigated the utility of 52 items, selected from a readily available item pool developed for instructional purposes, when the items are used to measure critical thinking abilities of biology students. The items yield scores that have reasonable internal consistency reliability. Furthermore, analyses involving ACT, Watson-Glaser Critical Thinking Appraisal, and Group Embedded Figures Test scores also suggest that the critical thinking test items have good concurrent validity. Thus, the measure may be useful in both science instruction and future research regarding critical thinking phenomena.     

Self‐report on the social skills rating system: Analysis of reliability and validity for an elementary sample

The Social Skills Rating System (SSRS; F.M. Gresham & S.N. Elliott, 1990) is a norm‐referenced measure of students' social and problem behaviors. Since its release, much of the published reliability and validity evidence for the SSRS has focused primarily on the Teacher Report Form. The purpose of this study was to explore reliability and validity evidence of scores on the SSRS‐Student Elementary Form (SSRS‐SEF) for children in Grades 3 to 5. Findings provided support for the use of Total scale as a measure of student social behavior for initial screening purposes; however, evidence for the subscales was not as strong as predicted. Directions for future research regarding reliability and validity of scores from the SSRS‐SEF are discussed. © 2005 Wiley Periodicals, Inc. Psychol Schs 42: 345–354, 2005.     

Reliability and validity of rubrics for assessment through writing

This experimental project investigated the reliability and validity of rubrics in assessment of students’ written responses to a social science “writing prompt”. The participants were asked to grade one of the two samples of writing assuming it was written by a graduate student. In fact both samples were prepared by the authors. The first sample was well written in terms of sentence structure, spelling, grammar, and punctuation; however, the author did not fully answer the question. The second sample fully answered each part of the question, but included multiple errors in structure, spelling, grammar and punctuation. In the first experiment, the first sample was assessed by participants once without a rubric and once with a rubric. In the second experiment, the second sample was assessed by participants once without a rubric and once with a rubric. The results showed that raters were significantly influenced by mechanical characteristics of students’ writing rather than the content even when they used a rubric. Study results also indicated that using rubrics may not improve the reliability or validity of assessment if raters are not well trained on how to design and employ them effectively.     

博士学位论文评审指标体系可靠吗——基于全数据的信度和效度研究

论文评审是学位论文质量评价的重要环节。为了使评审结果能真实地反映博士学位论文的质量和水平,必须首先保证论文评审指标体系的准确性、可靠性和有效性。信度和效度分析是验证指标体系准确性、可靠性和有效性的重要方法。文章利用北京师范大学五年博士学位论文匿名评审全数据量化结果,对评审指标体系的信度和效度进行实证研究。结果表明,目前广泛采用的评审指标体系具有良好的信度和效度,能够真实地反映博士学位论文的质量和水平。