Validity of the comprehensive receptive and expressive vocabulary test in assessment of children with speech and learning problems

The current researchers investigated construct, predictive, and differential validity for the Comprehensive Receptive and Expressive Vocabulary Test (CREVT). Participants were 243 public school students, ages 5.5 to 17.25 years. They represented four primary disabilities: Learning Disability (n = 115), Learning Disability with Speech Impairment (n = 29), Mental Retardation (n = 40), and Speech Impairment (n = 59). Adequate construct validity for the CREVT was documented, using the Wechsler Intelligence Test for Children–III as a criterion. Also, the CREVT significantly predicted the scores on the Wide Range Achievement Test–3. Lastly, the CREVT effectively differentiated between students with disabilities. These findings suggest that the CREVT may be helpful in identifying the presence of learning problems. © 2002 Wiley Periodicals, Inc. Psychol Schs 39: 613–619, 2002.     

Cattell‐Horn‐Carroll broad cognitive ability profiles of low math achievers

This study extends previous research examining the relations between Cattell‐Horn‐Carroll cognitive abilities and math achievement. The cognitive profiles of children with normative weaknesses in Math Calculation Skills or Math Reasoning were compared to those of their average‐achieving peers. The cognitive profile of the low Math Calculation Skills group (n = 68) was similar to that of their average‐achieving peers. The low Math Reasoning group (n = 52) scored lower than their average‐achieving peers on the cognitive abilities as a set and on Fluid Reasoning and Comprehension–Knowledge. When individual profiles were considered, approximately half of the children with normative math weaknesses demonstrated commensurate weaknesses in one or more cognitive abilities, which may inform diagnostic models of learning disabilities. © 2005 Wiley Periodicals, Inc. Psychol Schs 42: 1–12, 2005.     

Measurement characteristics of the finding embedded figures test

The study investigated the measurement characteristics of both forms of a multiple choice measure of field independence: the Finding Embedded Figures Test. Subjects (n = 302) completed both forms of the FEFT or one form of the FEFT and the Group Embedded Figures Test. Item performance was evaluated by computing classical item difficulty and item-by-total-score correlation coefficients; FEFT-by-GEFT concurrent validity correlation coefficients; and test-retest and alpha reliability coefficients. Results suggest that the FEFT forms provide reasonably reliable and valid data.     

Item Response Theory analysis and Differential Item Functioning across age,gender and country of a short form of the Advanced Progressive Matrices

Item Response Theory (IRT) models were applied to investigate the psychometric properties of the Arthur and Day's Advanced Progressive Matrices-Short Form (APM-SF; 1994) [Arthur and Day (1994). Development of a short form for the Raven Advanced Progressive Matrices test. Educational and Psychological Measurement, 54, 395–403] in order to test if the scale is a reliable and valid tool to assess general fluid ability in a short time frame. The APM-SF was administered to 2264 high-school and university students. Once attested the one-factor structure of the scale, unidimensional IRT analyses for dichotomous data were applied to investigate the increases in item difficulty levels, Test Information Function, and Differential Item Functioning across age, gender, and country (comparing Italian and British respondents). Additionally, validity measures were reported. Findings attest that the Arthur and Day's APM-SF is a sound instrument for assessing fluid ability within a short time frame.     

Measuring Elementary Student’s Mathematics Motivation: A Validity Study

The psychometric properties of a 10-item math motivation scale were empirically validated with an independent sample consisting of 182 elementary-school students. Analysis of the model dimensionality supported a one-factor structure fit. Item parameter estimates from a Classical Test Theory framework revealed that most items were highly discriminating, and the survey is informative for students of low to average math motivation. Differential item functioning (DIF) analyses found two items exhibiting gender bias. Overall, the instrument was a psychometrically valid instrument for measuring math motivation at the elementary level.     

Development of the self-regulated learning teacher belief scale

The present study describes the development and psychometric properties of the Self-Regulated Learning Teacher Belief Scale (SRLTB). The SRLTB is a self-report teacher scale with 10 items assessing teachers' beliefs about introducing self-regulated learning (SRL) in primary education. The process of item and scale development as well as testing and scale refinement procedure is presented. An explorative study (n=399) revealed a one-factor structure representing adherence of teachers for SRL in primary school. Next, Rasch analysis revealed good fit of the scale to the unidimensional continuum model. In a following study (n=553), construct validity of the SRLTB was confirmed. Finally, implications and limitations of the SRLTB for studying SRL are discussed. In general, the SRLTB appears to be a useful instrument for examining teacher beliefs about self-regulated learning practices in primary schools.     

Concurrent validity of the revised Peabody Picture Vocabulary Test

In this investigation, the relationships between the Revised Peabody Picture Vocabulary Test, the Peabody Individual Achievement Test, and McCarthy Scales of Children's Abilities were explored. The sample included 26 children randomly selected from three classes in an elementary school. The Revised Peabody Picture Vocabulary Test correlated positively and significantly with the Peabody Individual Achievement Test's total score (.53) and most of its subtests. The Revised Peabody Picture Vocabulary Test also correlated significantly with the Verbal, Quantitative, Memory, and General Cognitive Indexes of the McCarthy Scales (Median r = .76). The mean Peabody Picture Vocabulary Test score was 104.6 and nearly identical to the McCarthy General Cognitive Index (104.4) but significantly smaller than the Peabody Individual Achievement total test score (114.5).     

The concurrent validity of the peabody individual achievement test relative to the KeyMath diagnostic arithmetic test among adolescents

To investigate the concurrent validity of the Peabody Individual Achievement Test and the KeyMath Diagnostic Arithmetic Test, both instruments were administered to 171 adolescents. Math scores on the Peabody Individual Achievement Test were correlated with all scores on the KeyMath Diagnostic Arithmetic Test. Partial correlation coefficients were computed according to age, sex, race, IQ, and total group. The presence of relatively high and positive partial correlations between the scores supported the concurrent validity of the instruments.     

An empirical analysis of an alternative configuration of the Academic Motivation Scale

The purpose of this study was to examine an alternative configuration of the Academic Motivation Scale (AMS), a psychometric instrument used as a measure of academic motivation in various academic environments. The analyses utilised data from a convenience sample of 2354 business students, broken into two random subsamples of 1177 cases. Exploratory factor analysis of the 28 AMS scale items was conducted on the estimation sample. The results indicated good model fit for a four-factor configuration consisting of amotivation, external regulation, identified regulation, and intrinsic motivation as indicated by factor loadings, as well as internal consistency and reliability statistics.

Confirmatory factor analysis was conducted on the holdout sample to independently assess the construct validity and internal consistency of the item loadings on the reconfigured scale. Results indicated good model fit for the four-factor configuration and a significant loss of fit for competing three- and one-factor models. The uncovered factor structure advances our understanding of how the items on this scale cluster into theoretically meaningful constructs. This knowledge may be utilised in research designed to further assess the impact of motivational states on educational outcomes such as academic performance, absenteeism, dropout rates, etc.     

Reading comprehension test item difficulty as a function of cognitive processing variables.

This study reports the results of a componential analysis of items comprising Sections A and C of Form Z of the reading comprehension portions of the California Achievement Tests (CAT) (Tiegs & Clark, 1963). A set of problem components or attributes characterizing the test items in terms of manifest content, psychologically salient features, and processing demands was developed, including methods for their quantification. The contributions of these components to task difficulty were then evaluated using linear regression methodology. Item difficulty indices were transformations of the familiar proportion-correct item score, obtained from data gathered during the spring of 1989 from 158 deaf examinees. Variation in the item difficulty values was substantially accounted for in terms of a small number of predictor variables (R2 greater than or equal to .90). Implications of the results for construct validity and interpretation of test scores are discussed.     

Predictive validity of the K-ABC for exceptional preschoolers

This 5-year follow-up study examined the predictive validity of the Kaufman Assessment Battery for Children (K-ABC) for 39 children identified during preschool as exhibiting language impairment (n = 10), behavior control deficits (n = 13), or normal language and behavioral development (n = 16). Mean age at follow-up was 9.75 years. The results generally supported the predictive validity of the K-ABC (p<.001). However, analyses by group indicated that the relationship between baseline K-ABC and follow-up K-ABC, language, and achievement measures for the language-impaired subjects was weak (p>.05). Although these results must be interpreted cautiously because of the small sample size, they suggest that for young children likely to be referred as at risk for future learning problems, the K-ABC may not be useful for prediction of later cognitive skills. Implications for practitioners are discussed.     

Cross-Domain Trajectories of Students’ Ability Self-Concepts and Intrinsic Values in Math and Language Arts

Different cross-domain trajectories in the development of students’ ability self-concepts (ASCs) and their intrinsic valuing of math and language arts were examined in a cross-sequential study spanning Grades 1 through 12 (n = 1,069). Growth mixture modeling analyses identified a Moderate Math Decline/Stable High Language Arts class and a Moderate Math Decline/Strong Language Arts Decline class for students’ ASC trajectories. Students’ intrinsic value trajectories included a Strong Math Decline/Language Arts Decline Leveling Off, a Moderate Math Decline/Strong Language Arts Decline, and a Stable Math and Language Arts Trajectories class. These classes differed with regard to student characteristics, including gender, family background, and math and reading aptitudes. They also resulted in different high school math course enrollments, career aspirations, and adult careers.     

Technical characteristics of the peabody individual achievement test as a function of item arrangement and basal and ceiling rules

The effects that item order and basal and ceiling rules have on test means, variances, and internal consistency estimates for the PIAT mathematics and reading recognition subtests were examined. Seven items on the math subtest and one item on the reading recognition subtest were significantly easier or harder than their test placement indicated. The use of basal and ceiling rules had a pronounced effect on the means, variances, and reliabilities on the multiple choice math subtest, while the rules' effects on the reading recognition subtests were minor. Item order also affected scores on the math subtest.     

Evaluating Playfulness: Construct Validity of the Children's Playfulness Scale

This study was designed to examine the underlying structure of the Children's Playfulness Scale (CPS). The CPS was administered to 602 children who were randomly divided into two groups (calibration and validation group). The calibration group (n= 279) included 137 boys and 142 girls, and the validation group (n= 323) included 162 boys and 161 girls, ranging in age from 4 to 6 years. A one-factor model was postulated and supported. According to the model, 5 variables measuring children's playfulness loaded on one factor (playfulness). In addition, the proposed model was found to be invariant across the two groups. Good cross-generalizability of the CPS appears to support its validity. Educators working in a preschool/kindergarten setting may use it with confidence when evaluating children's playfulness.     

Assessing learning: How can classroom-based teachers assess students’ competencies in numeracy?

This paper examines a procedure measuring student competencies in numeracy using school-based assessments, and demonstrates how the procedure informs the school system on quality improvement. The sample consisted of 7648 students, attending three different types of urban schools including government, formal private and low cost private in poor informal settlements in Kenya. The numeracy tool measures six curriculum outcome areas. Each outcome area is defined by 2–11 measured items based on the emphasis given to an area in the curriculum. Within each of the outcome areas, items are also hypothesised to measure categories of the cognitive domain of learning. Using the Rasch, we examine the item difficulties, person location and differential item functioning. The paper provides important implications to provision of evidence to inform teaching in poor urban informal settlements in Kenya and this can also be replicated in other sub-Saharan African countries.

Abbreviations: APHRC: African Population and Health Research Center; CTT: Classical Test Theory; DIF: Differential Item Functioning; ICC: Item Characteristics Curve; INMS: Infit Mean Square; IRT: Item Response Theory; PASEC: Programme d’Analyse des Systèmes Educatifs de la Confemen/ The Programme for the Analysis of Education Systems; QUEST: The interactive test analysis system (software); RUMM: Rasch Unidimensional Measurement Models (software); SACMEQ: Southern and Eastern African Consortium for Monitoring Education Quality; SSA: Sub-Saharan Africa     

Using educational placement in third grade to select and validate a preschool screening measure

The early detection of children with mild learning problems remains a problem. New screening tests are being developed that will be more effective in identifying children who fall into this “difficult to identify” group. This study examined the predictive validity of one potential screening test. The scores of 268 children on each task of a nine‐task preschool cognitive battery were evaluated in terms of the accuracy with which they predicted the classification of those same children into regular education (n = 254) or special education (n = 14) 4 years later when the children were in third grade. A classification accuracy level of 79% was achieved for the exceptional children, and a classification level of 70% for the normally achieving children. Racial/ethnic status, testing language, and socioeconomic status all impacted the children's screening performance. Lower screening scores and a greater proportion of the sample with scores below the cut were associated with minority status, not being tested in English, and lower socioeconomic status. Preschool testing in Spanish or both English and Spanish was associated with poorer achievement 4 years later. Many of the children not in special education, but with a screening score below 13, had serious academic problems in third grade that were reflected in their low percentile scores on one or more subtests of the Stanford Achievement Test. © 2003 Wiley Periodicals, Inc. Psychol Schs 40: 565–582, 2003.     

Stability and correlates of student evaluations of teaching at a Chinese university

This paper examines the stability and validity of a student evaluations of teaching (SET) instrument used by the administration at a university in the PR China. The SET scores for two semesters of courses taught by 435 teachers were collected. Total 388 teachers (170 males and 218 females) were also invited to fill out the 60‐item NEO Five‐Factor Inventory together with a demographic information questionnaire. The SET responses were found to have very high internal consistency and confirmatory factor analysis supported a one‐factor solution. The SET re‐test correlations were .62 for both the teachers who taught the same course (n = 234) and those who taught a different course in the second semester (n = 201). Linguistics teachers received higher SET scores than either social science or humanities or science and technology teachers. Student ratings were significantly related to Neuroticism and Extraversion. Regression results showed that the Big‐Five personality traits as a group explained only 2.6% of the total variance of student ratings and academic discipline explained 12.7% of the total variance of student ratings. Overall the stability and validity of SET was supported and future uses of SET scores in the PR China are discussed.     

Self-Efficacy Beliefs and Mathematical Problem-Solving of Gifted Students

Path analysis was used to test the predictive and mediational role that self-efficacy beliefs play in the mathematical problem-solving of middle school gifted students (n= 66) mainstreamed with regular education students (n= 232) in algebra classes. Self-efficacy of gifted students made an independent contribution to the prediction of problem-solving in a model that controlled for the effects of math anxiety, cognitive ability, mathematics GPA, self-efficacy for self-regulated learning, and sex. Gifted girls surpassed gifted boys in performance but did not differ in self-efficacy. Gifted students reported higher math self-efficacy and self-efficacy for self-regulated learning as well as lower math anxiety than did regular education students. Although most students were overconfident about their capabilities, gifted students had more accurate self-perceptions and gifted girls were biased toward underconfidence. Results support the hypothesized role of self-efficacy in A. Bandura's (1986) social cognitive theory.     

Psychometric defensibility of the Intervention Selection Profile‐Social Skills (ISP‐SS) with students at risk for behavioral concerns

The purpose of the current study was to examine the validity and diagnostic accuracy of the Intervention Selection Profile—Social Skills (ISP‐SS), a brief social skills assessment tool intended for use with students in need of Tier 2 intervention. Participants included 160 elementary and middle school students who had been identified through universal screening as at risk for behavioral concerns. Teacher participants ( n = 71) rated each of these students using both the ISP‐SS and the Social Skills Improvement System—Rating Scales (SSiS‐RS), with the latter measure serving as the criterion within validity and diagnostic accuracy analyses. Confirmatory factor analysis supported ISP‐SS structural validity, indicating ISP‐SS items broadly conformed to a single “Social Skills” factor. Follow‐up analyses suggested ISP‐SS broad scale scores demonstrated adequate internal consistency reliability, with hierarchical omega coefficient equal to 0.86. Correlational analyses supported the concurrent validity of ISP‐SS items, finding each ISP‐SS item to be moderately or highly related to its corresponding SSiS‐RS subscale. Finally, analyses indicated three of the seven ISP‐SS items that demonstrated sufficient diagnostic accuracy; however, findings suggest additional revisions are needed if the ISP‐SS is to be appropriate for use in schools. Implications for practice and future research are discussed.