Improving Measurement for Graduate Admissions

In this study we examined alternative item types and section configurations for improving the discriminant and convergent validity of the GRE General Test. A computer-based test of reasoning items and a generating-explanations measure was administered to a sample of 388 examinees who previously had taken the General Test. Confirmatory factor analyses indicated that three dimensions of reasoning—verbal, analytical, and quantitative—and a fourth dimension of verbal fluency based on the generating-explanations task could be distinguished. Notably, generating explanations was as distinct from new variations of reasoning items as it was from verbal and quantitative reasoning. In the full sample, this differentiation was evident in relation to such external criteria as undergraduate grade point average (UGPA), self-reported accomplishments, and a measure of ideational fluency, with generating explanations relating uniquely to aesthetic and linguistic accomplishments and to ideational fluency. For the subset of participants with undergraduate majors in the humanities and social sciences, generating explanations added to the relationship with UGPA over that contributed by the General Test.     

Consistency across methods of observativn-an in-depth study of the cognitive preference test

If the same constructs embedded in different tests result in parallel or identical score patterns and high intercorrelations, this can be taken as evidence of construct validity. If results do not converge across instruments and/or response formats, this can be taken as evidence of lack of construct validity and/or impurity of the test as indicator of the constructs. In this study two response formats as well as a request for reasons-for-choices of the traditional Cognitive Preference Test (CPT), and an association (open-ended) CPT, were used in order to test for consistency across methods of observation on both the individual and the populational levels. Convergence of results was found to be minimal. None of the hypotheses was confirmed. It was concluded that construct validation of CPT constructs had not yet reached the state of unequivocality necessary for their application in curriculum research.     

Sensitivity to Visual-Tactile Colocation on the Body Prior to Skilled Reaching in Early Infancy

Two experiments examined perceptual colocation of visual and tactile stimuli in young infants. Experiment 1 compared 4- (n = 15) and 6-month-old (n = 12) infants’ visual preferences for visual-tactile stimulus pairs presented across the same or different feet. The 4- and 6-month-olds showed, respectively, preferences for colocated and noncolocated conditions, demonstrating sensitivity to visual-tactile colocation on their feet. This extends previous findings of visual-tactile perceptual colocation on the hands in older infants. Control conditions excluded the possibility that both 6- (Experiment 1), and 4-month-olds (Experiment 2, n = 12) perceived colocation on the basis of an undifferentiated supramodal coding of spatial distance between stimuli. Bimodal perception of visual-tactile colocation is available by 4 months of age, that is, prior to the development of skilled reaching.     

Reward and ideational fluency in preschool children

This study examined the effects of rewards on the ideational fluency of 75 preschool children. Assigned to a reward or nonreward condition, the children were administered two ideational fluency tasks, Unusual Uses and Pattern Meanings. Analyses revealed a significant main effect for reward on the ideational fluency components of originality, total fluency, and flexibility. In all cases the rewarded subjects scored lower than the nonrewarded group. These findings support the growing body of evidence that rewards are detrimental to creative functioning.     

Oral language facility,verbal intelligence and performance on tests of psycholinguistic ability

The performance of 152 children in the age range 7 years 5 months to 7 years 11 months on a battery comprising the Illinois Test of Psycholinguistic Abilities (ITPA), the English Picture Vocabulary Test 2 (EPVT), a test of auditory discrimination, a sentence repetition test and an orally administered verbal intelligence test was compared with the complexity and fluency of spoken language recorded from each child. Only a moderate correlation was found between the ITPA total score and the length‐complexity index (LCI) from the language samples. Moderate correlations were found between the LCI and several auditory‐vocal channel subtests from the ITPA; however, with the influence of verbal intelligence partialled out most of these were reduced to low or nonsignificant levels. The scores from the Manual Expression subtest in the visual‐motor channel were found to correlate as highly with LCI as did the ITPA total score.

This relationship was maintained to two situations, (i) when the factor of verbal intelligence was controlled statistically, (ii) when a sub‐sample of subjects within a narrow IQ range was selected for separate analysis. The Verbal Expression subtest also showed moderate correlation with LCI in four out of five analyses, thus lending support to the “process” construct in the ITPA model. No connection was found between any of the psycholinguistic subtests and fluency of language production. The validity of the ITPA as a test of oral language performance was questioned.     

Generalizability, Validity, and Examinee Perceptions of a Computer-Delivered Formulating-Hypotheses Test

The Formulating-Hypotheses (F-H) item presents a situation and asks examinees to generate as many explanations for it as possible. This study examined the generalizability, validity, and examinee perceptions of a computer-delivered version of the task. Eight F-H questions were administered to 192 graduate students. Half of the items restricted examinees to 7 words per explanation, and half allowed up to 15 words. Generalizability results showed high interrater agreement, with tests of between 2 and 4 items scored by one judge achieving coefficients in the .80s. Construct validity analyses found that F-H was only marginally related to the GRE General Test, and more strongly related than the General Test to a measure of ideational fluency. Different response limits tapped somewhat different abilities, with the 15-word constraint appearing more useful for graduate assessment. These items added significantly to conventional measures in explaining school performance and creative expression.     

The Effects of Play Tutoring on Young Children’s Cognitive Performance

Abstract

Seventeen preschoolers were divided into a play tutoring group and a skills tutoring group. Subjects in the first group received nine 20-minute sessions in which an adult attempted to enhance their make-believe play. The other group received an equal number of skills tutoring sessions. Both groups received a comparable amount of adult contact. The subjects were assessed on measures of play quality, verbal intelligence, and creativity , on three occasions: prior to training, immediately after training, and three months later. Results showed that the play tutoring and skills tutoring treatments both resulted in stable gains in verbal intelligence and ideational fluency. Neither treatment resulted in significant increases in play quality. It was concluded that the play tutoring gains were caused by adult contact rather than by enhanced levels of play.     

Imagery and Verbal Ability and Recognition Memory for Pictures and Words in Males and Females

Objective and subjective indices of imagery and verbal ability and their relationship to recognition memory for pictures and concrete words were examined in a large correlational study. Objective spatial tests of imagery proved to be better predictors of picture recognition than were self‐assessments, the Flags test excepted. Spatial tests also predicted word memory in males, but not females. Imagery control and imagery vividness bore little relation to recognition; vividness within specific modalities may have some influence, however, particularly in males. Verbal fluency and verbal comprehension were either unrelated or marginally negatively related to performance. In general, a more specialised pattern of ability‐performance correlations emerged for females than for males.     

Does Moral Judgment Development Reduce to Political Attitudes or Verbal Ability? Evidence Using the Defining Issues Test

Recent critics of the Defining Issues Test (DIT) suggest that moral judgment development as currently measured is neither developmental nor moral. Instead, scores on the DIT are claimed to be the result of political attitudes or verbal ability. In making such claims, these critics raise the possibility that the validity of the DIT is suspect as well as the construct it purports to measure. We begin our response with an overview of the various claims that DIT scores reduce to political attitudes or verbal ability. Then, a relevant sample of the DIT literature is summarized in order to assess the degree to which relationships between DIT scores and criterion variables typically used to support the validity of the DIT can be explained by political attitudes or verbal ability. This summary suggests that moral judgment development as measured by the DIT provides a unique source of information that cannot be explained by general/verbal ability or political attitudes.     

Reading‐retarded Icelandic Children: the discriminant validity of psychological tests

Discriminant validity of several psychological tests was investigated for a sample of reading‐retarded and normal Icelandic fifth grade children. Differentiation was greatest on verbal intelligence, accounting for 40% of the variation, and on various other verbal tests accounting for almost 40% of the remaining variation. Benton Visual Retention Test was the only visual‐spatial test that discriminated between the two groups. Apart from verbal intelligence the Expressive scale of the Luria‐Nebraska Neuropsychogical Battery discriminated most highly between the groups and was the only measure with incremental validity over the intelligence measures. Cluster analysis revealed two reading‐retarded subgroups. One was an undifferenti‐ated group characterized by lowered level of performance on all measures and heterogeneous individual profiles. The second cluster was very compact with specific delay on verbal measures, possibly indicating sequencing difficulties. The results indicate the need for an adequate intelligence measure in Iceland and the development of refined measures of specific verbal skills.

     

Assessing Teachers’ Attitudes Towards Students with Mental Health Disorders in 16 Postsecondary Institutions in Quebec

The number of students with mental health disorders (MHDs) in postsecondary institutions is increasing. Teachers’ attitudes are among the main factors that influence these students’ academic success. Unfortunately, their attitudes are little studied, in part because of lack of validated scales. Therefore, this article has two purposes: (1) analyse the construct validity of new scales assessing teachers’ attitudes, as well as existing scales without an established construct validity; and (2) report the correlations between the variables assessed in these scales, with a special focus on two variables: (a) teachers’ negative attitudes; and (b) the measures offered to students by teachers. The participants are 232 teachers from 16 French and English CEGEPS in Quebec, Canada. The questionnaire includes three scales created by the research team and three scales from the Mental Illness Awareness Survey. Correlational and factor analyses were conducted. The scales present good construct validity. Four variables were associated with the two variables of interest: confidence in one’s ability to convince a student to seek help; familiarity with MHDs; knowledge and understanding of MHDs; favourable perception of the accommodations offered by adapted and psychosocial services. Based on these results, strategies are proposed to improve teachers’ attitudes.     

Profiles of reading performance after exiting Tier 2 intervention

A fundamental assumption of multi-tiered systems of support is the bidirectional movement of students between tiers. In comparison to research on when to intensify instruction, less attention has been paid to the validity of decisions to exit students from supplemental supports. We used data from 554 third-grade students who met the criteria to be exited from a Tier 2 reading fluency intervention to answer two research questions. First, we used latent profile analysis to evaluate whether distinct profiles of student responding would emerge based upon initial skill level and intervention duration. Second, we evaluated whether the proportion of students who passed the end-of-year benchmark differed between profiles. Results favored a three-profile solution with a Below Average Start/Average Response; Average Start/Average Response; and an Above Average Start/Fast Response. The pooled mean proportion of students who did not achieve proficiency on the end-of-year benchmark assessment equaled 0.45, 0.33, and 0.13 across the three profiles, respectively. A series of χ² tests indicated the between-profile differences were statistically significant (p < .05). The results of this study suggest that there may be distinct profiles amongst students who show positive response and exit Tier 2 reading fluency interventions.     

Measuring service quality in higher education: three instruments compared

Measuring the quality of service in higher education is increasingly important, particularly as fees introduce a more consumerist ethic amongst students. This paper aims to test and compare the relative efficacy of three measuring instruments of service quality (namely HEdPERF, SERVPERF and the moderating scale of HEdPERF‐SERVPERF) within a higher education setting. The objective was to determine which instrument had the superior measuring capability in terms of unidimensionality, reliability, validity and explained variance. Tests were conducted utilizing a sample of higher education students, and the findings indicated that HEdPERF scale resulted in more reliable estimations, greater criterion and construct validity, greater explained variance, and consequently was a better fit than the other two instruments. Consequently, a modified five‐factor structure of HEdPERF is put forward as the more superior scale for the higher education sector.     

Cognitive Correlates of the Covariance in Reading and Arithmetic Fluency: Importance of Serial Retrieval Fluency

This study examines the core predictors of the covariance in reading and arithmetic fluency and the domain-general cognitive skills that explain the core predictors and covariance. Seven-year-old Finnish children (N = 200) were assessed on rapid automatized naming (RAN), phonological awareness, letter knowledge, verbal counting, number writing, number comparison, memory skills, and processing and articulation speed in the spring of Grade 1 and on reading and arithmetic fluency in the fall of Grade 2. RAN and verbal counting were strongly associated, and a constructed latent factor, serial retrieval fluency (SRF), was the strongest unique predictor of the shared variance. Other unique predictors were phonological awareness, number comparison, and processing speed. Findings highlight the importance of SRF in clarifying the relation between reading and arithmetic fluency.     

Convergent and discriminant construct validity of the internalizing symptoms scale for children with the BASC‐SRP‐C

This brief report details a study of the construct validity of the Internalizing Symptoms Scale for Children (ISSC) in comparison with the Child Self‐Report Form of the Behavioral Assessment System for Children (BASC). Using self‐reports of 121 students ages 8–12 from general education classes, who were administered both measures, correlational data were obtained between the scores of the two instruments. ISSC factor and total scores were found to have weak associations with the BASC School Adjustment Scales, and negatively associations with the BASC personal adjustment scales, providing discriminant evidence of the construct validity of the ISSC. Moderate to strong correlations were found between ISSC scores and the BASC Clinical Maladjustment, Other Problems, and Emotional Symptoms Index scores, evidencing convergent construct validity between the two measures. These results support the validity and clinical use of the ISSC as a measure of internalizing symptoms and emotional problems. © 2003 Wiley Periodicals, Inc. Psychol Schs 40: 139–144, 2003.     

The Relationship Between Children's Drawings of Rectangular Solids from Memory and Performance in Cognitive Tests,Art and Practical Subjects

A total of 127 11 + and 81 12 + subjects drew rectangular solids from memory, after viewing them from different positions. The results were correlated with IQ, vocabulary, verbal judgement and reasoning, and a spatial test with assessment in art and crafts. Unexpectedly high and significant correlations were yielded between drawing solids and cognitive tests involving verbal ability, when compared with correlations between solids, the spatial test and craft subjects, especially for the 12 + group. This appeared contrary to received views on the association of aptitude in drawing with spatial ability. Furthermore, as spatial factors are considered to underpin drawing and craft subjects, a closer association between these, rather than between drawing and verbal ability, was expected. This outcome raised certain theoretical questions concerning drawing and its interaction with perception, language, memory and intellectual development.     

Teachers using learning styles: Torn between research and accountability?

The proliferation of instruments reporting learning/cognitive style with school pupils is of particular interest, because most research on them focuses on applications in higher education, training and the adult workplace, where criticisms of their integrity, reliability and validity have been significant. This study examines two such popular instruments in highly effective schools in England, UK. Neither instrument demonstrated reasonable internal consistency or results according with theoretical constructs. Concerns about their usefulness in these contexts are explored. Reasons provided by faculty for their use were: face validity; external inspection; professional accountability; institutional policy; the legacy from initial training; established classroom practice.     

Is Earlier Better? Mastery of Reading Fluency in Early Schooling

Research Findings: The goal of the present study was to provide empirical evidence for the importance of mastering reading fluency in early schooling. Study participants were 1,322 students in 3rd grade in 42 schools in a northwestern state. These students were assessed using a battery of reading skill tests as well as comprehensive tests of more general reading outcomes from kindergarten through 3rd grade. Practice or Policy: The results of the study show that mastery of reading fluency prior to currently established benchmarks is a significant positive predictor of later reading skills in primary grades even after student demographic information and initial reading levels are controlled. The results provide additional evidence for the importance of early reading development, early intervention, and preventing reading difficulties as early as possible in schooling.     

FLUENCY AS A PERVASIVE ELEMENT IN THE MEASUREMENT OF CREATIVITY1

It was suggested that fluency, defined as number of responses, may misleadingly influence both high Intercorrelations sometimes reported among measures of creativity and low correlations sometimes reported between measures of creativity and intelligence. Subjects were 93 Saturday art school students between the ages of 9 and 15 years. Intercorrelations among five “creativity” scores derived from a slightly modified version of Torrance's Figure Completion Test and between these scores and Henmon-Nelson Intelligence were compared using both raw creativity scores and creativity scores corrected for the effect of fluency. Uncorrected creativity scores intercorrelated high among themselves (mean r= .45) and low with intelligence (mean r= .09), while corrected creativity scores showed low intercorrelations among themselves (mean r= .08) and with intelligence (mean r= .13). These findings are interpreted as confirming the influence of fluency upon high intercorrelations among so-called measures of creativity and as failing to support the suggestion that fluency may also influence low correlations among creativity and intelligence measures.     

Influence of Verbal Working Memory Depends on Vocabulary: Oral Reading Fluency in Adolescents With Dyslexia

Most research on dyslexia to date has focused on early childhood, while comparatively little is known about the nature of dyslexia in adolescence. The current study had two objectives. The first was to investigate the relative contributions of several cognitive and linguistic factors to connected‐text oral reading fluency in a sample of adolescents with dyslexia (n = 77). The second was to test the hypothesis that the effect of verbal working memory on connected‐text oral reading fluency is moderated by word‐level skills and/or vocabulary knowledge. The results suggest that many deficits associated with childhood dyslexia remain prominent in adolescence, but the nature of the relationships between key cognitive and linguistic predictors (i.e., word‐level reading, vocabulary, verbal working memory) and reading fluency appear to be different in adolescence. For example, while word‐level skills remain a significant predictor, the strength of the effect is relatively weak. In contrast, the data support an increased role for vocabulary and verbal working memory, including an interaction between these factors. The presence of an interaction can be interpreted as evidence that the influence of verbal working memory on connected‐text oral reading fluency in adolescents with dyslexia depends on individual differences in vocabulary knowledge. These results offer support for the changing nature of dyslexia across development, and suggest that researchers should study dyslexia in adolescents on its own terms, rather than treating it as an extension of reading problems in early childhood.