Evaluating Instrument Quality in Science Education: Rasch‐based analyses of a Nature of Science test

Given the central importance of the Nature of Science (NOS) and Scientific Inquiry (SI) in national and international science standards and science learning, empirical support for the theoretical delineation of these constructs is of considerable significance. Furthermore, tests of the effects of varying magnitudes of NOS knowledge on domain‐specific science understanding and belief require the application of instruments validated in accordance with AERA, APA, and NCME assessment standards. Our study explores three interrelated aspects of a recently developed NOS instrument: (1) validity and reliability; (2) instrument dimensionality; and (3) item scales, properties, and qualities within the context of Classical Test Theory and Item Response Theory (Rasch modeling). A construct analysis revealed that the instrument did not match published operationalizations of NOS concepts. Rasch analysis of the original instrument—as well as a reduced item set—indicated that a two‐dimensional Rasch model fit significantly better than a one‐dimensional model in both cases. Thus, our study revealed that NOS and SI are supported as two separate dimensions, corroborating theoretical distinctions in the literature. To identify items with unacceptable fit values, item quality analyses were used. A Wright Map revealed that few items sufficiently distinguished high performers in the sample and excessive numbers of items were present at the low end of the performance scale. Overall, our study outlines an approach for how Rasch modeling may be used to evaluate and improve Likert‐type instruments in science education.     

A longitudinal investigation of the role of quantity and quality of child-directed speech in vocabulary development

Quantity and quality of caregiver input was examined longitudinally in a sample of 50 parent–child dyads to determine which aspects of input contribute most to children’s vocabulary skill across early development. Measures of input gleaned from parent–child interactions at child ages 18, 30, and 42 months were examined in relation to children’s vocabulary skill on a standardized measure 1 year later (e.g., 30, 42, and 54 months). Results show that controlling for socioeconomic status, input quantity, and children’s previous vocabulary skill; using a diverse and sophisticated vocabulary with toddlers; and using decontextualized language (e.g., narrative) with preschoolers explains additional variation in later vocabulary ability. The differential effects of various aspects of the communicative environment at several points in early vocabulary development are discussed.     

Text density level as a design variable in instructional displays

Presenting narrative text in computer-based instruction (CBI) is more difficult than in print lessons due to limited screen resolution and display area. The present research aimed at achieving a more compatible match between lesson content and the attributes of the presentation media. Subjects received either a print or CBI statistics lesson containing low-density (concise) narrative text, high-density (conventional) text, or the density type they preferred (learner-control). Results showed the low-density lesson to reduce completion time relative to the high-density version, while yielding equivalent achievement. Subjects receiving the lesson in the print mode had shorter completion times and higher achievement than their CBI counterparts. Attitude results showed that CBI was favorably regarded, but perceived as longer and slow-moving than print. Implications of results for designing instructional material in accord with CBI attributes and learner characteristics are discussed.     

Using a hypertext environment for teaching process writing: An evaluation study of three student groups

The present research consisted of a comprehensive evaluation of a hypertext model for teaching process writing at the junior high and high school level. Interests were to determine how two teachers and three different age groups of students used and reacted to the model, specifically, its embedded design features of model stories, note cards, idea buttons, mini-lessons, branching buttons, and cut-and-paste-tools. Results showed applications of the embedded features to vary based on teacher attitudes, feature attributes (e.g., ease of use and appeal), and student characteristics. Older students made more usage of many of the features, but were less positive about the hypertext model given their greater involvement with completing writing assignments rather than with exploring new forms of writing. The implications of the results are discussed regarding the instructional design and classroom implementation of new technologies for teaching process writing strategies.This study was conducted as a doctoral dissertation completed by the first author at the University of Memphis under the supervision of the second and third authors.     

Infant vocalizing and phenotypic outcomes in autism: Evidence from the first 2 years

Infant vocalizations are early-emerging communicative markers shown to be atypical in autism spectrum disorder (ASD), but few longitudinal, prospective studies exist. In this study, 23,850 infant vocalizations from infants at low (LR)- and high (HR)-risk for ASD (HR-ASD = 23, female = 3; HR-Neg = 35, female = 13; LR = 32, female = 10; 80% White; collected from 2007 to 2017 near Philadelphia) were analyzed at 6, 12, and 24 months. At 12 months, HR-ASD infants produced fewer vocalizations than HR-Neg infants. From 6 to 24 months, HR-Neg infants demonstrated steeper vocalization growth compared to HR-ASD and LR infants. Finally, among HR infants, vocalizing at 12 months was associated with language, social phenotype, and diagnosis at age 2. Infant vocalizing is an objective behavioral marker that could facilitate earlier detection of ASD.     

The special magic of the eighties: Shaping words and shape-shifting words

She has a Master's degree in children's literature and a Ph.D. in Australian literature. One of her particular research interests is in teaching literature to ESL students.     

From conception to performance: How undergraduate students conceptualise and construct essays

Interviews with forty six undergraduate students enrolled in either first or third year of a Bachelor of Education explored how they conceptualised and undertook an essay writing task. The conceptual structure of their essays was analysed using the SOLO Taxonomy. Comparisons between students who wrote essays of differing levels of complexity revealed that there were major differences between students at every stage of the essay writing process. Compared to students writing essays with simple conceptual structures, students writing more complex essays engaged in processes of reconstruction rather than knowledge telling, put more effort into finding references, used organisational systems for integrating their notes according to topics or themes, built arguments rather than presented information when structuring and drafting their essays, were concerned with improving ideas and arguments as well as mechanics when revising their essays, had a more sophisticated understanding of the concepts underlying the assessment criteria, and expected and received higher grades. The results suggest a developmental process in underlying conceptualisations of both the body of knowledge forming the content of the essay, and the essay writing processes themselves. As such, attempts to improve students' essay writing skills need to shift from a focus on discrete skills to an emphasis on the relationship between students' understanding of the content and their ability to write about it. As part of the writing process, students need help building understandings representing the body of knowledge they are writing about, and this help needs to be geared to their current level of operation.     

The Vestiges of Childhood Interparental Conflict: Adolescent Sensitization to Recent Interparental Conflict

This study examined whether childhood interparental conflict moderated the mediational pathway involving adolescent exposure to interparental conflict, their negative emotional reactivity to family conflict, and their psychological problems in a sample of 235 children (M_age = 6 years). Significant moderated-mediation findings indicated that the mediational path among Wave 4 interparental conflict during adolescence, change in youth negative reactivity (Waves 4–5), and their psychological problems (Waves 4–6) was significant for teens who experienced low, rather than high, levels of childhood interparental conflict (Waves 1–3). Supporting the stress sensitization model, analyses showed that adolescents exposed to high interparental conflict during childhood evidenced greater increases in negative reactivity than their peers when recent parental conflicts were mild.     

Consistency of report card grades and external assessments in a Canadian province

The study investigated how well report card grades communicate to students and parents that state educational standards are being met, standards that are objectively measured by infrequently administered mandated assessments. Data sources were report card grades and external assessment scores for 2006–09 for Ontario Canada. The information that parents and students received about student performance from report cards and external assessments were similar (r _s  = .47) to the r = .40–.60 range previously reported. Teachers assigned higher grades than external assessments warranted, even after a major source of construct irrelevant variance in report card grades (teacher ratings on multiple scales measuring student effort and school commitment) was controlled. The relationship of grades to assessment scores was robust across genders, school district types (Public versus Catholic) and language (English and French). Agreement of assessments was higher for grade 6 than for grade 3 and for Writing than for Reading or Mathematics. Report cards provided information about students’ future achievement that was accurate and delivered up to 2 years prior to the administration of external assessments. Seventy to 80% of students who reached the provincial achievement standard on one or both prior report cards were successful on the subsequent external assessment, compared to 30–50% of students who failed to meet the report card standard at least once.