Articulating the validity evidence for a science alternate assessment

Students with the most significant cognitive disabilities (SCD) are the 1% of the total student population who have a disability or multiple disabilities that significantly impact intellectual functioning and adaptive behaviors and who require individualized instruction and substantial supports. Historically, these students have received little instruction in science and the science assessments they have participated in have not included age‐appropriate science content. Guided by a theory of action for a new assessment system, an eight‐state consortium developed multidimensional alternate content standards and alternate assessments in science for students in three grade bands (3–5, 6–8, 9–12) that are linked to the Next Generation Science Standards (NGSS Lead States, 2013 ) and A Framework for K‐12 Science Education (Framework; National Research Council, 2012 ). The great variability within the population of students with SCD necessitates variability in the assessment content, which creates inherent challenges in establishing technical quality. To address this issue, a primary feature of this assessment system is the use of hypothetical cognitive models to provide a structure for variability in assessed content. System features and subsequent validity studies were guided by a theory of action that explains how the proposed claims about score interpretation and use depend on specific assumptions about the assessment, as well as precursors to the assessment. This paper describes evidence for the main claim that test scores represent what students know and can do. We present validity evidence for the assumptions about the assessment and its precursors, related to this main claim. The assessment was administered to over 21,000 students in eight states in 2015–2016. We present selected evidence from system components, procedural evidence, and validity studies. We evaluate the validity argument and demonstrate how it supports the claim about score interpretation and use.     

Developing and Validating a Novel Anonymous Method for Matching Longitudinal School-Based Data

Prospective longitudinal data collection is an important way for researchers and evaluators to assess change. In school-based settings, for low-risk and/or likely-beneficial interventions or surveys, data quality and ethical standards are both arguably stronger when using a waiver of parental consent—but doing so often requires the use of anonymous data collection methods. The standard solution to this problem has been the use of a self-generated identification code. However, such codes often incorporate personalized elements (e.g., birth month, middle initial) that, even when meeting the technical standard for anonymity, may raise concerns among both youth participants and their parents, potentially altering willingness to participate, response quality, or generating outrage. There may be value, therefore, in developing a self-generated identification code and matching approach that not only is technically anonymous but also appears anonymous to a research-naive individual. This article provides a proof of concept for a novel matching approach for school-based longitudinal data collection that potentially accomplishes this goal.     

Virtual Laboratories in Undergraduate Science and Engineering Courses: a Systematic Review, 2009–2019

Journal of Science Education and Technology - Current technology has the capacity for affording a virtual experience that challenges the notion of a teaching laboratory for undergraduate science...     

Reading against all odds: a pilot study of two deaf students with dyslexia

Learning to read and write is a challenge for most deaf children due to their limited experiences with, and access to, spoken language. In the case of deaf students who have difficulty processing visual print, literacy becomes an even greater challenge. The study piloted an intervention procedure that incorporated the principles of automaticity, repetition, functional vocabulary, and a positive teacher-student relationship as recommended in programs for struggling readers and adapted them to the needs of two deaf high school students with dyslexia in an American Sign Language-English bilingual program. The findings reveal gains in reading ability on the formal measures, though not more than would be expected over a 6-month period simply due to development. The real improvements were noted in the students' attitudes toward literacy, improved social interaction, and increased self-confidence.     

Impact of a statewide home visiting program on parenting and on child health and development

OBJECTIVES: To assess the impact of a voluntary, paraprofessional home visiting program on promoting child health and development and maternal parenting knowledge, attitudes, and behaviors. METHODS: This collaborative, experimental study of 6 Healthy Families Alaska (HFAK) programs enrolled 325 families from 1/00 to 7/01, randomly assigned them to HFAK and control groups, interviewed mothers at baseline, and followed families until children were 2 years old (85% follow-up). Child outcomes included health care use, development and behavior. Parent outcomes included knowledge of infant development, parenting attitudes, quality of the home environment, and parent-child interaction. HFAK records were reviewed to measure home visiting services. Home visitors were surveyed to measure knowledge, perceived effectiveness and perceived training adequacy. RESULTS: There was no overall impact on child health, but HFAK group children had more favorable developmental and behavioral outcomes. HFAK and control mothers had similar parenting outcomes except that HFAK mothers had greater parenting self-efficacy (35.1 vs. 34.6 based on the Teti Self-Efficacy Scale, p<.05). Fewer HFAK families had a poor home environment for learning (20% vs. 31%, p<.001). HFAK families were more likely to use center-based parenting services (48% vs. 39%, p<.05). The impact was greater for families with lower baseline risk (Family Stress Checklist scores<45). There was little evidence of efficacy for families with a higher dose of service. CONCLUSIONS: The program promoted child development and reduced problem behaviors at 2 years. Impact could be strengthened by improving home visitor effectiveness in promoting effective parenting. Future research is needed to determine whether short-term benefits are sustained.     

bioRxiv: Trends and analysis of five years of preprints

bioRxiv was founded on the premise that publicly posting preprints would allow authors to receive feedback and submit improved papers to journals. This paper analyses a number of trends against this stated purpose, namely, the timing of preprint postings relative to submission to accepting journals; trends in the rate of unpublished preprints over time; trends in the timing of publication of preprints by accepting journals; and trends in the concentration of published, reviewed preprints by publisher. Findings show that a steady c.30% of preprints remain unpublished and that the majority is posted onto bioRxiv close to or after submission – therefore giving no time for feedback to help improve the articles. Four publishers (Elsevier, Nature, PLOS, and Oxford University Press) account for the publication of 47% of bioRxiv preprints. Taken together, it appears that bioRxiv is not accomplishing its stated goals and that authors may be using the platform more to establish priority, as a marketing enhancement of papers, and as functional Green OA, rather than as a community‐driven source of prepublication review.     

How Much Have They Retained? Making Unseen Concepts Seen in a Freshman Electromagnetism Course at MIT

The introductory freshmen electromagnetism course at MIT has been taught since 2000 using a studio physics format entitled TEAL—Technology Enabled Active Learning. TEAL has created a collaborative, hands-on environment where students carry out desktop experiments, submit web-based assignments, and have access to a host of visualizations and simulations. These learning tools help them visualize unseen electromagnetic concepts and develop stronger intuition about related phenomena. A previous study has shown that students who took the course in the TEAL format (the experimental group) gained significantly better conceptual understanding than those who took it in the traditional lecture-recitation format (the control group). The present longitudinal study focuses on the extent to which these two research groups (experimental and control) retain conceptual understanding about a year to 18 months after finishing the course. It also examines students attitudes about whether the teaching format (TEAL or traditional) contributes to their learning in advanced courses. Our research has indicated that the long-term effect of the TEAL course on students’ retention of concepts was significantly stronger than that of the traditional course. This research is significant because it documents the long-term cognitive and affective impact of the TEAL studio physics format on learning outcomes of MIT students.

Race,school choice and transfers to opportunity: implications for educational stratification in South Africa

This study investigates the links between race, school choice and educational stratification in South Africa. It focuses on racial differences in families’ efforts to take advantage of choice by transferring to opportunity (i.e., transferring to a school perceived to offer students better access to educational opportunity than their current school) and explores the implications of those transfers for emerging patterns of racial stratification in access to high quality schools. Findings reveal significant racial variation in transfers to opportunity and suggest a complex link between race and the ways in which historically disadvantaged groups have engaged with the post-apartheid educational opportunity structure.     

Obstacles to learning from experience and principal training

This paper discusses the obstacles to on-the-job learning for school administrators. Applying Kolb's (1984) experiential learning model to understanding the on-the-job preparation of administrators, it describes how characteristics of the work, properties of the organization, and norms of the occupation make this form of learning often slow, unreliable, counter-productive, and biased. The four stages of experiential learning (concrete experience, reflective analysis, abstract conceptualization, and active experimentation) are analyzed and applied to the learning of school principals with particular attention to the obstacles to effective learning at each stage. Finally, the paper suggests ways that those who prepare, train, and work alongside principals can improve the quality of learning-from-experience for novice as well as veteran administrators.Paper presented at the Annual Meeting of the American Education Research Association in Chicago, April, 1985.     

Instructional systems development in a large-scale education project

The Mission of Botswana's Junior Secondary Education Improvement Project is to expand the country's junior secondary education program, which is roughly equivalent to the junior high school level in the United States. An instructional systems design (ISD) model was used in the design of the project. As such, the project has provided an opportunity to study the practical use and implementation of an ISD approach to a sector-wide education intervention within the setting of a developing country. In this article, issues pertaining to sectorwide planning strategies of educational development are explored. Particularly stressed is the importance of conducting a contextual review at the outset of a large-scale educational project. Note: In 1984, USAID began a ten-year project called “Improving the Efficiency of Educational Systems” (IEES) that has provided technical assistance to a number of countries around the world through a consortium of institutions including Florida State University, the State University of New York at Albany, and the Institute for International Research. The Junior Secondary Education Improvement Project (JSEIP) in Botswana has been one of the IEES projects.