Digital Module 18: Automated Scoring https://ncme.elevate.commpartners.com

In this digital ITEMS module, Dr. Sue Lottridge, Amy Burkhardt, and Dr. Michelle Boyer provide an overview of automated scoring. Automated scoring is the use of computer algorithms to score unconstrained open-ended test items by mimicking human scoring. The use of automated scoring is increasing in educational assessment programs because it allows scores to be returned faster at lower cost. In the module, they discuss automated scoring from a number of perspectives. First, they discuss benefits and weaknesses of automated scoring, and what psychometricians should know about automated scoring. Next, they describe the overall process of automated scoring, moving from data collection to engine training to operational scoring. Then, they describe how automated scoring systems work, including the basic functions around score prediction as well as other flagging methods. Finally, they conclude with a discussion of the specific validity demands around automated scoring and how they align with the larger validity demands around test scores. Two data activities are provided. The first is an interactive activity that allows the user to train and evaluate a simple automated scoring engine. The second is a worked example that examines the impact of rater error on test scores. The digital module contains a link to an interactive web application as well as its R-Shiny code, diagnostic quiz questions, activities, curated resources, and a glossary.     

Effects of developmental relationships with teachers on middle-school students’ motivation and performance

We examined how middle-school students’ motivation, belonging, school climate, and grade point average (GPA) are affected by students experiencing developmental relationships—those that go beyond teachers being caring (e.g., showing warmth to students) and providing challenge (e.g., high expectations) to also include teachers providing support, sharing power, and expanding students’ sense of possibilities. We also examined variations in those associations by student socioeconomic status (SES). The study included 534 diverse Grades 6–8 students (51% female, 46% non-White, 33% eligible for free and reduced price meals). Structural equation modeling and regressions showed that students with better developmental relationships with their teachers had better outcomes. Developmental relationships strongly predicted academic motivation at both the beginning and end of the school year, and also directly predicted students’ sense of belonging and school climate. Relationships indirectly predicted GPA, through motivation. Student–teacher relationship quality for low-SES students was lower and declined more than for other students. The results both reflect current literature in showing the importance of strong student–teacher relationships, and extend it in showing the worsening relationships quality for low-income students, and in suggesting that strengthening multiple facets of student–teacher developmental relationships may have important effects on motivation and achievement of middle-school students.     

Influence of Adiposity and Maturation on the Motor Performance of Girls Aged 8 to 16 Years

ABSTRACT

Excess adipose tissue may impact the motor performance of youth. Therefore, the current study examined (a) if adiposityinfluenced performance in the jump and reach and 30-yard dash and (b) if maturation predicted motor performance. Participants (N = 267) included girls aged 8 to 16 years. Latent growth curves were used to assess changes in motor performance. Sum of skinfolds predicted jump and reach performance from age 8.5 to 10.5 years, 11.5–12.0 years, 13.0 years, and from age 14 to 14.5 years (b = ?0.013 to 0.051, p < 0.05). Sum of skinfolds predicted 30-yard dash performance from age 8.5 to 14.0 years and at age 15.5 years (b = 0.003 to 0.005, p < 0.05). Maturation did not predict performance. Future research should examine determinants that explain the link between adiposity and motor performance.     

Contributions of the emergent literacy environment to literacy outcomes for young children who are deaf

Specific characteristics of early literacy environments support hearing children's emergent literacy. The researchers investigated these characteristics' role in emergent literacy in young deaf and hard of hearing (DHH) children, using the Early Language and Literacy Classroom Observation (ELLCO; M. W. Smith, Dickinson, Sangeorge, & Anastasopoulos, 2002). Eighteen self-contained classrooms of preschool, kindergarten, and first-grade DHH children (N = 40) were studied. Hierarchical linear analysis was used to examine study participants' classroom environment and growth in emergent literacy skills. Correlations suggested that classroom environment was more closely related to vocabulary and phonological awareness in DHH children than in typically hearing children. Major differences among classrooms were also indicated. However, growth in children's skills did not correlate strongly with attributes captured by the ELLCO. This suggests that classrooms promoting emergent literacy skills acquisition in DHH children may differ from classrooms of typically developing hearing children.     

Contemporary trends in East Asian higher education: dispositions of international students in a Taiwan university

This article details a mixed methods study conducted during the 2007–2008 academic year at the National Chengchi University (NCCU) in Taipei Taiwan. It contributes to discourse examining the opportunities and challenges of international student enrollments in institutions of higher learning around the globe. In scope it details an empirical study exploring the dispositions of NCCU international students in terms of their academic and social spheres. Trends in Taiwan reflect traditional East Asian patterns; substantial numbers of university students from Taiwan studied in the United States and Britain while very few incoming international students chose the island nation as a host destination. In recent years the influx of international students to Taiwan has increased significantly, rising from 6,380 in 2001 to 21,005 in 2007 (Ko, , 2008). The use of both quantitative and qualitative methods provided clarity and extended critical interpretations of the issues and dilemmas surrounding the international student experience in Taiwan. Results indicate that the capability of the NCCU as a host institution to sustain and attract increasing numbers of incoming international students is linked to factors such as the unique opportunity to study traditional as opposed to simplified Chinese characters, the availability and accessibility of Taiwan government sponsored scholarships, and the high standard of the NCCU Mandarin Studies program. Implications suggest that universities committed to internationalization are called to address the realities—both positive and negative—of operating as globally competitive institutions. As such, attracting the right kind of international students at the NCCU and determining standards for their contribution to campus life are more important goals than the total number of international students.     

Making formative assessment discernable to pre‐service teachers of science

The purpose of this pragmatic action research study was to explore our re‐conceptualization efforts in preparing pre‐service teachers to guide the inquiry process with formative assessment and subsequently use the understandings to improve our teacher preparation program. The process was guided by two questions: to what extent did course re‐conceptualization efforts lead to a more informed understanding of formative assessment by pre‐service teachers and did strategies enacted in the re‐conceptualized methods course foster or hinder pre‐service teachers' understanding of formative assessment? Data from this study support the following findings: (1) a substantial pre‐ to post‐methods course difference was realized in the pre‐service teachers' understanding of formative assessment; (2) explicit and contextualized approaches to formative assessment in the methods course led to increased understandings by pre‐service teachers; (3) an implicit approach led to improvements in course structure but did not foster pre‐service teachers' understanding of the reflexive nature of formative assessment; and (4) a field‐based case study on elementary science teaching both hindered and fostered our efforts with formative assessment. This study yields implications for pre‐service teacher education on formative assessment. To foster pre‐service teachers' knowledge and skills, we suggest explicit instruction on formative assessment combined with case studies, field experiences, and ongoing reflection. © 2009 Wiley Periodicals, Inc. J Res Sci Teach 47: 402–421, 2010     

They Said So on the News: Parsing Media Reports About Birth

In this column, the authors reprise recent selections from the Lamaze International research blog, Science & Sensibility. Each selection discusses shortcomings in the news media coverage of childbirth issues. The authors demonstrate how to identify misleading claims in the media and highlight how childbirth educators can apply a common-sense approach and careful fact checking to help women understand the whole story.