Using automated analysis to assess middle school students' competence with scientific argumentation

Argumentation is fundamental to science education, both as a prominent feature of scientific reasoning and as an effective mode of learning—a perspective reflected in contemporary frameworks and standards. The successful implementation of argumentation in school science, however, requires a paradigm shift in science assessment from the measurement of knowledge and understanding to the measurement of performance and knowledge in use. Performance tasks requiring argumentation must capture the many ways students can construct and evaluate arguments in science, yet such tasks are both expensive and resource-intensive to score. In this study we explore how machine learning text classification techniques can be applied to develop efficient, valid, and accurate constructed-response measures of students' competency with written scientific argumentation that are aligned with a validated argumentation learning progression. Data come from 933 middle school students in the San Francisco Bay Area and are based on three sets of argumentation items in three different science contexts. The findings demonstrate that we have been able to develop computer scoring models that can achieve substantial to almost perfect agreement between human-assigned and computer-predicted scores. Model performance was slightly weaker for harder items targeting higher levels of the learning progression, largely due to the linguistic complexity of these responses and the sparsity of higher-level responses in the training data set. Comparing the efficacy of different scoring approaches revealed that breaking down students' arguments into multiple components (e.g., the presence of an accurate claim or providing sufficient evidence), developing computer models for each component, and combining scores from these analytic components into a holistic score produced better results than holistic scoring approaches. However, this analytical approach was found to be differentially biased when scoring responses from English learners (EL) students as compared to responses from non-EL students on some items. Differences in the severity between human and computer scores for EL between these approaches are explored, and potential sources of bias in automated scoring are discussed.     

“We thought it might encourage participation.” Using lottery incentives to improve LibQUAL+™ response rates among students

Libraries deploying the LibQUAL+? survey can offer a lottery incentive and many do so in the hope of increasing response rates. Other libraries may be prohibited from offering one because of Institutional Review Board restrictions, as is the case at Oregon State University. We wanted to discover why libraries offer lottery incentives, what kinds of incentives, and if they believe these incentives have a positive impact on their response rates. The responding libraries hold a general belief that lottery incentives are effective, but base this on feeling rather than research. We examine what the literature says about lottery incentives and student populations.     

I Am Sure There May Be a Planet There: Student articulation of uncertainty in argumentation tasks

We investigated how students articulate uncertainty when they are engaged in structured scientific argumentation tasks where they generate, examine, and interpret data to determine the existence of exoplanets. In this study, 302 high school students completed 4 structured scientific arguments that followed a series of computer-model-based curriculum module activities simulating the radial velocity and/or the transit method. Structured scientific argumentation tasks involved claim, explanation, uncertainty rating, and uncertainty rationale. We explored (1) how students are articulating uncertainty within the various elements of the task and (2) the relationship between the way the task is presented and the way students are articulating uncertainty. We found that (1) while the majority of students did not express uncertainty in either explanation or uncertainty rationale, students who did express uncertainty in their explanations did so scientifically without being prompted explicitly, (2) students’ uncertainty ratings and rationales revealed a mix of their personal confidence and uncertainty related to science, and (3) if a task presented noisy data, students were less likely to express uncertainty in their explanations.     

An Investigation of the New York State Regents Examinations in Science

This study examined the effects of realism on subjects’ recognition memory for computergraphics. Children and adults in six age groups (N = 189) were presented with pictorial computergraphic materials at three levels of realism: low (abstract or schematic—images reduced to simple flat shapes), medium (linear—simple line drawings), and high (realistic—photographic images and detailed drawings). Pictorial content was sampled widely from existing computer materials with each image containing only one pictorial element: person, animal, or object. Recognition memory was tested at immediate and a week’s delay. Recognition of the three kinds of image showed an unusual pattern of decline over time with the initial advantage for realistic and abstract computergraphics disappearing at a week’s delay, while recognition for linear images was superior to that of realistic and abstract. Although recognition memory increased with age, no interaction between age and realism was found. That is, all age groups, 6 years through middle age, retained the linear computergraphics best. Discussion includes interpretations of these findings and their instructional implications.     

‘They did things differently then’: diversity and challenge from Key Stage 1

This paper is based on the premise that controversial issues related to diversity are an important aspect of teaching and learning with all pupils. It was stimulated by a newspaper article which criticised teaching Key Stage 1 (5–7-year-old children) about Grace O'Malley, an Irish Queen. We argue that sixteenth century Anglo-Irish relations can be taught across all key stages and illustrate this through a detailed examination of how this was undertaken with Key Stage 1 pupils. To support our case we draw on literature related to diversity in a range of contexts and age groups.     

Remedial Testing and Placement in Community Colleges

Almost half of students who enter college require some sort of remedial coursework. Further, states are increasingly moving the responsibility of postsecondary remediation away from four-year campuses to two-year institutions. Scholars and policymakers have grappled with best practice for successfully filling in academic gaps and moving students forward, and there is variation both within and between states as to the ways in which remediation is defined, determined, and administered (Perin, 2006 Perin , D. ( 2006 ). Can community colleges protect both access and standards? The problem of remediation . Teachers College Record 108 ( 3 ), 339 – 373 .[Crossref], [Web of Science ®] , [Google Scholar]). Using a regression discontinuity design, this study seeks to answer the following question on one community college campus: How does placement into remedial services affect student outcomes, in particular, successful completion of an introductory college-level English course? Although this study focuses on a single community college system, the findings have bearing on a wider sector as the developmental placement practices of Harper Community College District are not unlike those implemented by two- and four-year campuses across the country.     

Organizational Integration,Strategic Planning,and Staff Assessment in Publicly Funded Libraries

Library and information center services are at risk during times of extensive budget reductions. Publicly funded institutions labeled as inessential or as auxiliary departments may lose the revenue necessary to maintain full staffing. Financial circumstances of recent years highlight the importance of strategic planning in library and information center management. Although typically characterized as formal nonprofit organizations that depend on government expenditures and private donations, libraries are also open systems. The operations of an open system can be explained by reference to a levers model. Open systems are sensitive to the external environment and depend upon competent management for successful operation. Assessment measures are necessary for management evaluation. Poor decision making can be attributed to a lack of coherent policies in regards to employee evaluation and retention. A coherent assessment strategy is mutually reinforcing and can be illustrated by examples in the form of tables or other diagrams that locate integration policies vertically, in regards to the chosen strategy, and horizontally, in regards to how the policies support each other. The proper role of HR departments is to fairly implement the policies. Strategic planning and assessments should include standards of conduct and abide by established ethical criteria.     

Self‐definition of women experiencing a nontraditional graduate fellowship program

Women continue to be underrepresented in the fields of science, technology, engineering, and mathematics (STEM). One factor contributing to this underrepresentation is the graduate school experience. Graduate programs in STEM fields are constructed around assumptions that ignore the reality of women's lives; however, emerging opportunities may lead to experiences that are more compatible for women. One such opportunity is the Graduate Teaching Fellows in K–12 Education (GK–12) Program, which was introduced by the National Science Foundation in 1999. Although this nontraditional graduate program was not designed explicitly for women, it provided an unprecedented context in which to research how changing some of the basic assumptions upon which a graduate school operates may impact women in science. This exploratory case study examines the self‐definition of 8 women graduate students who participated in a GK–12 program at a major research university. The findings from this case study contribute to higher education's understanding of the terrain women graduate students in the STEM areas must navigate as they participate in programs that are thought to be more conducive to their modes of self‐definition while they continue to seek to be successful in the historically Eurocentric, masculine STEM fields. © 2006 Wiley Periodicals, Inc. J Res Sci Teach 43: 852–873, 2006