Using automated analysis to assess middle school students' competence with scientific argumentation

Argumentation is fundamental to science education, both as a prominent feature of scientific reasoning and as an effective mode of learning—a perspective reflected in contemporary frameworks and standards. The successful implementation of argumentation in school science, however, requires a paradigm shift in science assessment from the measurement of knowledge and understanding to the measurement of performance and knowledge in use. Performance tasks requiring argumentation must capture the many ways students can construct and evaluate arguments in science, yet such tasks are both expensive and resource-intensive to score. In this study we explore how machine learning text classification techniques can be applied to develop efficient, valid, and accurate constructed-response measures of students' competency with written scientific argumentation that are aligned with a validated argumentation learning progression. Data come from 933 middle school students in the San Francisco Bay Area and are based on three sets of argumentation items in three different science contexts. The findings demonstrate that we have been able to develop computer scoring models that can achieve substantial to almost perfect agreement between human-assigned and computer-predicted scores. Model performance was slightly weaker for harder items targeting higher levels of the learning progression, largely due to the linguistic complexity of these responses and the sparsity of higher-level responses in the training data set. Comparing the efficacy of different scoring approaches revealed that breaking down students' arguments into multiple components (e.g., the presence of an accurate claim or providing sufficient evidence), developing computer models for each component, and combining scores from these analytic components into a holistic score produced better results than holistic scoring approaches. However, this analytical approach was found to be differentially biased when scoring responses from English learners (EL) students as compared to responses from non-EL students on some items. Differences in the severity between human and computer scores for EL between these approaches are explored, and potential sources of bias in automated scoring are discussed.     

Preparing Teachers to Make the Formative Assessment Process Integral to Science Teaching and Learning

Formative assessment provides a means to successfully scaffold learning. Unfortunately, few teachers understand the pedagogical implications of such scaffolding or their role in utilizing formative assessments. The purpose of our study was to develop an understanding of the experience of being a teacher that is seeking to improve learning through formative assessment and using that understanding to improve our practices in teacher education. Three categories emerged from the data analysis: (1) questioning the validity of tacit pedagogical understandings, (2) understanding the experiences of the students, and (3) exploring the need to develop a more reflexive classroom. The findings from this study are useful for persons seeking to prepare teachers to guide science learning.     

Sex equity in workplace conflict management

Three hundred two managers and employees from eleven different organizations were investigated to determine whether there was a significant difference in the selection of conflict management strategies based on: 1) the sex of the respondent; 2) the sex of the other person; and 3) the sex of the supervisor. The results indicated: a) females were no more compromising nor any less competitive than males; b) respondents report using different strategies in a narrative question than they do answering a self‐report, checklist questionnaire.     

“We thought it might encourage participation.” Using lottery incentives to improve LibQUAL+™ response rates among students

Libraries deploying the LibQUAL+? survey can offer a lottery incentive and many do so in the hope of increasing response rates. Other libraries may be prohibited from offering one because of Institutional Review Board restrictions, as is the case at Oregon State University. We wanted to discover why libraries offer lottery incentives, what kinds of incentives, and if they believe these incentives have a positive impact on their response rates. The responding libraries hold a general belief that lottery incentives are effective, but base this on feeling rather than research. We examine what the literature says about lottery incentives and student populations.     

It isn't a matter of just being funny: Humor production by organizational leaders

This study explored whether organizational leaders recognize humor as part of their management style and how compatible the leaders ‘ humor strategy is with their overall communication style. Results reveal that leaders use more pro‐social humor than anti‐social humor in the workplace. Several relationships between the emotional aspects of humor production and the communicator style constructs indicate that leaders use humor as a socially acceptable communication technique to control groups and enhance their own communicator image.     

I Am Sure There May Be a Planet There: Student articulation of uncertainty in argumentation tasks

We investigated how students articulate uncertainty when they are engaged in structured scientific argumentation tasks where they generate, examine, and interpret data to determine the existence of exoplanets. In this study, 302 high school students completed 4 structured scientific arguments that followed a series of computer-model-based curriculum module activities simulating the radial velocity and/or the transit method. Structured scientific argumentation tasks involved claim, explanation, uncertainty rating, and uncertainty rationale. We explored (1) how students are articulating uncertainty within the various elements of the task and (2) the relationship between the way the task is presented and the way students are articulating uncertainty. We found that (1) while the majority of students did not express uncertainty in either explanation or uncertainty rationale, students who did express uncertainty in their explanations did so scientifically without being prompted explicitly, (2) students’ uncertainty ratings and rationales revealed a mix of their personal confidence and uncertainty related to science, and (3) if a task presented noisy data, students were less likely to express uncertainty in their explanations.     

An Investigation of the New York State Regents Examinations in Science

This study examined the effects of realism on subjects’ recognition memory for computergraphics. Children and adults in six age groups (N = 189) were presented with pictorial computergraphic materials at three levels of realism: low (abstract or schematic—images reduced to simple flat shapes), medium (linear—simple line drawings), and high (realistic—photographic images and detailed drawings). Pictorial content was sampled widely from existing computer materials with each image containing only one pictorial element: person, animal, or object. Recognition memory was tested at immediate and a week’s delay. Recognition of the three kinds of image showed an unusual pattern of decline over time with the initial advantage for realistic and abstract computergraphics disappearing at a week’s delay, while recognition for linear images was superior to that of realistic and abstract. Although recognition memory increased with age, no interaction between age and realism was found. That is, all age groups, 6 years through middle age, retained the linear computergraphics best. Discussion includes interpretations of these findings and their instructional implications.     

Structuring experimental learning into a traditional political science curriculum

This paper describes the introduction of an experimental element into a traditional political science undergraduate curriculum. Careful consideration of this case study may enable those designing similar programs elsewhere to be more successful.     

Organizational Integration,Strategic Planning,and Staff Assessment in Publicly Funded Libraries

Library and information center services are at risk during times of extensive budget reductions. Publicly funded institutions labeled as inessential or as auxiliary departments may lose the revenue necessary to maintain full staffing. Financial circumstances of recent years highlight the importance of strategic planning in library and information center management. Although typically characterized as formal nonprofit organizations that depend on government expenditures and private donations, libraries are also open systems. The operations of an open system can be explained by reference to a levers model. Open systems are sensitive to the external environment and depend upon competent management for successful operation. Assessment measures are necessary for management evaluation. Poor decision making can be attributed to a lack of coherent policies in regards to employee evaluation and retention. A coherent assessment strategy is mutually reinforcing and can be illustrated by examples in the form of tables or other diagrams that locate integration policies vertically, in regards to the chosen strategy, and horizontally, in regards to how the policies support each other. The proper role of HR departments is to fairly implement the policies. Strategic planning and assessments should include standards of conduct and abide by established ethical criteria.