Development of a ‘universal’ rubric for assessing undergraduates' scientific reasoning skills using scientific writing

We developed a rubric for measuring students' ability to reason and write scientifically. The Rubric for Science Writing (Rubric) was tested in a variety of undergraduate biology laboratory courses (total n = 142 laboratory reports) using science graduate students (teaching assistants) as raters. Generalisability analysis indicates that the Rubric provides a reliable measure of students' abilities (g = 0.85) in these conditions. Comparison of student performance in various biology classes indicated that some scientific skills are more challenging for students to develop than others and identified a number of previously unappreciated gaps in the curriculum. Our findings suggest that use of the Rubric provides three major benefits in higher education: (1) to increase substance and consistency of grading within a course, particularly those staffed by multiple instructors or graduate teaching assistants; (2) to assess student achievement of scientific reasoning and writing skills; and (3) when used in multiple courses, to highlight gaps in alignment among course assignments and provide a common metric for assessing to what extent the curriculum is achieving programmatic goals. Lastly, biology graduate students reported that use of the Rubric facilitated their teaching and recommended that training on the Rubric be provided to all teaching assistants.     

Design and Reflection Help Students Develop Scientific Abilities: Learning in Introductory Physics Laboratories

Design activities, when embedded in an inquiry cycle and appropriately scaffolded and supplemented with reflection, can promote the development of the habits of mind (scientific abilities) that are an important part of scientific practice. Through the Investigative Science Learning Environment (ISLE), students construct physics knowledge by engaging in inquiry cycles that replicate the approach used by physicists to construct knowledge. A significant portion of student learning occurs in ISLE instructional labs where students design their own experiments. The labs provide an environment for cognitive apprenticeship enhanced by formative assessment. As a result, students develop interpretive knowing that helps them approach new problems as scientists. This article describes a classroom study in which the students in the ISLE design lab performed equally well on traditional exams as ISLE students who did not engage in design activities. However, the design group significantly outperformed the non-design group while working on novel experimental tasks (in physics and biology), demonstrating the application of scientific abilities to an inquiry task in a novel content domain. This research shows that a learning environment that integrates cognitive apprenticeship and formative assessment in a series of conceptual design tasks provides a rich context for helping students build scientific habits of mind.     

From organelle to protein gel: a 6-wk laboratory project on flagellar proteins

Research suggests that undergraduate students learn more from lab experiences that involve longer-term projects. We have developed a one-semester laboratory sequence aimed at sophomore-level undergraduates. In designing this curriculum, we focused on several educational objectives: 1) giving students a feel for the scientific research process, 2) introducing them to commonly used lab techniques, and 3) building skills in both data analysis and scientific writing. Over the course of the semester, students carry out two project-based lab experiences and write two substantial lab reports modeled on primary literature. Student assessment data indicate that this lab curriculum achieved these objectives. This article describes the first of these projects, which uses the biflagellate alga Chlamydomonas reinhardtii to introduce students to the study of flagellar motility, protein synthesis, microtubule polymerization, organelle assembly, and protein isolation and characterization.     

Argument-Driven Inquiry as a Way to Help Undergraduate Students Write to Learn by Learning to Write in Chemistry

This exploratory study examined how undergraduate students’ ability to write in science changed over time as they completed a series of laboratory activities designed using a new instructional model called argument-driven inquiry. The study was conducted in a single section of an undergraduate general chemistry lab course offered at a large two-year community college located in the southeast USA. The intervention took place over a 15-week semester and consisted of six laboratory activities. During each laboratory activity, the undergraduates wrote investigation reports, participated in a double-blind group peer review of the reports, and revised their reports based on the reviews. The reports written during each laboratory activity were used to examine changes in the students’ writing skills over time and to identify aspects of scientific writing that were the most difficult for the undergraduates in this context. The reviews produced by the students during each report were used to evaluate how well undergraduates engage in the peer-review process. The results of a quantitative and qualitative analysis of the reports and reviews indicate that the participants made significant improvements in their ability to write in science and were able to evaluate the quality of their peers’ writing with a relatively high degree of accuracy, but they also struggled with several aspects of scientific writing. The conclusions and implications of the study include recommendations for helping undergraduate students learn to write by writing to learn in science and new directions for future research.     

The road to self-assessment: exemplar marking before peer review develops first-year students’ capacity to judge the quality of a scientific report

Lack of clarity about assessment criteria and standards is a source of anxiety for many first-year university students. The Developing Understanding of Assessment for Learning (DUAL) programme was designed as a staged approach to gradually familiarise students with expectations, and to provide opportunities for the development of the skills required to successfully complete assessment tasks. This paper investigated the students’ perceptions of the first two components of the DUAL programme, which assist first-year biology students to engage with stated assessment criteria and standards in order to develop their capacity to make judgements about scientific report exemplars, their peers’ scientific reports and ultimately their own. The study found strong evidence (96% of responses) that the marking and discussion of exemplar reports with peers and demonstrators clarified expectations of scientific report writing. A key feature of this element of DUAL was the opportunity for structured discussion about assessment criteria and standards between peers and markers (demonstrators). During these discussions, students can clarify explicit statements and develop a tacit knowledge base to enhance their ability to judge the quality of others’ work and their own. The peer review exercise (the second element of DUAL) was not rated as highly, with 65% of students finding the process helpful for improving their report. The negative reactions by a sizeable minority of students highlight the need to clearly communicate the expectations and benefits of peer review, with a focus on how the process of giving feedback to peers might benefit a student as much as receiving feedback on their own report.     

Learning science through writing: associations with prior conceptions of writing and perceptions of a writing program

Students in a large undergraduate biology course were expected to write a scientific report as a key part of their course design. This study investigates the quality of learning arising from the writing experience and how it relates to the quality of students’ preconceptions of learning through writing and their perceptions of their writing program that led to their report. Closed‐ended questionnaires investigating student conceptions and perceptions of writing, and approaches to writing, were completed by 121 students. Significant associations were found amongst qualitatively different prior and post conceptions of writing, approaches to writing and achievement. The results of the analyses suggest that the effective support of student experiences of writing reports requires teachers to be aware of the type of conceptions that students bring to their course and the perceptions they hold about the purpose of the writing program in which they are engaged.     

Young science journalism: writing popular scientific articles may contribute to an increase of high-school students’ interest in the natural sciences

Far too few high-school students choose subjects from the natural sciences (NaSc) for their majors in many countries. Even fewer study biology, chemistry or physics at university. Those, that do, often lack training to present and discuss scientific results and ideas in texts. To meet these challenges the center for didactics of biology of Graz University has set up the program Young Science Journalism. This new workshop-based interdisciplinary concept was tested in an exploratory study with grade 10 students of one Austrian high school, engaging both the biology and the German teacher of the class. It was our aim to raise students’ interest in the NaSc by encouraging them to write popular scientific articles about self-chosen topics, and to help them improve their writing competence. In this paper we focus on interest development through writing. Results from this pilot study were promising. Using a mixed-method approach (comparing pre- and post-test questionnaires and semi-structured interviews from different time points analyzed qualitatively), we found that almost all students valued the project-related work highly. Most of them showed higher interest in the NaSc at project end with girls, in average, seeming to profit more from project participation. We thus recommend integrating such writing tasks into school curricula to increase students’ interest in NaSc or to even create new interest. Additionally, we introduce a network presentation of questionnaire data as a powerful tool to visualize the effect of an intervention on individual students and student profile groups.     

Investigating Turkish Primary School Students’ Interest in Science by Using Their Self-Generated Questions

This paper reports on an attempt to investigate Turkish primary school students’ interest in science by using their self-generated questions. We investigated students’ interest in science by analyzing 1704 self-generated science-related questions. Among them, 826 questions were submitted to a popular science magazine called Science and Children. Such a self-selected sample may represent a group of students who have a higher level of motivation to seek sources of information outside their formal education and have more access to resources than the students of low social classes. To overcome this problem, 739 students were asked to write a question that they wanted to learn from a scientist and as a result 878 questions were gathered. Those students were selected from 13 different schools at 9 cities in Turkey. These schools were selected to represent a mixture of socioeconomic areas and also to cover different students’ profile. Students’ questions were classified into two main categories: the field of interest and the cognitive level of the question. The results point to the popularity of biology, astrophysics, nature of scientific inquiry, technology and physics over other science areas, as well as indicating a difference in interest according to gender, grade level and the setting in which the questions were asked. However, our study suggests that only considering questions submitted to informal learning environments, such as popular science magazines or Ask-A-Scientist Internet sites has limitations and deficiencies. Other methodologies of data collection also need to be considered in designing teaching and school science curriculum to meet students’ needs and interest. The findings from our study tend to challenge existing thinking from other studies. Our results show that self-generated questions asked in an informal and a formal setting have different patterns. Some aspects of students’ self-generated questions and their implications for policy, science curriculum reform and teaching are discussed in this paper.     

The influences of integrating reading,peer evaluation,and discussion on undergraduate students’ scientific writing

Scientific writing is related to the practice of communicating scientific knowledge. This study treats scientific writing as a social practice, taking as its premise the notion that participating in related activities such as reading, peer evaluation, and discussion would positively affect the competence of students’ scientific writing, by developing their epistemic cognition regarding scientific knowledge communication and legitimation. An empirical study was conducted with twenty-two Chinese undergraduate students to test this premise. These students were majoring in chemistry and undertook a researcher-designed intervention course (Advanced Organic Chemistry Experiment), which embodied the integrated strategy of reading, peer evaluation, and discussion on scientific writing. Based on data pertaining to those students’ performance in terms of the normativity, objectivity, and logicality of scientific writing drawn from a previous study by the current author (Deng, Kelly, & Xiao, 2019. The development of Chinese undergraduate students’ competence of scientific writing in the context of advanced organic chemistry experiment course. Chemistry Education Research and Practice, 20, 270–287), alongside data drawn from students’ written texts in reading reflections and on-line discourse related to peer evaluation and discussion, this study claimed that the tasks of reading, peer evaluation, and discussion were seen gradually to lead to the development of Chinese undergraduate students’ greater competence in scientific writing regarding the synthetic experiments of organic chemistry.     

Synergy Between Teacher Practices and Curricular Scaffolds to Support Students in Using Domain-Specific and Domain-General Knowledge in Writing Arguments to Explain Phenomena

We investigated how 2 different curricular scaffolds (context-specific vs. generic), teacher instructional practices, and the interaction between these 2 types of support influenced students' learning of science content and their ability to write scientific arguments to explain phenomena. The context-specific scaffolds provided students with hints about the task and what content knowledge to use in or incorporate into their writing. The generic scaffolds supported students in understanding a general framework (i.e., claim, evidence, and reasoning) regardless of the content area or task. This study focused on an 8-week middle school chemistry curriculum that was enacted by 6 teachers with 578 students during the 2004–2005 school year. Analyses of identical pre- and posttests as well as videotapes of teacher enactments revealed that the curricular scaffolds and teacher instructional practices were synergistic in that the effect of the written curricular scaffolds depended on the teacher's enactment of the curriculum. The context-specific curricular scaffolds were more successful in supporting students in writing scientific arguments to explain phenomena, but only when teachers' enactments provided explicit domain-general support for the claim, evidence, and reasoning framework, suggesting the importance of both types of support in successful learning environments.     

Processes Involving Perceived Instructional Support,Task Value,and Engagement in Graduate Education

The purpose of this study was to investigate the relations among perceived instructional support (provision of relevance and involvement), subjective task value beliefs (utility, attainment, and intrinsic value), and engagement (behavioral and emotional) over the course of a semester for graduate students enrolled in an introductory research methods course in a college of education (N = 217). Structural equation modeling (SEM) results suggested that each source of perceived support exerted unique influence on students' beliefs about utility value, intrinsic value, and attainment value of their research methods coursework. In turn, midsemester value beliefs predicted semester-end student reports of behavioral and cognitive engagement. Direct and indirect effects between perceived provisions of support and engagement were inconsistent. This study supports the notion that different patterns of predictors and outcomes may occur for different aspects of value beliefs and that multiple sources of instructional support are important for supporting student motivation for adult learners.     

Implementing Recommendations for Introductory Biology by Writing a New Textbook

We redesigned the undergraduate introductory biology course by writing a new textbook (Integrating Concepts in Biology [ICB]) that follows first principles of learning. Our approach emphasizes primary data interpretation and the utility of mathematics in biology, while de-emphasizing memorization. This redesign divides biology into five big ideas (information, evolution, cells, emergent properties, homeostasis), addressing each at five levels of organization (molecules, cells, organisms, populations, ecological systems). We compared our course outcomes with two sections that used a traditional textbook and were taught by different instructors. On data interpretation assessments administered periodically during the semester, our students performed better than students in the traditional sections (p = 0.046) and exhibited greater improvement over the course of the semester (p = 0.015). On factual content assessments, our students performed similarly to students in the other sections (p = 0.737). Pre- and postsemester assessment of disciplinary perceptions and self-appraisal indicate that our students acquired a more accurate perception of biology as a discipline and may have developed a more realistic evaluation of their scientific abilities than did the control students (p < 0.05). We conclude that ICB improves critical thinking, metacognition, and disciplinary perceptions without compromising content knowledge in introductory biology.     

Translating expertise into effective instruction: The impacts of cognitive task analysis (CTA) on lab report quality and student retention in the biological sciences

Poor instruction has been cited as a primary cause of attrition from STEM majors and a major obstacle to learning for those who stay [Seymour and Hewitt [1997]. Talking about leaving: Why undergraduates leave the sciences. Boulder, CO: Westview]. Using a double‐blind design, this study tests the hypothesis that the lack of explicit instructions in scientific inquiry skills is a major factor in both low STEM retention and academic underperformance. This project delivered supplemental instruction to students in a laboratory‐based undergraduate biology course (n = 314) that was derived either from cognitive task analyses (CTAs) conducted with expert biologists (treatment) or was authored and delivered by an award‐winning biology instructor (control). Students receiving traditional instruction were almost six times more likely to withdraw from the course than students in the treatment condition (8.1% vs. 1.4% of initial enrollment). Of the students who completed the course, those who received the CTA‐based instruction demonstrated significantly higher levels of performance in the discussion section of their written laboratory reports. Significantly higher performances were seen specifically in the areas of analyzing data to formulate valid conclusions, considering alternative explanations, consideration for the limitations of the experimental design and implications of the research. © 2010 Wiley Periodicals, Inc. J Res Sci Teach 47: 1165–1185, 2010     

Influences of Writing Tasks on Students' Answers to Recall and Higher-Level Test Questions

This paper reports on two inter-related studies that examined the use of non-traditional writing strategies within secondary school science classrooms. The first study involved Year 10 students who incorporated one letter writing experience into the learning sequence when studying genetics. The second study was with Year 9 students who used both a non-traditional laboratory writing heuristic and letter writing as part of the learning sequence when studying a topic on light. The same teacher was involved in both case studies. A higher-level analogy question was added to the teacher-prepared tests for each study to examine if students who participated in writing-to-learn activities were able to perform significantly better as a group than a group of students who completed traditional teacher directed laboratory activities and note-taking. Results indicate that for the first study there was not a significant difference using t-test analysis, while for the second study involving two writing treatments there was a statistically significant difference using t-test analysis. There was no statistically significant difference in responses between treatment and control groups when answering low level recall questions for either case study. Student interviews indicated awareness by students of the metacognitive value gained by using the non-traditional writing types.     

Directed-inquiry approach to learning science process skills: treatment effects and aptitude-treatment interactions

The directed-inquiry approach to learning science process skills and scientific problem solving [DIAL(SPS)2] was developed to help high-school students gain the critical thinking skills required to solve problems in the biology lab. This curriculum integrated several learning strategies into a single approach: advance organizers, the learning cycle, concept maps, Vee diagrams, a focusing strategy, and writing. Two general questions were addressed. First, was the DIAL(SPS)2 treatment more effective than a conventional treatment? Second, was there evidence of an aptitude-treatment interaction? Four high-school biology classes taught by this investigator were used to test the DIAL(SPS)2 curriculum. Scheduling of students involved ability grouping. To test the curriculum in the most rigorous way, the experimental group consisted of average ability students and the comparison group consisted of above-average students. Both the groups were pretested in August and posttested in May. In the intervening time, the experimental group received the DIAL(SPS)2 treatment while the comparison group received a more traditional approach. Analysis of covariance revealed that the DIAL(SPS)2 curriculum had no significant effect on the learning of science process skills or on cognitive development. Aptitude-treatment interaction analyses revealed an interaction of DIAL(SPS)2 treatment and cognitive development.     

Essay writing in biology: An example of effective student learning?

The views of first-year biology students (N=337) on an essay writing assignment were evaluated by means of a questionnaire. The students were asked to reflect on the strategies they employed, the number and type of resources used, their areas of difficulty and to evaluate their own performance. The data were used to elucidate possible areas of discrepancy between the approach taken by the students and that suggested by the Biology Department via information in student manuals and evaluation criteria. The data were also compared to similar studies on student writing previously reported for students of psychology and history. Finally a series of recommendations is made to help staff to allow their students to develop improved writing strategies, minimise the possible difficulties encountered and allow the writing exercise to fulfil its desired outcome, that of being an integral part of the process of learning.     

The development of scientific reasoning skills in conjunction with collaborative writing assignments: An interpretive study of six ninth-grade students

In this investigation, three classes of ninth-grade general science students participated in a collaborative report-writing intervention. The purpose of this portion of the study was to evaluate students' collaboratively written laboratory reports for evidence of the use of scientific reasoning skills and to document qualitative changes in reasoning skill use over time. The participants in the study were 6 ninth-grade students, representing three collaborative writing pairs. During the intervention, students wrote 10 laboratory reports over a 4.5-month period. The author and classroom teacher designed report guideline prompts to scaffold students in the use of relevant scientific reasoning skills. The results indicated that students used reasoning skills to assess their current models of scientific understanding, make observations, interpret the meaning of results, and generate new models based on their data and relevant information. Participants showed the most improvement in writing that reflected the reasoning skills of (a) selecting and processing textbook passages, (b) drawing conclusions and formulating models, and (c) comparing/contrasting. Over time, participants improved their ability to compose explanations that represented a synthesis of prior knowledge, activity observations, and other sources of information. Collaborative writing encouraged students to construct their own understandings of science concepts by creating an environment in which thinking, reasoning, and discussion were valued.     

Examining the Value of a Scaffolded Critique Framework to Promote Argumentative and Explanatory Writings Within an Argument-Based Inquiry Approach

This study investigated the value of using a scaffolded critique framework to promote two different types of writing—argumentative writing and explanatory writing—with different purposes within an argument-based inquiry approach known as the Science Writing Heuristic (SWH) approach. A quasi-experimental design with sixth and seventh grade students taught by two teachers was used. A total of 170 students participated in the study, with 87 in the control group (four classes) and 83 in the treatment group (four classes). All students used the SWH templates as an argumentative writing to guide their written work and completed these templates during the SWH investigations of each unit. After completing the SWH investigations, both groups of students were asked to complete the summary writing task as an explanatory writing at the end of each unit. All students’ writing samples were scored using analytical frameworks developed for the study. The results indicated that the treatment group performed significantly better on the explanatory writing task than the control group. In addition, the results of the partial correlation suggested that there is a very strong significantly positive relationship between the argumentative writing and the explanatory writing.