Development of the Statistical Reasoning in Biology Concept Inventory (SRBCI)

We followed established best practices in concept inventory design and developed a 12-item inventory to assess student ability in statistical reasoning in biology (Statistical Reasoning in Biology Concept Inventory [SRBCI]). It is important to assess student thinking in this conceptual area, because it is a fundamental requirement of being statistically literate and associated skills are needed in almost all walks of life. Despite this, previous work shows that non–expert-like thinking in statistical reasoning is common, even after instruction. As science educators, our goal should be to move students along a novice-to-expert spectrum, which could be achieved with growing experience in statistical reasoning. We used item response theory analyses (the one-parameter Rasch model and associated analyses) to assess responses gathered from biology students in two populations at a large research university in Canada in order to test SRBCI’s robustness and sensitivity in capturing useful data relating to the students’ conceptual ability in statistical reasoning. Our analyses indicated that SRBCI is a unidimensional construct, with items that vary widely in difficulty and provide useful information about such student ability. SRBCI should be useful as a diagnostic tool in a variety of biology settings and as a means of measuring the success of teaching interventions designed to improve statistical reasoning skills.     

The EvoDevoCI: A Concept Inventory for Gauging Students’ Understanding of Evolutionary Developmental Biology

The American Association for the Advancement of Science 2011 report Vision and Change in Undergraduate Biology Education encourages the teaching of developmental biology as an important part of teaching evolution. Recently, however, we found that biology majors often lack the developmental knowledge needed to understand evolutionary developmental biology, or “evo-devo.” To assist in efforts to improve evo-devo instruction among undergraduate biology majors, we designed a concept inventory (CI) for evolutionary developmental biology, the EvoDevoCI. The CI measures student understanding of six core evo-devo concepts using four scenarios and 11 multiple-choice items, all inspired by authentic scientific examples. Distracters were designed to represent the common conceptual difficulties students have with each evo-devo concept. The tool was validated by experts and administered at four institutions to 1191 students during preliminary (n = 652) and final (n = 539) field trials. We used student responses to evaluate the readability, difficulty, discriminability, validity, and reliability of the EvoDevoCI, which included items ranging in difficulty from 0.22–0.55 and in discriminability from 0.19–0.38. Such measures suggest the EvoDevoCI is an effective tool for assessing student understanding of evo-devo concepts and the prevalence of associated common conceptual difficulties among both novice and advanced undergraduate biology majors.     

Longitudinal Study of Student Attitudes in a Biology Program

This is among the first longitudinal studies to report student attitudes across 4 yr of a university program. We found that the attitudes of students in biology become significantly more expert-like from the first year to the fourth year of the program, that is, there was a significant positive shift in students’ overall percent favorable scores from 64.5 to 72%, as opposed to the expert response, which averaged 90%. There was a significant positive shift for the real world connection category (78–85%), the enjoyment (personal interest) category (74–82%), and the conceptual connections/memorization category (66–74%). Moreover, there was a significant correlation between students’ overall percent favorable scores and performance (cumulative grade point average) at the end, but not at the beginning, of the fourth year, with high-performing students having significantly more expert-like attitudes than low-performing students. The correlation between percent favorable score and performance was the strongest for the problem solving: synthesis and application category, in which the highest-performing students finished their fourth year with 90% favorable compared with 35% favorable for the lowest-performing students. A comparison of these results with previously reported results and their implications for teaching are discussed.     

A Bookstore for Bailey: A Novel Approach to Teaching a Small‐Business Management Course

This article presents the development of an online, student‐centered, introductory small‐business management course that uses an educational business novel, hypertext graphic‐design features, an interactive workbook, and a student‐authored final chapter. Student learning was assessed through the use of a pre‐ and posttest survey. Student reflection was incorporated to make students consider the impact of learning. Results showed that student learning and understanding increased in two separate courses.     

Conceptual Learning Outcomes of Virtual Experiential Learning: Results of Google Earth Exploration in Introductory Geoscience Courses

Virtual globe programs such as Google Earth replicate real-world experiential learning of spatial and geographic concepts by allowing students to navigate across our planet without ever leaving campus. However, empirical evidence for the learning value of these technological tools and the experience students gain by exploration assignments framed within them remains to be quantified and compared by student demographics. This study examines the impact of a Google Earth-based exploration assignment on conceptual understanding in introductory geoscience courses at a research university in the US Midwest using predominantly traditional college-age students from a range of majors. Using repeated-measures ANOVA and paired-samples t tests, we test the significance of the activity using pretest and posttest scores on a subset of items from the Geoscience Concept Inventory, and the interactive effects of student gender and ethnicity on student score improvement. Analyses show that learning from the Google Earth exploration activity is highly significant overall and for all but one of the concept inventory items. Furthermore, we find no significant interactive effects of class format, student gender, or student ethnicity on the magnitude of the score increases. These results provide strong support for the use of experiential learning in virtual globe environments for students in introductory geoscience and perhaps other disciplines for which direct observation of our planet’s surface is conceptually relevant.     

Initial teacher trainees and their views of teaching and learning

One hundred and sixty-two student secondary teachers were followed through a postgraduate certificate in education course at a university department of education in England to identify, using a variety of methods, changes in their thinking about the teaching and learning process and initial training course. While it was found that student teachers' knowledge of teaching gained from earlier experience was highly influential in their views on teaching and learning and interpretation of the course, differences between individuals and curriculum groups emerged which suggest that the course of training could not be considered a constant, as had been assumed by earlier studies. Several issues are raised and discussed concerning the influence of preservice courses on student teachers' thinking and classroom practice, together with their implications for course design and the selection of students.     

A Placement Test for Computer Science: Design,Implementation, and Analysis

An introductory CS1 course presents problems for educators and students due to students' diverse background in programming knowledge and exposure. Students who enroll in CS1 also have different expectations and motivations. Prompted by the curricular guidelines for undergraduate programmes in computer science released in 2001 by the ACM/IEEE, and driven by a departmental project to reinvent the undergraduate computer science and computer engineering curricula at the University of Nebraska-Lincoln, we are currently implementing a series of changes which will improve our introductory courses. One key component of our project is an online placement examination tied to the cognitive domain that assesses student knowledge and intellectual skills. Our placement test is also integrated into a comprehensive educational research design containing a pre- and posttest framework for assessing student learning and providing valuable feedback for needed instructional revisions. In this paper, we focus on the design and implementation of our placement exam and present an analysis of the data collected to date.     

A High-Enrollment Course-Based Undergraduate Research Experience Improves Student Conceptions of Scientific Thinking and Ability to Interpret Data

We present an innovative course-based undergraduate research experience curriculum focused on the characterization of single point mutations in p53, a tumor suppressor gene that is mutated in more than 50% of human cancers. This course is required of all introductory biology students, so all biology majors engage in a research project as part of their training. Using a set of open-ended written prompts, we found that the course shifts student conceptions of what it means to think like a scientist from novice to more expert-like. Students at the end of the course identified experimental repetition, data analysis, and collaboration as important elements of thinking like a scientist. Course exams revealed that students showed gains in their ability to analyze and interpret data. These data indicate that this course-embedded research experience has a positive impact on the development of students’ conceptions and practice of scientific thinking.     

A model for using a concept inventory as a tool for students' assessment and faculty professional development

This essay describes how the use of a concept inventory has enhanced professional development and curriculum reform efforts of a faculty teaching community. The Host Pathogen Interactions (HPI) teaching team is composed of research and teaching faculty with expertise in HPI who share the goal of improving the learning experience of students in nine linked undergraduate microbiology courses. To support evidence-based curriculum reform, we administered our HPI Concept Inventory as a pre- and postsurvey to approximately 400 students each year since 2006. The resulting data include student scores as well as their open-ended explanations for distractor choices. The data have enabled us to address curriculum reform goals of 1) reconciling student learning with our expectations, 2) correlating student learning with background variables, 3) understanding student learning across institutions, 4) measuring the effect of teaching techniques on student learning, and 5) demonstrating how our courses collectively form a learning progression. The analysis of the concept inventory data has anchored and deepened the team's discussions of student learning. Reading and discussing students' responses revealed the gap between our understanding and the students' understanding. We provide evidence to support the concept inventory as a tool for assessing student understanding of HPI concepts and faculty development.     

Development and Effectiveness of an Educational Card Game as Supplementary Material in Understanding Selected Topics in Biology

The complex concepts and vocabulary of biology classes discourage many students. In this study, a pretest–posttest model was used to test the effectiveness of an educational card game in reinforcing biological concepts in comparison with traditional teaching methods. The subjects of this study were two biology classes at Bulacan State University–Sarmiento Campus. Both classes received conventional instruction; however, the experimental group''s instruction was supplemented with the card game, while the control group''s instruction was reinforced with traditional exercises and assignments. The score increases from pretest to posttest showed that both methods effectively reinforced biological concepts, but a t test showed that the card game is more effective than traditional teaching methods. Additionally, students from the experimental group evaluated the card game using five criteria: goals, design, organization, playability, and usefulness. The students rated the material very satisfactory.     

From F = ma to Flying Squirrels: Curricular Change in an Introductory Physics Course

We present outcomes from curricular changes made to an introductory calculus-based physics course whose audience is primarily life sciences majors, the majority of whom plan to pursue postbaccalaureate studies in medical and scientific fields. During the 2011–2012 academic year, we implemented a Physics of the Life Sciences curriculum centered on a draft textbook that takes a novel approach to teaching physics to life sciences majors. In addition, substantial revisions were made to the homework and hands-on components of the course to emphasize the relationship between physics and the life sciences and to help the students learn to apply physical intuition to life sciences–oriented problems. Student learning and attitudinal outcomes were assessed both quantitatively, using standard physics education research instruments, and qualitatively, using student surveys and a series of postsemester interviews. Students experienced high conceptual learning gains, comparable to other active learning–based physics courses. Qualitatively, a substantial fraction of interviewed students reported an increased interest in physics relative to the beginning of the semester. Furthermore, more than half of students self-reported that they could now relate physics topics to their majors and future careers, with interviewed subjects demonstrating a high level of ability to come up with examples of how physics affects living organisms and how it helped them to better understand content presented in courses in their major.     

The Colorado Learning Attitudes about Science Survey (CLASS) for use in Biology

This paper describes a newly adapted instrument for measuring novice-to-expert-like perceptions about biology: the Colorado Learning Attitudes about Science Survey for Biology (CLASS-Bio). Consisting of 31 Likert-scale statements, CLASS-Bio probes a range of perceptions that vary between experts and novices, including enjoyment of the discipline, propensity to make connections to the real world, recognition of conceptual connections underlying knowledge, and problem-solving strategies. CLASS-Bio has been tested for response validity with both undergraduate students and experts (biology PhDs), allowing student responses to be directly compared with a consensus expert response. Use of CLASS-Bio to date suggests that introductory biology courses have the same challenges as introductory physics and chemistry courses: namely, students shift toward more novice-like perceptions following instruction. However, students in upper-division biology courses do not show the same novice-like shifts. CLASS-Bio can also be paired with other assessments to: 1) examine how student perceptions impact learning and conceptual understanding of biology, and 2) assess and evaluate how pedagogical techniques help students develop both expertise in problem solving and an expert-like appreciation of the nature of biology.     

Development of a Meiosis Concept Inventory

We have designed, developed, and validated a 17-question Meiosis Concept Inventory (Meiosis CI) to diagnose student misconceptions on meiosis, which is a fundamental concept in genetics. We targeted large introductory biology and genetics courses and used published methodology for question development, which included the validation of questions by student interviews (n = 28), in-class testing of the questions by students (n = 193), and expert (n = 8) consensus on the correct answers. Our item analysis showed that the questions’ difficulty and discrimination indices were in agreement with published recommended standards and discriminated effectively between high- and low-scoring students. We foresee other institutions using the Meiosis CI as both a diagnostic tool and an instrument to assess teaching effectiveness and student progress, and invite instructors to visit http://q4b.biology.ubc.ca for more information.     

Writing Assignments with a Metacognitive Component Enhance Learning in a Large Introductory Biology Course

Writing assignments, including note taking and written recall, should enhance retention of knowledge, whereas analytical writing tasks with metacognitive aspects should enhance higher-order thinking. In this study, we assessed how certain writing-intensive “interventions,” such as written exam corrections and peer-reviewed writing assignments using Calibrated Peer Review and including a metacognitive component, improve student learning. We designed and tested the possible benefits of these approaches using control and experimental variables across and between our three-section introductory biology course. Based on assessment, students who corrected exam questions showed significant improvement on postexam assessment compared with their nonparticipating peers. Differences were also observed between students participating in written and discussion-based exercises. Students with low ACT scores benefited equally from written and discussion-based exam corrections, whereas students with midrange to high ACT scores benefited more from written than discussion-based exam corrections. Students scored higher on topics learned via peer-reviewed writing assignments relative to learning in an active classroom discussion or traditional lecture. However, students with low ACT scores (17–23) did not show the same benefit from peer-reviewed written essays as the other students. These changes offer significant student learning benefits with minimal additional effort by the instructors.     

The Proof is in the Project: Integrating the Framework for Information Literacy in Developmental Education Through Project-Based Learning

ABSTRACT

Reading and library faculty collaborated in the revision of the developmental reading curriculum at Chandler-Gilbert Community College in an effort to strengthen students’ reading and information literacy skills through project-based learning. This article describes how the new curriculum addresses the challenging concepts defined in the Framework for Information Literacy for Higher Education in an effort to prepare students more effectively for college-level courses. The results of this new curriculum included stronger posttest scores and student success data.     

Using the retrospective pretest to get usable,indirect evidence of student learning

Minimal evidence exists regarding the retrospective pretest’s effectiveness as a tool to obtain usable, indirect evidence of student learning in open admissions institutions. The researcher conducted this study to determine if a retrospective pretest more accurately detected a change in students’ knowledge compared to a conventional pretest–posttest. The researcher designed a 17‐item inventory covering course objectives, and students were randomly assigned to either a retrospective pretest (referred to as post‐then) or a conventional pretest–posttest group. Findings indicated that participants in the post‐then group reported a greater change in knowledge compared to the conventional group. This finding supported the use of the retrospective pretest as a tool for getting usable, indirect evidence of student learning. Future research needs to examine response effect, alternative ways of calculating treatment effect and the confounding influences of other perceptual variables.     

Profiles of Motivated Self-Regulation in College Computer Science Courses: Differences in Major versus Required Non-Major Courses

The goal of the present study was to utilize a profiling approach to understand differences in motivation and strategic self-regulation among post-secondary STEM students in major versus required non-major computer science courses. Participants were 233 students from required introductory computer science courses (194 men; 35 women; 4 unknown) at a large Midwestern state university. Cluster analysis identified five profiles: (1) a strategic profile of a highly motivated by-any-means good strategy user; (2) a knowledge-building profile of an intrinsically motivated autonomous, mastery-oriented student; (3) a surface learning profile of a utility motivated minimally engaged student; (4) an apathetic profile of an amotivational disengaged student; and (5) a learned helpless profile of a motivated but unable to effectively self-regulate student. Among CS majors and students in courses in their major field, the strategic and knowledge-building profiles were the most prevalent. Among non-CS majors and students in required non-major courses, the learned helpless, surface learning, and apathetic profiles were the most prevalent. Students in the strategic and knowledge-building profiles had significantly higher retention of computational thinking knowledge than students in other profiles. Students in the apathetic and surface learning profiles saw little instrumentality of the course for their future academic and career objectives. Findings show that students in STEM fields taking required computer science courses exhibit the same constellation of motivated strategic self-regulation profiles found in other post-secondary and K-12 settings.     

Problem-solving in a case-based course: Strategies for facilitating coached expertise

This paper proposes the use of specific coaching strategies to facilitate student use of expert-like problem-solving strategies while analyzing and solving instructional design case studies. Findings from an exploratory study, designed to examine changes in students' problem-solving skills as they analyzed case studies, suggested that students could show expert characteristics at times, under some circumstances, but did not perform like experts on a regular basis. At two midwestern universities, 37 students analyzed 6 to 10 case studies both in class and in on-line discussions. Comparisons were made both within and across students, as well as across time, to examine patterns and changes in student problem-solving approaches. Findings suggested that primary influences on the incidence of expert performance were more external than internal and might be more aptly characterized as “coached expertise”. Specific suggestions are included for coaching the development of student problem-solving skills within a case-based course.     

Multimodal learning analytics for game-based learning

A distinctive feature of game-based learning environments is their capacity to create learning experiences that are both effective and engaging. Recent advances in sensor-based technologies such as facial expression analysis and gaze tracking have introduced the opportunity to leverage multimodal data streams for learning analytics. Learning analytics informed by multimodal data captured during students’ interactions with game-based learning environments hold significant promise for developing a deeper understanding of game-based learning, designing game-based learning environments to detect maladaptive behaviors and informing adaptive scaffolding to support individualized learning. This paper introduces a multimodal learning analytics approach that incorporates student gameplay, eye tracking and facial expression data to predict student posttest performance and interest after interacting with a game-based learning environment, Crystal Island . We investigated the degree to which separate and combined modalities (ie, gameplay, facial expressions of emotions and eye gaze) captured from students (n = 65) were predictive of student posttest performance and interest after interacting with Crystal Island . Results indicate that when predicting student posttest performance and interest, models utilizing multimodal data either perform equally well or outperform models utilizing unimodal data. We discuss the synergistic effects of combining modalities for predicting both student interest and posttest performance. The findings suggest that multimodal learning analytics can accurately predict students’ posttest performance and interest during game-based learning and hold significant potential for guiding real-time adaptive scaffolding.