Development and evaluation of the conceptual inventory of natural selection

Natural selection as a mechanism of evolution is a central concept in biology; yet, most nonbiology‐majors do not thoroughly understand the theory even after instruction. Many alternative conceptions on this topic have been identified, indicating that the job of the instructor is a difficult one. This article presents a new diagnostic test to assess students' understanding of natural selection. The test items are based on actual scientific studies of natural selection, whereas previous tests have employed hypothetical situations that were often misleading or oversimplified. The Conceptual Inventory of Natural Selection (CINS) is a 20‐item multiple choice test that employs common alternative conceptions as distractors. An original 12‐item version of the test was field‐tested with 170 nonmajors in 6 classes and 43 biology majors in 1 class at 3 community colleges. The test scores of one subset of nonmajors (n = 7) were compared with the students' performances in semistructured interviews. There was a positive correlation between the test scores and the interview scores. The current 20‐item version of the CINS was field‐tested with 206 students in a nonmajors' general biology course. The face validity, internal validity, reliability, and readability of the CINS are discussed. Results indicate that the CINS will be a valuable tool for instructors. © 2002 Wiley Periodicals, Inc. J Res Sci Teach 39: 952–978, 2002     

A Resource for Eliciting Student Alternative Conceptions: Examining the Adaptability of a Concept Inventory for Natural Selection at the Secondary School Level

The Conceptual Inventory of Natural Selection (CINS) is an example of a research-based instrument that assesses conceptual understanding in an area that contains well-documented alternative conceptions. Much of the CINS’s use and original validation has been relegated to undergraduate settings, but the information learned from student responses on the CINS can also potentially be a useful resource for teachers at the secondary level. Because of its structure, the CINS can have a role in eliciting alternative conceptions and induce deeper conceptual understanding by having student ideas leveraged during instruction. In a first step toward this goal, the present study further investigated the CINS’s internal properties by having it administered to a group (n?=?339) of students among four different biology teachers at a predominantly Latino, economically disadvantaged high school. In addition, incidences of the concept inventory’s use among the teachers’ practices were collected for support of its adaptability at the secondary level. Despite the teachers’ initial enthusiasm for the CINS’s use as an assessment tool in the present study, results from a principal components analysis demonstrate inconsistencies between the original and present validations. Results also reveal how the teachers think CINS items may be revised for future use among secondary student populations.     

Feeling of certainty: Uncovering a missing link between knowledge and acceptance of evolution

We propose a new model of the factors influencing acceptance of evolutionary theory that highlights a novel variable unexplored in previous studies: the feeling of certainty (FOC). The model is grounded in an emerging understanding of brain function that acknowledges the contributions of intuitive cognitions in making decisions, such as whether or not to accept a particular theoretical explanation of events. Specifically, we examine the relationships among religious identity, level of education, level of knowledge, FOC, and level of evolutionary acceptance to test whether our proposed model accurately predicts hypothesized pathways. We employ widely used measures—the CINS, MATE, and ORI—in addition to new variables in multiple regression and path analyses in order to test the interrelationships among FOC and acceptance of evolutionary theory. We explore these relationships using a sample of 124 pre‐service biology teachers found to display comparable knowledge and belief levels as reported in previous studies on this topic. All of our hypothesis tests corroborated the idea that FOC plays a moderating role in relationships among evolutionary knowledge and beliefs. Educational research into acceptance of evolutionary theory will likely benefit from increased attention to non‐conscious intuitive cognitions that give rise to feeling of knowing or certainty. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 49: 95–121, 2012     

The two-tier instrument on photosynthesis: What does it diagnose?

The goal of this study was to understand how six college biology students complete the tasks of a traditional paper and pencil instrument designed to detect alternative conceptions about photosynthesis. Participants responded to relevant items in a two-tier diagnostic instrument in a think-aloud task. Responses to the traditional content question (first tier) were correct more often than reasons (second tier). However the participants' verbal data indicated that they relied upon test-taking strategies, not retrieval from memory, to choose their reasons. Some distractors caused participants to accept incorrect propositions being considered for the first time (rather than eliciting a misconception from extant knowledge). They also considered relevant exceptions and subtle language cues that justified their choices of incorrect reasons. Participants voiced concerns about the conscientiousness with which students complete such instruments. These findings raise concerns about the validity of using such instruments for diagnosing alternative conceptions.     

Evaluation of Students' Conceptual Understanding of Malaria

In this study, a two‐tier diagnostic test for understanding malaria was developed and administered to 314 Bruneian students in Year 12 and in a nursing diploma course. The validity, reliability, difficulty level, discriminant indices, and reading ability of the test were examined and found to be acceptable in terms of measuring students' understanding and identifying alternative conceptions with respect to malaria. Results showed that students' understanding of malaria was high for content, low for reasons, and limited and superficial for both content and reasons. The instrument revealed several common alternative conceptual understandings students' hold about malaria. The MalariaTT2 instrument developed could be used in classroom lessons for challenging alternative conceptions and enhancing conceptions of malaria.     

An analysis of 16–17-year-old students' understanding of solution chemistry concepts using a two-tier diagnostic instrument

This study focused on the development of a two-tier multiple-choice diagnostic instrument, which was designed and then progressively modified, and implemented to assess students' understanding of solution chemistry concepts. The results of the study are derived from the responses of 756 Grade 11 students (age 16–17) from 14 different high schools who participated in the study. The final version of the instrument included a total of 13 items that addressed the six aspects of solution chemistry, and students' understandings in the test were challenged in multiple contexts with multiple modes and levels of representation. Cronbach alpha reliability coefficients for the content tier and both tiers of the test were found to be 0.697 and 0.748, respectively. Results indicated that a substantial number of students held an inadequate understanding of solution chemistry concepts. In addition, 21 alternative conceptions observed in more than 10% of the students were reported, along with discussion on possible sources of such conceptions.     

Confidence versus Performance as an Indicator of the Presence of Alternative Conceptions and Inadequate Problem‐Solving Skills in Mechanics

This study investigated the use of performance–confidence relationships to signal the presence of alternative conceptions and inadequate problem‐solving skills in mechanics. A group of 33 students entering physics at a South African university participated in the project. The test instrument consisted of 20 items derived from existing standardised tests from literature, each of which was followed by a self‐reported measure of confidence of students in the correctness of their answers. Data collected for this study included students’ responses to multiple‐choice questions and open‐ended explanations for their chosen answers. Fixed response physics and confidence data were logarithmically transformed according to the Rasch model to linear measures of performance and confidence. The free response explanations were carefully analysed for accuracy of conceptual understanding. Comparison of these results with raw score data and transformed measures of performance and confidence allowed a re‐evaluation of the model developed by Hasan, Bagayoko, and Kelley in 1999 for the detection of alternative conceptions in mechanics. Application of this model to raw score data leads to inaccurate conclusions. However, application of the Hasan hypothesis to transformed measures of performance and confidence resulted in the accurate identification of items plagued by alternative conceptions. This approach also holds promise for the differentiation between over‐confidence due to alternative conceptions or due to inadequate problem‐solving skills. It could become a valuable tool for instructional design in mechanics.     

Development and Application of a Three‐Tier Diagnostic Test to Assess Secondary Students’ Understanding of Waves

This study focused on the development and application of a three‐tier multiple‐choice diagnostic test (or three‐tier test) on the nature and propagation of waves. A question in a three‐tier test comprises the content tier, which measures content knowledge; the reason tier, which measures explanatory knowledge; and the confidence tier, which measures the strength of conceptual understanding of the respondents. This paper presents results based on the responses of 243 Grade 10 students after they were formally instructed on the topic. The vast majority of the respondents showed an inadequate grasp of concepts about waves. Eleven alternative conceptions (ACs), which were expressed with confidence by more than 10% of the students, were identified; four of these ACs were expressed with high confidence.     

Chinese pre-service biology teachers’ evolutionary knowledge,reasoning patterns,and acceptance levels

The goal of our study was to examine a large (>400), cross-sectional sample of Chinese pre-service biology teachers (PBTs) in order to document their evolution acceptance levels, evolution knowledge, and evolutionary reasoning patterns. This approach was taken in order to better understand the degree to which particular evolutionary reasoning difficulties exist independent of religious worldviews. The sample included (1) 160 PBTs tasked with completing four items from the ACORNS instrument, (2) 320 PBTs who completed the CINS and MATE instruments, and (3) 32 teachers who completed semi-structured clinical interviews using four ACORNS items. Findings from these samples revealed that Chinese PBTs’ knowledge (CINS) and acceptance (MATE) were equivalent with teachers’ scores from other countries, whereas performance on explanation tasks was lower. Scores from the CINS, MATE, and ACORNS did not reveal any significant improvements through the four-year teacher education programme. Although a large body of work has shown the important roles that religious affiliation and religiosity can play in evolutionary understanding and acceptance, our findings demonstrate that many evolutionary reasoning difficulties extend beyond religious factors, and add to a growing body of work showing that religiosity does not adequately account for PBTs’ moderate evolution acceptance.     

Conceptual Inventory of Natural Selection as a Tool for Measuring Greek University Students' Evolution Knowledge: Differences between novice and advanced students

The primary objective of this research was to compare various groups of Greek university students for their level of knowledge of Evolution by means of Natural Selection (ENS). For the purpose of the study, we used a well known questionnaire the Conceptual Inventory of Natural Selection (CINS) and 352 biology majors and non-majors students from the University of Athens took part in it. A principal components analysis revealed problems with the items designed to assess the concepts of population stability, differential survival and variation inheritable, therefore these items need to be reconsidered. Nonetheless, the results of the CINS for each Greek sub-group showed that the higher the involvement in evolution education, the higher the students' performances on the CINS test. This linear correlation, together with other evidence, supports the CINS authors' claims about the usefulness of the CINS as an assessment of instruction. Unfortunately, Greek university students gave many teleological and proximate answers to many of the CINS items. Comparisons between least and most evolutionary educated university students revealed that the latter gave more evolutionary answers. Oddly, advanced biology majors students did not show an improvement in all the 20 items of the CINS (only in 14 out of the 20 items) compared to novice biology students. They even gave more teleological answers to the concept natural resources are limited than novice biology majors students. Finally, Greek university students' level of knowledge of ENS seems to be closer to Canadian than US students'.     

The Dominance Concept Inventory: A Tool for Assessing Undergraduate Student Alternative Conceptions about Dominance in Mendelian and Population Genetics

Despite the impact of genetics on daily life, biology undergraduates understand some key genetics concepts poorly. One concept requiring attention is dominance, which many students understand as a fixed property of an allele or trait and regularly conflate with frequency in a population or selective advantage. We present the Dominance Concept Inventory (DCI), an instrument to gather data on selected alternative conceptions about dominance. During development of the 16-item test, we used expert surveys (n = 12), student interviews (n = 42), and field tests (n = 1763) from introductory and advanced biology undergraduates at public and private, majority- and minority-serving, 2- and 4-yr institutions in the United States. In the final field test across all subject populations (n = 709), item difficulty ranged from 0.08 to 0.84 (0.51 ± 0.049 SEM), while item discrimination ranged from 0.11 to 0.82 (0.50 ± 0.048 SEM). Internal reliability (Cronbach''s alpha) was 0.77, while test–retest reliability values were 0.74 (product moment correlation) and 0.77 (intraclass correlation). The prevalence of alternative conceptions in the field tests shows that introductory and advanced students retain confusion about dominance after instruction. All measures support the DCI as a useful instrument for measuring undergraduate biology student understanding and alternative conceptions about dominance.     

Developing,Evaluating and Demonstrating the Use of a Conceptual Survey in Mechanical Waves

An understanding of mechanical waves is a pre‐requisite for the study of many topics in advanced physics, and indeed in many other disciplines. There have been many research studies in mechanical waves, all of which have revealed that students have trouble with the basic concepts. Therefore, in order for teachers to prepare appropriate instruction for their classes, it is useful to diagnose their students' conceptions—if possible before they enter class. It is for this purpose that many diagnostic instruments have been developed, often in the form of multiple‐choice tests. In this study, we have used the open‐ended Wave Diagnostic Test to develop a multiple‐choice conceptual survey in an evolutionary manner. The two‐year development procedure included open‐ended surveys and interviews involving 299 Thai students and 88 Australian students. The final version, called the Mechanical Waves Conceptual Survey, has been administered to 632 Australian students from high school to second‐year university and 270 Thai high school students. Standard statistical analyses show that the survey is reliable and valid. Further validity checks, including consultation with experts, were also carried out. The survey has four subtopics—propagation, superposition, reflection, and standing waves—and the teachers can choose the subtopics relevant for their students. In this paper we also demonstrate the use of a typical survey question to test students' conceptual understanding and identify common alternative conceptions.     

What Do You Know about Alternative Energy? Development and Use of a Diagnostic Instrument for Upper Secondary School Science

The need for renewable and non-fossil fuels is now recognised by nations throughout the world. Consequently, an understanding of alternative energy is needed both in schools and in everyday life-long learning situations. This study developed a two-tier instrument to diagnose students' understanding and alternative conceptions about alternative energy in terms of: sources of alternative energy, greenhouse gas emission, as well as advantages, and disadvantages. Results obtained with Years 10 and 11 students (n?=?491) using the 12-item two-tier instrument (α?=?0.61) showed that students' understanding of alternative energy was low (M?=?7.03; SD?=?3.90). The 23 alternative conceptions about alternative energy sources that could be identified from the instrument are reported. The implications for teaching and learning about alternative energy and suggestions for further development and improvement of the instrument are presented.     

Development and Evaluation of the Diagnostic Power for a Computer-Based Two-Tier Assessment

This study adopted a quasi-experimental design with follow-up interview to develop a computer-based two-tier assessment (CBA) regarding the science topic of electric circuits and to evaluate the diagnostic power of the assessment. Three assessment formats (i.e., paper-and-pencil, static computer-based, and dynamic computer-based tests) using two-tier items were conducted on Grade 4 (n = 90) and Grade 5 (n = 86) students, respectively. One-way ANCOVA was conducted to investigate whether the different assessment formats affected these students’ posttest scores on both the phenomenon and reason tiers, and confidence rating for an answer was assessed to diagnose the nature of students’ responses (i.e., scientific answer, guessing, alternative conceptions, or knowledge deficiency). Follow-up interview was adopted to explore whether and how the various CBA representations influenced both graders’ responses. Results showed that the CBA, in particular the dynamic representation format, allowed students who lacked prior knowledge (Grade 4) to easily understand the question stems. The various CBA representations also potentially encouraged students who already had learning experience (Grade 5) to enhance the metacognitive judgment of their responses. Therefore, CBA could reduce students’ use of test-taking strategies and provide better diagnostic power for a two-tier instrument than the traditional paper-based version.     

Just do it? impact of a science apprenticeship program on high school students' understandings of the nature of science and scientific inquiry

The purpose of this study was to explicate the impact of an 8‐week science apprenticeship program on a group of high‐ability secondary students' understandings of the nature of science and scientific inquiry. Ten volunteers (Grades 10–11) completed a modified version of the Views of Nature of Science, Form B both before and after their apprenticeship to assess their conceptions of key aspects of the nature of science and scientific inquiry. Semistructured exit interviews provided an opportunity for students to describe the nature of their apprenticeship experiences and elaborate on their written questionnaire responses. Semistructured exit interviews were also conducted with the scientists who served as mentors for each of the science apprentices. For the most part, students held conceptions about the nature of science and scientific inquiry that were inconsistent with those described in current reforms. Participating science mentors held strong convictions that their apprentices had learned much about the scientific enterprise in the course of doing the science in their apprenticeship. Although most students did appear to gain knowledge about the processes of scientific inquiry, their conceptions about key aspects of the nature of science remained virtually unchanged. Epistemic demand and reflection appeared to be crucial components in the single case where a participant experienced substantial gains in her understandings of the nature of science and inquiry. © 2003 Wiley Periodicals, Inc. J Res Sci Teach 40: 487–509, 2003     

Conceptualisation,Development and Validation of an Instrument for Investigating the Metacognitive Orientation of Science Classroom Learning Environments: The Metacognitive Orientation Learning Environment Scale – Science (MOLES-S)

Metacognition refers to an individual's knowledge, control and awareness of his/her learning processes. An important goal of education is to develop students as metacognitive, life-long learners. However, developing students' metacognition and evaluating whether classrooms are oriented to the development of students' metacognition are difficult and often time-consuming tasks. Further, no instruments that measure key dimensions related to classroom factors that specifically influence the development of students' metacognition have been available. This article describes the conceptualisation, design, and validation of an instrument for evaluating the metacognitive orientation of science classroom learning environments. The metacognitive orientation of a learning environment is the extent to which that environment supports the development and enhancement of students' metacognition. Social constructivism was the guiding referent informing the instrument's orientation and development. This instrument measures students' perceptions of the extent to which certain psychosocial dimensions, evident in learning environments where interventions have resulted in enhanced student metacognition, are evident in their science classrooms. Findings from the use of this instrument complement what is already known from research studies to be generally the case in relation to science classrooms' metacognitive orientation. This revised version was published online in July 2006 with corrections to the Cover Date.     

Measuring instructional practice in science using classroom artifacts: lessons learned from two validation studies

With growing interest in the role of teachers as the key mediators between educational policies and outcomes, the importance of developing good measures of classroom processes has become increasingly apparent. Yet, collecting reliable and valid information about a construct as complex as instruction poses important conceptual and technical challenges. This article summarizes the results of two studies that investigated the properties of measures of instruction based on a teacher‐generated instrument (the Scoop Notebook) that combines features of portfolios and self‐report. Classroom artifacts and teacher reflections were collected from samples of middle school science classrooms and rated along 10 dimensions of science instruction derived from the National Science Education Standards; ratings based on direct classroom observations were used as comparison. The results suggest that instruments that combine artifacts and self‐reports hold promise for measuring science instruction with reliability similar to, and sizeable correlations with, measures based on classroom observation. We discuss the implications and lessons learned from this work for the conceptualization, design, and use of artifact‐based instruments for measuring instructional practice in different contexts and for different purposes. Artifact‐based instruments may illuminate features of instruction not apparent even through direct classroom observation; moreover, the process of structured collection and reflection on artifacts may have value for professional development. However, their potential value and applicability on a larger scale depends on careful consideration of the match between the instrument and the model of instruction, the intended uses of the measures, and the aspects of classroom practice most amenable to reliable scoring through artifacts. We outline a research agenda for addressing unresolved questions and advancing theoretical and practical knowledge around the measurement of instructional practice. © 2011 Wiley Periodicals, Inc. J Res Sci Teach 49: 38–67, 2012     

A comparison of applied and theoretical knowledge of concepts based on the particulate nature of matter

This study compares 183 high school chemistry students' applied and theoretical knowledge of selected concepts based on the particulate theory. The concepts are dissolution, diffusion, effusion, and states of matter. A two-form instrument called the Physical Changes Concepts Test (PCCT) was developed for this study. The application form measures students' knowlege using everyday language. The theoretical form measures students' knowledge using scientific language. Students' formal reasoning ability was measured using the Test Of Logical Thinking (TOLT). The overall results of the two forms of the PCCT indicate that more than 40% of the students displayed alternative conceptions (ACs) of the concepts covered in the PCCT. The study found that students' formal reasoning ability and their preexisting knowledge are associated with their conceptions and use of the particulate theory. The analysis of the nature of students' ACs and their use of the particulate theory revealed a significant difference between students' applied and theoretical knowledge.     

Conceptions over time: Are language and the here-and-now up to the task?

Is it possible to explain students’ conceptions of natural phenomena purely in terms of the interactions between two people and the language they use during an interview? I argue that this hypothesis cannot be accepted on several grounds. First, contextual factors prior to the interview influence the course of its events, and that these in turn influence future events. Second, people have agency over their interactions and the ability to use language creatively in ways that a strong version of language preordination inherent in this hypothesis would not permit. Third, people bring language fluency and ideas to an interview that allow them to grapple with phenomena and issues they might not have previously considered, and formulate conceptions that they can and do use in future interactions. In addition, I argue that the field of science education is able to consider curricular and instructional issues relating to students’ conceptions without resorting to the extremes of cultural relativism or intellectual imperialism, and that conceptual change theory addresses both the processes and outcomes of students’ interactions.     

A stratified study of students’ understanding of basic optics concepts in different contexts using two‐tier multiple‐choice items

A large scale study involving 1786 year 7–10 Korean students from three school districts in Seoul was undertaken to evaluate their understanding of basic optics concepts using a two‐tier multiple‐choice diagnostic instrument consisting of four pairs of items, each of which evaluated the same concept in two different contexts. The instrument, which proved to be reliable, helped identify several context‐dependent alternative conceptions that were held by about 25% of students. At the same time, students’ performance on the diagnostic test correlated with the location of the schools, students’ achievement in school science and their attitudes to science learning. However, students’ grade levels had limited influence on their understanding of basic concepts in optics as measured by the instrument.