Is It the Intervention or the Students? Using Linear Regression to Control for Student Characteristics in Undergraduate STEM Education Research

Although researchers in undergraduate science, technology, engineering, and mathematics education are currently using several methods to analyze learning gains from pre- and posttest data, the most commonly used approaches have significant shortcomings. Chief among these is the inability to distinguish whether differences in learning gains are due to the effect of an instructional intervention or to differences in student characteristics when students cannot be assigned to control and treatment groups at random. Using pre- and posttest scores from an introductory biology course, we illustrate how the methods currently in wide use can lead to erroneous conclusions, and how multiple linear regression offers an effective framework for distinguishing the impact of an instructional intervention from the impact of student characteristics on test score gains. In general, we recommend that researchers always use student-level regression models that control for possible differences in student ability and preparation to estimate the effect of any nonrandomized instructional intervention on student performance.     

Sense of belonging or feeling marginalized? Using PISA 2012 to assess the state of academically gifted students within the EU

There are two competing stereotypes of gifted students: harmony theory (gifted students are well adjusted and successful in life) and disharmony theory (giftedness forms a threat to a harmonious development). In this context, the PISA 2012 data were used to explore middle-school students’ experiences in terms of sense of belonging, student–teacher relations and attitudes toward school concerning learning activities/outcomes. Fifteen-year-old students from 13 European countries were selected for this data-set (normative = 79,550, gifted = 1956). Student’s scores on the four scales were tested for significant differences with students from that same country. Tests revealed no significant differences for 55% of the comparisons, 40% of comparisons had positive effect sizes for gifted students, and 4% had negative effect sizes. The evidence presented in this study supports the existence of harmony theory. More specifically, the vast majority of gifted students in this study reported equal or higher level of belonging. Contained within the findings in this study on students’ to the sense of belonging is a lack of evidence of gifted students reporting higher levels of loneliness. These findings strongly reflect the Terman’s assertion almost a century ago stating the lack of gifted children tending to be more socially maladjusted.     

Does Environmental Education have ‘Street Credibility’ and the Potential to Reduce Pupil Disaffection Within and Beyond their School Curriculum?

Pupils express their disaffection with the school curriculum through their disruptive behaviour, disengagement from the learning process and through under‐achievement in their work. Given that the majority of pupils are in school, in body if not always in mind, then does learning about the environment switch pupils on to learning? This paper considers pupils’ and teachers’ perceptions of the effect that environmental education has on pupils’ attitudes to learning, on their ability to demonstrate positive achievement, on their involvement and cooperation in lessons, on their behaviour and their attitude to school, their community and the world outside school.     

Should students design or interact with models? Using the Bifocal Modelling Framework to investigate model construction in high school science

The Bifocal Modelling Framework (BMF) is an approach for science learning which links students’ physical experimentation with computer modelling in real time, focusing on the comparison of the two media. In this paper, we explore how a Bifocal Modelling implementation supported learning outcomes related to both content and metamodeling knowledge, focusing on the role of designing models. Our study consisted of three conditions implemented with a total of 69 9th grade high-school students. The first and second classes were assigned two implementation modes of BMF: with and without a model design module. The third condition, employed as a control, consisted of a class that received instruction in the school’s traditional approach. Our results indicate that students participating in both BMF implementations demonstrated improved content knowledge and a better understanding of metamodeling. However, only the ‘BMF-with-design’ group improved significantly in both content and metamodeling knowledge. Our qualitative analyses indicate that both BMF groups designed detailed models that included scientific explanations. However only students who engaged in the model design component: (1) completed a detailed model displaying molecular interaction; and (2) developed a critical perspective about models. We discuss the implications of those results for teaching scientific science concepts and metamodeling knowledge.     

Does it pay to attend an elite private college? Evidence on the effects of undergraduate college quality on graduate school attendance

Much attention has recently focused on the rapidly rising costs of a college education, and whether the benefits of attending an elite private college have kept pace with the increasing costs. In this paper we analyze whether undergraduate college quality affects the likelihood that an individual attends graduate school. Using data on three cohorts of students from the National Longitudinal Study of the High School Class of 1972 and High School and Beyond, we find that on balance attendance at an elite private college significantly increases the probability of attending graduate school, and more specifically, graduate school at a major research institution.     

Do Students Know What They Know and What They Don’t Know? Using a Four-Tier Diagnostic Test to Assess the Nature of Students’ Alternative Conceptions

This study reports on the development and application of a four-tier multiple-choice (4TMC) diagnostic instrument, which has not been reported in the literature. It is an enhanced version of the two-tier multiple-choice (2TMC) test. As in 2TMC tests, its answer and reason tiers measure students’ content knowledge and explanatory knowledge, respectively. The two additional tiers measure the level of confidence of students in the correctness of their chosen options for the answer and reason tiers respectively. The 4TMC diagnostic test focused on the properties and propagation of mechanical waves. It was administered to 598 upper secondary students after they were formally instructed on the foregoing topics. The vast majority of the respondents were found to have an inadequate grasp of the topics tested. Mean scores and mean confidence associated with the answer tier was higher than those associated with the reason tier. The students tended to be poorly discriminating between what they know and what they do not know. Familiarity with the topic tested was associated with greater percentage of students giving correct answers, higher confidence, and better discrimination quotient. Nine genuine alternative conceptions (which were expressed with moderate levels of confidence by students) were identified.     

Local Equating Using the Rasch Model,the OPLM,and the 2PL IRT Model—or—What Is It Anyway if the Model Captures Everything There Is to Know About the Test Takers?

Local equating (LE) is based on Lord's criterion of equity. It defines a family of true transformations that aim at the ideal of equitable equating. van der Linden (this issue) offers a detailed discussion of common issues in observed‐score equating relative to this local approach. By assuming an underlying item response theory model, one of the main features of LE is that it adjusts the equated raw scores using conditional distributions of raw scores given an estimate of the ability of interest. In this article, we argue that this feature disappears when using a Rasch model for the estimation of the true transformation, while the one‐parameter logistic model and the two‐parameter logistic model do provide a local adjustment of the equated score.