The Reliability of College Grades

Little is known about the reliability of college grades relative to how prominently they are used in educational research, and the results to date tend to be based on small sample studies or are decades old. This study uses two large databases (N > 800,000) from over 200 educational institutions spanning 13 years and finds that both first‐year and overall college GPA can be expected to be highly reliable measures of academic performance, with reliability estimated at .86 for first‐year GPA and .93 for overall GPA. Additionally, reliabilities vary moderately by academic discipline, and within‐school grade intercorrelations are highly stable over time. These findings are consistent with a hierarchical structure of academic ability. Practical implications for decision making and measurement using GPA are discussed.     

The Role of Competitive and Cooperative Norms in the Development of Deviant Evaluations

The present study examined how peer group norms influence children's evaluations of deviant ingroup members. Following the manipulation of competitive or cooperative norms, participants (children, M_age = 8.69; adolescents, M_age = 13.81; adults, M_age = 20.89; n = 263) evaluated deviant ingroup members from their own and the group's perspective. Children rated cooperative deviancy positively and believed their group would do the same. Adolescents and adults believed that their group would negatively evaluate cooperative deviancy when their group supported a competitive allocation strategy. Reasoning varied based on norm and participants’ agreement with deviancy. Understanding an ingroup may not be favorable toward a cooperative deviant in a competitive context is a developmental challenge requiring the coordination of social and moral norms.     

On the Superior Statistical Properties of Frequency Scales in Job Analyses

This study compared the statistical properties of four job analysis task survey response scale types: criticality, difficulty in learning, importance, and frequency. We used nine job analysis studies spanning two fields, medical imaging and allied health professionals, to compare the job analysis scales in terms of variability and interrater agreement. Results showed that frequency scales using absolute anchors had greater between-task variability and higher interrater agreement for all nine studies. This may have occurred due to what has been described by past research as self-presentation bias. In this case, an aggregate base percentage of respondents always responded that tasks in their domain are highly critical, highly important, and easy to learn. These results showed that frequency scales with absolute anchors yielded data with better statistical performance than other more subjective scales. These properties do not answer the question of whether a scale matches an exam's purpose, which is the most important consideration for job analyses. They do, however, suggest that, if statistics are a primary deciding factor, strong consideration should be given to using frequency scales with absolute anchors.     

Covariate Measurement Error Correction for Student Growth Percentiles Using the SIMEX Method

In this study, we examined the impact of covariate measurement error (ME) on the estimation of quantile regression and student growth percentiles (SGPs), and find that SGPs tend to be overestimated among students with higher prior achievement and underestimated among those with lower prior achievement, a problem we describe as ME endogeneity in this article. We proceeded to assess the effect of covariate ME correction on SGP estimation at two levels—the individual (student) and the aggregate (classroom). Our ME correction approach was limited to the simulation‐extrapolation method known as SIMEX. For both the individual and aggregate SGP, we find SIMEX effective in bias reduction. Further, because SIMEX is especially effective in reducing SGP bias for students with very high or very low prior achievement, it significantly weakens the ME endogeneity. SIMEX is also effective in reducing the MSE of aggregate SGP, provided that the students are sorted to some extent on their latent prior achievement. Our empirical study confirms the pattern of the simulation results: SIMEX mainly affects the mean SGP of classes in the highest and lowest quintiles of the prior score distribution, and significantly lowers the correlation between class SGP and prior achievement.     

Beyond the Central Dogma: Model-Based Learning of How Genes Determine Phenotypes

In an introductory biology course, we implemented a learner-centered, model-based pedagogy that frequently engaged students in building conceptual models to explain how genes determine phenotypes. Model-building tasks were incorporated within case studies and aimed at eliciting students’ understanding of 1) the origin of variation in a population and 2) how genes/alleles determine phenotypes. Guided by theory on hierarchical development of systems-thinking skills, we scaffolded instruction and assessment so that students would first focus on articulating isolated relationships between pairs of molecular genetics structures and then integrate these relationships into an explanatory network. We analyzed models students generated on two exams to assess whether students’ learning of molecular genetics progressed along the theoretical hierarchical sequence of systems-thinking skills acquisition. With repeated practice, peer discussion, and instructor feedback over the course of the semester, students’ models became more accurate, better contextualized, and more meaningful. At the end of the semester, however, more than 25% of students still struggled to describe phenotype as an output of protein function. We therefore recommend that 1) practices like modeling, which require connecting genes to phenotypes; and 2) well-developed case studies highlighting proteins and their functions, take center stage in molecular genetics instruction.