The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments

Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities.     

Pedagogical content knowledge of experts and novices—what knowledge do they activate when analyzing science lessons?

In the current debate on pedagogical content knowledge (PCK), the term is used to refer to the context‐specific knowledge that teachers activate when reflecting on practice. Against the background of this debate, we conducted an empirical study and sought to answer the question of which knowledge experts and novices activated in assessing a videotaped lesson in relation to its effectiveness for learning. Our assumption was that the participants activate their PCK as a blending of content knowledge (CK) and pedagogical knowledge (PK) as suggested by Shulman's amalgam thesis. The participants (9 experts and 9 novices) were shown a lesson on optics, in which the law of refraction (Snell's law) was being studied. In a subsequent interview, the participants were asked to analyze the observed lesson. A qualitative and quantitative evaluation of the interviews showed that experts activated both CK and PK intensively and in this respect they differed significantly from novices. Further analysis of the expert statements also proved that they do not activate their CK and PK in isolation, but instead combine both kinds of knowledge together, in line with Shulman's amalgam thesis. © 2017 Wiley Periodicals, Inc. J Res Sci Teach 55: 44–67, 2018     

Does the accuracy matter? Accurate concept map feedback helps students improve the cohesion of their explanations

Students are often challenged by the demand of writing cohesive explanatory texts. Prior research has shown that providing students with concept map feedback that visualizes explanatory cohesion deficits helped students generate more cohesive explanations. We conducted an experiment to investigate whether the accuracy of the provided information within the concept map feedback affected students’ improvements of cohesion. Accordingly, we varied the represented accuracy of information within such concept maps: Students either received accurate concept map feedback that depicted the real relations between concepts, as well as the authentic cohesion gaps in their explanations, or students received inaccurate concept map feedback, which depicted randomly drawn relations and random cohesion gaps. Additionally, in a baseline condition, students did not receive any feedback. We found that the students in the accurate feedback condition generated more cohesive explanations than the students in the no-feedback condition, whereas the students in the inaccurate feedback condition lay in-between. Evidently, providing feedback in general can be regarded as beneficial to enhance students’ writing. However, the accuracy of the provided feedback further impacts the effectiveness of computer-generated concept maps.     

Follow-up of a preschool epidemiological sample: cross-age continuities and predictions of later adjustment with internalizing and externalizing dimensions of behavior

The continuity of behavioral adjustment from preschool through elementary school and junior high school years was examined. 541 children aged 9-15 years from a preschool epidemiological study were relocated and behavior checklist data obtained. Analyses focused on the relationship between internalizing and externalizing behavior dimensions from the preschool to follow-up periods. It was found that preschool externalizing symptoms were positively correlated with later externalizing and internalizing symptoms in the entire sample. Preschool internalizing symptoms, however, were predictive of later internalizing symptoms only for 2-year-old girls and 5- and 6-year-old boys. Similar results were obtained for clinically disturbed preschoolers. Results are consistent with previous findings regarding the longitudinal continuity of externalizing behavior but are at odds with reports of equal or greater stability of internalizing behavior. Given the magnitude of obtained correlations, even when significant, the results support the view that discontinuity rather than continuity in behavioral adjustment from preschool to later ages is the rule. The importance of examining other mediating variables in the prediction of behavioral adjustment and the need for models of development that encompass both stability and change are discussed.