The Benefits of Fixed Item Parameter Calibration for Parameter Accuracy in Small Sample Situations in Large‐Scale Assessments

Large‐scale assessments such as the Programme for International Student Assessment (PISA) have field trials where new survey features are tested for utility in the main survey. Because of resource constraints, there is a trade‐off between how much of the sample can be used to test new survey features and how much can be used for the initial item response theory (IRT) scaling. Utilizing real assessment data of the PISA 2015 Science assessment, this article demonstrates that using fixed item parameter calibration (FIPC) in the field trial yields stable item parameter estimates in the initial IRT scaling for samples as small as n = 250 per country. Moreover, the results indicate that for the recovery of the county‐specific latent trait distributions, the estimates of the trend items (i.e., the information introduced into the calibration) are crucial. Thus, concerning the country‐level sample size of n = 1,950 currently used in the PISA field trial, FIPC is useful for increasing the number of survey features that can be examined during the field trial without the need to increase the total sample size. This enables international large‐scale assessments such as PISA to keep up with state‐of‐the‐art developments regarding assessment frameworks, psychometric models, and delivery platform capabilities.     

Persisting in Physics and the Physics Olympiad — Impact of Gender Identification and Sense of Belonging on Expectancy-Value Outcomes

The German Physics Olympiad is an extracurricular science contest for students. Here, they have the opportunity to compete against other talented students, can do physics outside of school, and take a first step to more engagement in the domain. Yet, female students participate in the competition in fewer numbers and are disproportionally more likely to drop out of the contest earlier than the male students.

The present study hence explored the question to which extent the German Physics Olympiad provides a threatening environment for female contestants’ intentions of persisting in physics. A total of 298 participants (28% female) were surveyed with respect to stereotype and social identity threat, as well as gender identification and sense of belonging as predictors of success expectations for and value of choosing to study physics. Success expectations and value of choosing to study physics were used as a measure for career intentions within the expectancy-value model.

The results support the conclusion that the contest presents an equally supporting environment for female and male participants. We found no gender differences in success expectations for and value of studying physics. Sense of belonging and gender identification significantly predicted success expectations but not value of choosing to study physics. Female participants in the German Physics Olympiad were also neither affected by stereotype threat nor by social identity threat in their sense of belonging or gender identification.

     

Pedagogical content knowledge of experts and novices—what knowledge do they activate when analyzing science lessons?

In the current debate on pedagogical content knowledge (PCK), the term is used to refer to the context‐specific knowledge that teachers activate when reflecting on practice. Against the background of this debate, we conducted an empirical study and sought to answer the question of which knowledge experts and novices activated in assessing a videotaped lesson in relation to its effectiveness for learning. Our assumption was that the participants activate their PCK as a blending of content knowledge (CK) and pedagogical knowledge (PK) as suggested by Shulman's amalgam thesis. The participants (9 experts and 9 novices) were shown a lesson on optics, in which the law of refraction (Snell's law) was being studied. In a subsequent interview, the participants were asked to analyze the observed lesson. A qualitative and quantitative evaluation of the interviews showed that experts activated both CK and PK intensively and in this respect they differed significantly from novices. Further analysis of the expert statements also proved that they do not activate their CK and PK in isolation, but instead combine both kinds of knowledge together, in line with Shulman's amalgam thesis. © 2017 Wiley Periodicals, Inc. J Res Sci Teach 55: 44–67, 2018     

Lesen Mädchen anders?

Zusammenfassung Die Gender bezogene Forschung der letzten Jahre setzte sich insbesondere mit Differenzen in mathematischen und naturwissenschaftlichen Kompetenzen auseinander, w?hrend die Unterschiede im Leseverst?ndnis weniger Beachtung fanden. Dabei sind national wie auch international die Befunde von gro? angelegten Schulvergleichsstudien konsistent: M?dchen lernen schneller und besser lesen, und auch wenn die Jungen in der Sekundarstufe aufholen, so erreichen sie den Stand der M?dchen auch im Jugendalter noch nicht. Anhand der 2001 durchgeführten Internationalen Grundschul-Lese-Untersuchung (IGLU) wird untersucht, ob der Vorsprung der M?dchen im Leseverst?ndnis schon in der Grundschule angelegt ist. Anhand differenzieller Item Analysen wird der Frage nachgegangen, ob sich auch bei einzelnen Fragen systematische Unterschiede zwischen M?dchen und Jungen nachweisen lassen. Anhand der Analysen kann gezeigt werden, dass geringe Geschlechtsdifferenzen bezüglich des Frageformats (offenes Antwortformat vs. Multiple Choice) und den Leseleistungen bei literarischen und Informationstexten bestehen. Bei Betrachtung der in IGLU getesteten Verstehensaspekte ergeben sich keine Differenzen zwischen Jungen und M?dchen. Hingegen ist bezüglich der Aufgabenschwierigkeiten ein Zusammenhang mit geschlechtsspezifischen L?sungsh?ufigkeiten festzustellen, was ?ltere Befunde stützt, die gezeigt haben, dass M?dchen routinierter lesen. Als m?gliche Konsequenz dieser Untersuchung k?nnte die Anregung gegeben werden, im Unterricht vermehrt Leseanreize zu geben, welche Jungen eher ansprechen, um sie so zum vermehrten Lesen zu führen. In sp?teren Untersuchungen w?re dann zu kl?ren, ob die Jungen so mehr Sicherheit bei routinierten Leseaufgaben erreichen k?nnten und ggf. zu dem von M?dchen gezeigten Niveau des Leseverst?ndnisses aufschlie?en k?nnten.

---

Summary Over the last few years research has particularly concerned itself with gender differences between competencies in mathematics and natural sciences, whilst differences in reading comprehension have had little attention. At the same time, national and international evidence from large-scale school comparisons has shown consistently that girls learn to read faster and better. Even when boys catch up at secondary level, they do not reach the girl’ standard in their teens. On the basis of the international reading study IGLU carried out in 2001, this contribution will ask whether the girl’ head-start can already be observed at primary school level. Using differential item-analysis, the question of whether answers to individual questions show systematic differences will be investigated. The analysis shows only small gender differences regarding question format (open questions vs. multiple choice) and reading performance for literary and informational texts. Also, no differences between boys and girls can be found in the aspects of comprehension tested for in the IGLU-study. However, there is a connection between the level of task difficulty and the frequency of solving tasks by gender, which supports previous evidence that girls read more proficiently. A possible consequence of this study could be the using of reading incentives in class, which are particularly aimed at boys, to encourage their reading. Further studies would have to investigate whether boys achieve an improved confidence in tasks calling for reading proficiency and are therefore able to reach the standards of reading comprehension set by the girls.

     

Does the accuracy matter? Accurate concept map feedback helps students improve the cohesion of their explanations

Students are often challenged by the demand of writing cohesive explanatory texts. Prior research has shown that providing students with concept map feedback that visualizes explanatory cohesion deficits helped students generate more cohesive explanations. We conducted an experiment to investigate whether the accuracy of the provided information within the concept map feedback affected students’ improvements of cohesion. Accordingly, we varied the represented accuracy of information within such concept maps: Students either received accurate concept map feedback that depicted the real relations between concepts, as well as the authentic cohesion gaps in their explanations, or students received inaccurate concept map feedback, which depicted randomly drawn relations and random cohesion gaps. Additionally, in a baseline condition, students did not receive any feedback. We found that the students in the accurate feedback condition generated more cohesive explanations than the students in the no-feedback condition, whereas the students in the inaccurate feedback condition lay in-between. Evidently, providing feedback in general can be regarded as beneficial to enhance students’ writing. However, the accuracy of the provided feedback further impacts the effectiveness of computer-generated concept maps.     

Outcomes of family group conferencing in Sweden. A 3-year follow-up

OBJECTIVE: Between 1995 and 1997, the Swedish Association of Local Authorities implemented Family Group Conferences (FGC) in 10 local authorities throughout Sweden. This study reports on client outcomes of this implementation. METHOD: 97 children involved in 66 FGCs between November 1996 and October 1997 were compared with 142 children from a random sample of 104 traditional child protection investigations by the Child Protective Services (CPS). All children were followed for exactly 3 years for future child maltreatment events reported to CPS. Effects were modeled using multiple regressions, controlling for the child's age, gender, family background, and type and severity of problems. RESULTS: After controlling for initial differences, FGC-children experienced higher rates of re-referral to CPS compared to the group that had been processed in traditional investigations. They were more often re-referred due to abuse, were more often re-referred by the extended family, were longer in out-of-home placements, but tended over time to get less intrusive support from the CPS. FGCs were not related to re-referrals of neglect, of case-closure after 3 years or number of days of received services. The results suggest that the impact of the FGC was scant, accounting for 0-7% of the statistical variance of outcome variables. CONCLUSIONS: The findings did not support the alleged effectiveness of the FGC model compared to traditional investigations in preventing future maltreatment cases. If these results are confirmed in future research, they serve as a reminder of the necessity to evaluate models based on untested theories or on extrapolations from other countries/cultures, before these models are widely spread in a national practice context.