Null and Void? Errors in Meta-analysis on Perceptual Disfluency and Recommendations to Improve Meta-analytical Reproducibility

Educational Psychology Review - In the 2018 meta-analysis of Educational Psychology Review entitled “Null effects of perceptual disfluency on learning outcomes in a text-based educational...     

Beyond Graduation: Socio-economic Background and Post-university Outcomes of Australian Graduates

Research in Higher Education - Research consistently shows that higher-education participation has positive impacts on individual outcomes. However, few studies explicitly consider differences in...     

Evaluating item response theory linking and model fit for data from PISA 2000–2012

ABSTRACT

Based on concerns about the item response theory (IRT) linking approach used in the Programme for International Student Assessment (PISA) until 2012 as well as the desire to include new, more complex, interactive items with the introduction of computer-based assessments, alternative IRT linking methods were implemented in the 2015 PISA round. The new linking method represents a concurrent calibration using all available data, enabling us to find item parameters that maximize fit across all groups and allowing us to investigate measurement invariance across groups. Apart from the Rasch model that historically has been used in PISA operational analyses, we compared our method against more general IRT models that can incorporate item-by-country interactions. The results suggest that our proposed method holds promise not only to provide a strong linkage across countries and cycles but also to serve as a tool for investigating measurement invariance.     

Improving the judgment of task difficulties: prospective teachers’ diagnostic competence in the area of functions and graphs

To teach adaptively, teachers should be able to take the students’ level of knowledge into account. Therefore, a key component of pedagogical content knowledge (PCK) is the ability to assume the students’ perspectives. However, due to the so-called expert blind spot, teachers tend to misestimate their students’ knowledge, such as when estimating the difficulty of a given task. This empirically well-documented estimation bias is predicted by Nickerson’s anchoring and adjustment model, which generally explains how people take on other people’s perspectives. In this article, we present an intervention study that aims to improve the accuracy of prospective teachers’ judgments of task difficulty in the area of functional thinking. Two types of treatments are derived from Nickerson’s model. In the first condition (PCK group), participants acquire knowledge about task characteristics and students’ misconceptions. The second condition (sensitizing group) serves to control the idea that potential improvements in the PCK group are not merely based on a pure sensitization of the expert’s estimation bias. Accordingly, these participants are only informed about the general tendency of overestimating task difficulties. The results showed that the PCK group improved both in terms of the accuracy of the estimated solution rates and the accuracy of rank order, whereas the sensitizing group only improved in regard to the former. Thus, the study shows that prospective teachers’ diagnostic judgments can be improved by teaching them relevant PCK in a short amount of time.     

The risk of delayed school entry in relation to migration background in different institutional settings

In Germany, children of immigrants are faced with delayed school entry about twice as often as children of native-born parents because they are more often diagnosed not being ready for school. We investigate these early educational inequalities, focusing on the interplay of individual competencies and the institutional context in terms of the school enrollment procedure. We compare the conventional procedure, where school enrollment is supposed to be bound on school readiness, with reformed procedures where school enrollment is not supposed to be bound on school readiness. Data of school entry examinations from Hesse, Germany from 2008–2012 are analyzed (N = 50,717). The results show that children of immigrants from all origin groups have a substantially higher risk of obtaining a recommendation for delayed school entry than children of native-born parents. However, these ethnic differences are strongly reduced in the reformed procedures as compared to the conventional school enrollment procedure, indicating that these procedures could be an effective measure to reduce ethnic educational inequalities at school start.     

Milieuspezifische Bildungsstrategien in Familie und Gleichaltrigengruppe

Zusammenfassung Der Topos Bildungsungleichheit hat in der jüngsten Zeit eine Renaissance erfahren. Wie im Anschluss an den so genannten Sputnik-Schock und die Diskussionen der 1960er-bis 70er-Jahre wird durch den PISA-Schock die Frage nach der Qualit?t von Bildung erneut virulent. Gleichzeitig jedoch scheint die heutige bildungstheoretische Debatte zu eingeengt. Sie reduziert sich auf institutionalisierte schulische Bildung und vernachl?ssigt die zentrale Bedeutung au?erschulischer Bildungsprozesse, die sozialstrukturell variieren. So lassen sich aufgrund der — allerdings trotz sozialwissenschaftlicher Dauerbeobachtungen bis heute leider dünnen — Forschungsbasis deutliche Differenzen in milieuspezifischen Bildungsstrategien von Heranwachsenden und ihren Eltern feststellen. In Familie und Gleichaltrigengruppe herrschen demnach erfahrungsbezogene Bildungsinhalte und Strategien vor, die mit den in der Schule dominierenden Leistungs- und Qualifikationsanforderungen auf sehr unterschiedliche Weise zusammentreffen. Die vorgelegte mesotheoretische Bestimmung von Bildungsstrategien weist auf M?glichkeiten einer differenzierten Analyse von Bildungsprozessen hin, die an die makrostrukturellen Pr?missen des Systems sozialer Ungleichheit ebenso anschlussf?hig ist wie an mikrostrukturelle, lebensweltliche Erfahrungsbezüge der Schülerinnern und Schüler.

---

Summary Milieu-specific Educational Strategies in Families and Peer Groups “Educational disadvantage” is experiencing a renaissance as a topic of educational debate. Just as following the so-called Sputnik-Shock and discussions in the ’60s and ’70s, PISA has renewed the debate on the quality of education. At the same time, the perspective of current educational theory seems too narrow. It focuses on institutionalized school education and overlooks the central importance of educational processes outside of the school, which vary according to social structure. Despite continuous observations within this field, scientific research is weaker than might be expected. However, differences in milieu-specific strategies for education are clearly identifiable between young people and their parents. We propose that experience-based educational content and strategies predominate in both families and peer groups, and that these coincide with the achievements- and qualifications-based educational demands within schools in quite different ways. The following meso-theoretic investigation of educational strategies points to the potential of analyses, which can integrate both the macro-structural conditions of systems of inequality and the experiential, micro-structural reference points of pupils.

     

Das MINERVA-Projekt: Datenbankselektion für Peer-to-Peer-Websuche

Zusammenfassung In diesem Artikel wird MINERVA pr?sentiert, eine prototypische Implementierung einer verteilten Suchmaschine basierend auf einer Peer-to-Peer (P2P)-Architektur. MINERVA setzt auf die in der P2P-Welt verbreitete Technik verteilter Hash-Tabellen auf und benutzt diese zum Aufbau eines verteilten Verzeichnisses. Peers in unserem Ansatz entsprechen v?llig autonomen Benutzern mit ihren lokalen Suchm"oglichkeiten, die bereit sind, ihr lokales Wissen und ihre lokalen Suchm?glichkeiten im Rahmen einer Kollaboration zur Verfügung zu stellen. Wir formalisieren unsere Systemarchitektur und beschreiben das zentrale Problem einer effizienten Suche nach vielversprechenden Peers für eine konkrete Anfrage innerhalb des Verbundes. Wir greifen dabei auf existierende Methoden zurück and passen diese an unseren Systemkontext an. Wir pr?sentieren Experimente auf realen Daten, die verschiedene dieser Ans?tze vergleichen. Diese Experimente zeigen, dass die Qualit?t der Ans?tze variiert und untermauern damit die Wichtigkeit und den Einfluss einer leistungsstarken Methode zur Auswahl guter Datenbanken. Unsere Experimente deuten an, dass eine geringe Anzahl sorgf?ltig ausgew?hlter Datenbanken typischerweise bereits einen Gro?teil aller relevanten Ergebnisse des Gesamtsystems liefert.

---

This paper presents the MINERVA project that protoypes a distributed search engine based on P2P techniques. MINERVA is layered on top of a Chord-style overlay network and uses a powerful crawling, indexing, and search engine on every autonomous peer. We formalize our system model and identify the problem of efficiently selecting promising peers for a query as a pivotal issue. We revisit existing approaches to the database selection problem and adapt them to our system environment. Measurements are performed to compare different selection strategies using real-world data. The experiments show significant performance differences between the strategies and prove the importance of a judicious peer selection strategy. The experiments also present first evidence that a small number of carefully selected peers already provide the vast majority of all relevant results.

CR Subject Classification H.4,H.3.3,H3.4     

The “big red button” is too late: an alternative model for the ethical evaluation of AI systems

As a way to address both ominous and ordinary threats of artificial intelligence (AI), researchers have started proposing ways to stop an AI system before it has a chance to escape outside control and cause harm. A so-called “big red button” would enable human operators to interrupt or divert a system while preventing the system from learning that such an intervention is a threat. Though an emergency button for AI seems to make intuitive sense, that approach ultimately concentrates on the point when a system has already “gone rogue” and seeks to obstruct interference. A better approach would be to make ongoing self-evaluation and testing an integral part of a system’s operation, diagnose how the system is in error and to prevent chaos and risk before they start. In this paper, we describe the demands that recent big red button proposals have not addressed, and we offer a preliminary model of an approach that could better meet them. We argue for an ethical core (EC) that consists of a scenario-generation mechanism and a simulation environment that are used to test a system’s decisions in simulated worlds, rather than the real world. This EC would be kept opaque to the system itself: through careful design of memory and the character of the scenario, the system’s algorithms would be prevented from learning about its operation and its function, and ultimately its presence. By monitoring and checking for deviant behavior, we conclude, a continual testing approach will be far more effective, responsive, and vigilant toward a system’s learning and action in the world than an emergency button which one might not get to push in time.