Are governments complying with transparency? Findings from 15 years of evaluation

How well do governments comply with their own transparency statutes? Alarmingly, answers to this question are in short supply because of patchy comparative data. This study begins to address this gap by analyzing an exhaustive compilation of 265 transparency compliance evaluations (i.e. audits) authored by NGOs, academics, and government oversight authorities across Latin America between 2003 and 2018. Aggregating data on patterns of evaluation and public sector compliance with transparency statutes, we find modest increases in compliance over time, strikingly low compliance with passive transparency compliance (i.e. governmental responses to citizen requests) at the local versus national levels of government, and significantly higher compliance scores when government oversight agencies act as evaluators. Our data also reveal worrying gaps in evaluation efforts. Evaluators tend to focus far more on active transparency (i.e. website-based disclosure) than passive transparency, more on national level governments than subnational governments, and preponderantly on the executive as opposed to legislative or judicial branches of government. Textual analyses show that education is the most evaluated policy theme, with financial policies and social services close behind. In presenting the first large-scale cross-national assessment of transparency compliance, the current study brings to light trends in the effectiveness of transparency regimes and patterns of transparency evaluation across Latin America.     

Note-taking within MetaTutor: interactions between an intelligent tutoring system and prior knowledge on note-taking and learning

Hypermedia learning environments (HLE) unevenly present new challenges and opportunities to learning processes and outcomes depending on learner characteristics and instructional supports. In this experimental study, we examined how one such HLE—MetaTutor, an intelligent, multi-agent tutoring system designed to scaffold cognitive and metacognitive self-regulated learning (SRL) processes—interacts with learner’s prior domain knowledge to affect their note-taking activities and subsequent learning outcomes. Sixty (N = 60) college students studied with MetaTutor for 120 min and took notes on hypermedia content of the human circulatory system. Log-files and screen recordings of learner-system interactions were used to analyze notes for several quantitative and qualitative variables. Results show that most note-taking was a verbatim copy of instructional content, which negatively related to the post-test measure of learning. There was an interaction between prior knowledge and pedagogical agent scaffolding, such that low prior knowledge students took a greater quantity of notes compared to their high prior knowledge counterparts, but only in the absence of MetaTutor SRL scaffolding; when agent SRL scaffolding was present, the note-taking activities of low prior knowledge students were statistically equivalent to the number of notes taken by their high prior knowledge counterparts. Theoretical and instructional design implications are discussed.     

Can a systematic assessment moderation process assure the quality and integrity of assessment practice while supporting the professional development of casual academics?

There has been a threefold increase in the employment of casual academics in Australian universities within the last 20 years, to the extent that most teaching and marking is now undertaken by casual academics, also known as sessional staff. Yet, casualised teaching and assessment has been considered a risk to student engagement and success, and casual academics report a lack of professional development and increased feelings of marginalisation within the academy. Concurrently, the quality assurance of teaching and assessment in higher education has become a central focus of the government-funded regulatory organisation, the Tertiary Education Quality and Standards Agency (TEQSA). Situated within this context, we report on an assessment moderation process that could support casual academics’ contextualised professional development, generate a sense of connectedness and collegiality and fulfil the requirements of TEQSA. Such processes may ensure that workforce growth in the higher education system supports a robust quality assurance and regulatory framework.     

Latent Variable Modeling and Adaptive Testing for Experimental Cognitive Psychopathology Research

The adaptation of experimental cognitive tasks into measures that can be used to quantify neurocognitive outcomes in translational studies and clinical trials has become a key component of the strategy to address psychiatric and neurological disorders. Unfortunately, while most experimental cognitive tests have strong theoretical bases, they can have poor psychometric properties, leaving them vulnerable to measurement challenges that undermine their use in applied settings. Item response theory–based computerized adaptive testing has been proposed as a solution but has been limited in experimental and translational research due to its large sample requirements. We present a generalized latent variable model that, when combined with strong parametric assumptions based on mathematical cognitive models, permits the use of adaptive testing without large samples or the need to precalibrate item parameters. The approach is demonstrated using data from a common measure of working memory—the N-back task—collected across a diverse sample of participants. After evaluating dimensionality and model fit, we conducted a simulation study to compare adaptive versus nonadaptive testing. Computerized adaptive testing either made the task 36% more efficient or score estimates 23% more precise, when compared to nonadaptive testing. This proof-of-concept study demonstrates that latent variable modeling and adaptive testing can be used in experimental cognitive testing even with relatively small samples. Adaptive testing has the potential to improve the impact and replicability of findings from translational studies and clinical trials that use experimental cognitive tasks as outcome measures.     

Early education teachers’ conceptualizations and strategies for managing closeness with the class in child care

This study investigated teachers’ experiences of closeness during interactions with the group of children in child care. Structured interviews were conducted with 24 female teachers who were teaching children between the ages of three and five (mean age = 3.9) regarding their perceptions of closeness with the group of children in the class. Qualitative analyses of the interviews revealed a group solidarity domain of teacher–group relationships composed of three dimensions which comprise emotional solidarity, consensual solidarity, and functional solidarity, with emotional solidarity most dominant in the teachers’ perceptions. This group solidarity domain was interpreted as being consistent with qualities found in Grusec and Davidov’s group participation domain. The findings that, teachers’ closeness with the group of children in the class were dominated by the emotional dimension is similar to previous research results which report the predominance of personal and intimate interactions in relational closeness between teachers and individual children. Teachers described intentional strategies for managing close group relationships that made use of interactional histories between the teachers and children in the class. These discoveries provide a new perspective on teacher closeness with groups, an area of research not often investigated. It also provides insights into the importance teachers place on connecting emotionally with children when they are with the collective group in the classroom.     

On Going (Way) Beyond One’s Data: A Proposal to Restrict Recommendations for Practice in Primary Educational Research Journals

To counteract what we see as a growing research-reporting concern, we propose the following editorial-policy change regarding the content of primary research articles in educational research journals: Contributors should restrict their discussion and conclusions to their data and not offer recommendations for educational practice nor speculate about the educational policy implications of their research. We believe that this change will be beneficial for maintaining a separation between evidence and opinion concerning the legitimate warrants of empirical research. We suggest viable accompaniments and alternatives to authors offering practical recommendations and policy speculations in primary-research journal articles. Among these is to rely on all sources of evidence, including systematic research syntheses based on the aggregated contribution of multiple research studies from different sources, with recommendations for practice tailored to specific situations so that both their validity and utility will be increased. It is hoped that our proposed policy change will initiate a conversation among researchers, editors, reviewers, and educational-policy experts with respect to the scope and purpose of primary research journal articles.