Validating simulated interaction for retrieval evaluation

A searcher’s interaction with a retrieval system consists of actions such as query formulation, search result list interaction and document interaction. The simulation of searcher interaction has recently gained momentum in the analysis and evaluation of interactive information retrieval (IIR). However, a key issue that has not yet been adequately addressed is the validity of such IIR simulations and whether they reliably predict the performance obtained by a searcher across the session. The aim of this paper is to determine the validity of the common interaction model (CIM) typically used for simulating multi-query sessions. We focus on search result interactions, i.e., inspecting snippets, examining documents and deciding when to stop examining the results of a single query, or when to stop the whole session. To this end, we run a series of simulations grounded by real world behavioral data to show how accurate and responsive the model is to various experimental conditions under which the data were produced. We then validate on a second real world data set derived under similar experimental conditions. We seek to predict cumulated gain across the session. We find that the interaction model with a query-level stopping strategy based on consecutive non-relevant snippets leads to the highest prediction accuracy, and lowest deviation from ground truth, around 9 to 15% depending on the experimental conditions. To our knowledge, the present study is the first validation effort of the CIM that shows that the model’s acceptance and use is justified within IIR evaluations. We also identify and discuss ways to further improve the CIM and its behavioral parameters for more accurate simulations.     

Accountability through access,authenticity and advocacy when researching with young children

ABSTRACT

Research with children involving their use of digital and mobile technologies either as a methodological tool or in relation to their learning foregrounds emerging ethical issues and practices. This paper explores some of the ethical and practical challenges we faced in studies involving the recruitment of young children as research participants, and where the integrity of these research collaborations was critical. We propose an ethical framework to foreground these challenges that is shaped by a view of children as social actors and experts on their own lives, information and communication technologies as ubiquitous in children’s lives, and ethics as a situated and multifaceted responsibility. This framework has three aspects: access, authenticity and advocacy. We draw on examples from different research projects and use ethically important moments to illustrate how notions of access, authenticity and advocacy can foreground the ethical challenges in teaching–learning research contexts to better consider and offer children greater agency in research collaborations.     

On two effects of signaling the consequences for remembering

Five pigeons were trained to perform a delayed matching-to-sample task in which red- and green-colored keys were presented as sample and choice stimuli, and the duration of a delay interval varied across trials. Experiment 1 investigated the effects on delayed-matching accuracy of signaling different durations of food access for the two correct responses (the differential-outcomes effect), and of signaling nondifferential but larger durations for both responses (the signaled-magnitudes effect). In Condition 1, a vertical bar on either sample signaled different rewards (or different outcomes, DOs) for correct red and correct green responses (0.5 and 3.5 sec, respectively), and a horizontal bar signaled equal durations of food access (or same outcomes, SOs) for these responses (1.5 sec). In Condition 2, the horizontal bar signaled equally large rewards for the two correct responses (3.5 sec), and the vertical bar signaled equally small rewards (0.5 sec). Delayed-matching accuracies were higher on DO trials than on SO trials, and they were higher on large-reward trials than on small-reward trials. However, analyses of discriminability estimates as a function of delay-interval duration revealed differences between the forgetting functions reflecting these two effects. Signaling DOs increased the initial level of the function and reduced its slope relative to signaling SOs, whereas signaling larger rewards increased the initial level of the function but did not affect its slope relative to signaling smaller rewards. Experiment 2 investigated whether the difference between the initial levels of DO and SO functions in Condition 1 resulted from overall longer food access on the former trials. However, varying the food-access times on SO trials across three conditions (0.5, 3.5, and 1.5 sec) failed to produce systematic effects consistent with this hypothesis. The results are discussed with respect to the mechanisms that could be responsible for the two effects.     

From novice to no know-how: A longitudinal study of implicit motor learning

The aim of this study was to ascertain whether the performances of implicit and explicit learners would converge over an extended period of learning. Participants practised a complex motor skill - golf putting - for 3000 trials, either with a concurrent secondary, tone-counting task (implicit learning) or without such a task (explicit learning). The cognitive demands of the secondary task were predicted to prevent the accumulation of verbalizable rules about the motor task. The implicit group reported significantly fewer rules than the explicit group on subsequent verbal protocols. The performance of the implicit group remained below that of the explicit group throughout the learning phase. However, no significant differences were found between groups during a delayed retention test. Additionally, for the participants in the explicit group only, a Reinvestment Scale score correlated positively with the number of rules accrued and negatively with overall putting performance during the learning phase. We use the results to argue against the excessive use of verbal instruction during skill acquisition, which might be unnecessary and ultimately might hamper performance under stressful conditions.     

The integration of an anatomy massive open online course (MOOC) into a medical anatomy curriculum

Massive open online courses (MOOCs) are designed as stand‐alone courses which can be accessed by any learner around the globe with only an internet‐enabled electronic device required. Although much research has focused on the enrolment and demographics of MOOCs, their impact on undergraduate campus‐based students is still unclear. This article explores the impact of integrating an anatomy MOOC in to the anatomy curriculum of a year 1 medical degree program at the University of Leeds, United Kingdom. The course did not replace any teaching that was already being delivered, and was used to supplement this teaching to support the students' consolidation and revision. Analysis of student feedback indicates a high level of usage, with evidence to suggest that female learners may have approached the course in a more personalized manner. Although the video based resources and quizzes were greatly appreciated as learning tools, significant evidence suggests the students did not engage, or were inclined to engage, with the discussion fora. Furthermore, a significant majority of students did not want the MOOC to replace the existing teaching they received. Given the feedback provided, this research suggests that although the student population believe there to be value in having access to MOOC material, their role as replacements to campus‐based teaching is not supported. Details regarding the enrolment and engagement of the general public with the MOOC during the two runs are also documented, with the suggestion that graduates employed in the healthcare sector were the primary users of the course. Anat Sci Educ 10: 53–67. © 2016 American Association of Anatomists.     

Action cycles: rhythmic actions for engagement with children and young adults with profound mental handicap

ABSTRACT

Rhythmic groups of cyclical movements—action cycles—were identified during interactions between adults and children and young adults with profound mental handicap. The cycles were allocated to five categories according to rate, each action cycle clearly having its own communicative purpose. There was a striking regularity of rate within each performed set of action cycles, this being observed in the people with profound mental handicap as well as the adults, with the regularity more marked in the faster cycles. The people with profound mental handicap performed similar actions for apparently similar reasons at similar rates to their caregivers, giving strong indication of a biologically robust system of basic emotional communication which can quite literally be tapped into by caregivers. Comparisons of video records of interactions between mothers and their non‐handicapped babies (4‐8 weeks old) in Scotland and Nigeria yielding similar action cycles and rates to those seen in the interactions with people with profound mental handicap support this hypothesis.