An analysis of evaluation campaigns in ad-hoc medical information retrieval: CLEF eHealth 2013 and 2014

Since its inception in 2013, one of the key contributions of the CLEF eHealth evaluation campaign has been the organization of an ad-hoc information retrieval (IR) benchmarking task. This IR task evaluates systems intended to support laypeople searching for and understanding health information. Each year the task provides registered participants with standard IR test collections consisting of a document collection and topic set. Participants then return retrieval results obtained by their IR systems for each query, which are assessed using a pooling procedure. In this article we focus on CLEF eHealth 2013 and 2014s retrieval task, which saw topics created based on patients’ information needs associated with their medical discharge summaries. We overview the task and datasets created, and the results obtained by participating teams over these two years. We then provide a detailed comparative analysis of the results, and conduct an evaluation of the datasets in the light of these results. This twofold study of the evaluation campaign teaches us about technical aspects of medical IR, such as the effectiveness of query expansion; the quality and characteristics of CLEF eHealth IR datasets, such as their reliability; and how to run an IR evaluation campaign in the medical domain.     

Constructing and Using Multimodal Narratives to Research in Science Education: Contributions Based on Practical Classroom

This study deals with the problem of how to collect genuine and useful data about science classroom practices, and preserving the complex and holistic nature of teaching and learning. Additionally, we were looking for an instrument that would allow comparability and verifiability for teaching and research purposes. Given the multimodality of teaching and learning processes, we developed the multimodal narrative (MN), which describes what happens during a task and incorporates data such as examples of students’ work, photos, diagrams, etc. Also, it describes teachers’ intentions, preserving the nature of teaching practice in natural settings and it is verifiable and comparable. In this paper, we show how the MN was developed and present the protocol that was used for its construction. We identify the main characteristics of the MN and place it in the context of international research. We explore the potential of the MN for research purposes, illustrating its use in a research study that we carried out. We find that the MN provides a way to gather, organize and transform data, avoiding confusing and time-consuming manipulation of data, while minimizing the natural subjectivity of the narrator. The same MN can be used by the same or by different researchers for different purposes. Furthermore, the same MN can be used with different analysis techniques. It is also possible to study research practices on a large scale using MNs from different teachers and lessons. We propose that MNs can also be useful for teachers’ professional development.     

Teacher and school determinants of perceived classroom discipline: a multilevel analysis of TALIS 2013

Learning Environments Research - Classroom discipline is a significant concern in most educational systems and a critical element of an effective learning environment. In this article, we present a...