Crowdsourcing interactions: using crowdsourcing for evaluating interactive information retrieval systems

In the field of information retrieval (IR), researchers and practitioners are often faced with a demand for valid approaches to evaluate the performance of retrieval systems. The Cranfield experiment paradigm has been dominant for the in-vitro evaluation of IR systems. Alternative to this paradigm, laboratory-based user studies have been widely used to evaluate interactive information retrieval (IIR) systems, and at the same time investigate users’ information searching behaviours. Major drawbacks of laboratory-based user studies for evaluating IIR systems include the high monetary and temporal costs involved in setting up and running those experiments, the lack of heterogeneity amongst the user population and the limited scale of the experiments, which usually involve a relatively restricted set of users. In this paper, we propose an alternative experimental methodology to laboratory-based user studies. Our novel experimental methodology uses a crowdsourcing platform as a means of engaging study participants. Through crowdsourcing, our experimental methodology can capture user interactions and searching behaviours at a lower cost, with more data, and within a shorter period than traditional laboratory-based user studies, and therefore can be used to assess the performances of IIR systems. In this article, we show the characteristic differences of our approach with respect to traditional IIR experimental and evaluation procedures. We also perform a use case study comparing crowdsourcing-based evaluation with laboratory-based evaluation of IIR systems, which can serve as a tutorial for setting up crowdsourcing-based IIR evaluations.     

A measure of emergent literacy for students with severe developmental disabilities

This study examined the underlying structure of the Nonverbal Literacy Assessment (NVLA), an instrument designed to measure emergent literacy for K–fourth‐grade students with severe developmental disabilities. The NVLA was conceptualized as having six constructs that reflected emergent literacy skills: (a) phonemic awareness, (b) phonics, (c) comprehension, (d) vocabulary, (e) listening comprehension, and (f) text awareness. Confirmatory factor analysis using data from 207 student administrations was used to examine the six‐factor model and two alternative models. Results suggested that all three models fit the data, but the high corrections coefficients among the constructs suggested that a one‐factor model of emergent literacy was the best‐fitting model. Implications and limitations are discussed. © 2010 Wiley Periodicals, Inc.     

Job-related requirements and competences of educational science graduates

All over the world, universities have to cope with the mission to promote the employability of their graduates, implying that competences should be trained which are relevant for the professional experience. In this context, it is of special interest where graduates work and what they are doing at work. In this study we focus on educational science graduates (N = 1565) and compare them to graduates of other study programmes (N = 10,224). Using data of the German DZHW graduate panel 2005 (N = 11,789), we address the questions (1) in which fields graduates of educational science work, (2) whether a theoretical-based four-dimensional internal structure of graduates’ job-related requirements and competences can be confirmed, (3) what kind of self-rated requirements educational science graduates experience at work, and (4) which self-rated levels of competences they possess, compared to other graduates. The analyses show that most graduates of educational science find an appropriate job in a clearly educational setting and that generic competences (systematic, social, and personal competences) are rated as more important than knowledge processing.     

Post-secondary Science Students’ Explanations of <Emphasis Type="Italic">Randomness</Emphasis> and <Emphasis Type="Italic">Variation</Emphasis> and Implications for Science Learning

The concepts of randomness and variation are pervasive in science. The purpose of this study was to document how post-secondary life science students explain randomness and variation, infer relationships between their explanations, and ability to describe and identify appropriate and inappropriate variation, and determine if students can identify sources of variation. An instrument designed to test statistical concepts was administered to 282 college students from three universities, ranging from introductory non-science majors to science graduate students. Students readily distinguished between causes of variation. A naïve no-pattern concept of randomness persisted from first-year non-science majors to senior-level science majors, contributing to incorrect responses on the variation instrument. Students’ expressions of randomness were better predictors of performance on the variation instrument than their expressions of variation. It is argued that inclusion of everyday language uses of randomness in instruction can bridge the gap between vernacular and scientific uses of this term.     

Analog of parental empathy: Association with physical child abuse risk and punishment intentions

Objectives

Current research has been inconsistent in corroborating that parents’ compromised empathy is associated with elevated physical child abuse risk, perhaps in part because of an emphasis on dispositional empathy rather than empathy directed at their own children. Research has also relied on self-reports of empathy that are susceptible to participant misrepresentation. The present study utilized an analog task of parental empathy to investigate the association of parental empathy toward one's own child with physical child abuse potential and with their tendency to punish perceived child misbehavior.

Methods

A sample of 135 mothers and their 4–9 year old children were recruited, with mothers estimating their children's emotional reactions using a behavioral simulation of parental empathy. Mothers also provided self-reports on two measures of child abuse potential, a measure of negative attributions and expected punishment of children using vignettes, as well as a traditional measure of dispositional empathic concern and perspective-taking.

Results

Findings suggest that parental demonstration of poorer empathic ability on the analog task was significantly related to increased physical abuse potential, likelihood to punish, and negative child attributions. However, self-reported dispositional empathy exhibited the pattern of inconsistent associations previously observed in the literature.

Conclusions

Parental empathy appears to be a relevant target for prevention and intervention programs. Future research should also consider similar analog approaches to investigate such constructs to better uncover the factors that elevate abuse risk.     

Differential effects of reading trainings on reading processes: a comparison in Grade 2

Phonics, fluency, and reading strategy trainings are evidence-based interventions that foster the reading skills of poor readers in primary school. The purpose of the present study was to compare differential effects of the three types of trainings on the efficiency of component processes on word, sentence, and text level immediately after the training and at a 3-month follow-up. The 235 poor readers were randomly allocated to one of the reading interventions or to a control condition. All interventions consisted of 25 sessions that were scheduled twice a week and lasted 45 min. Results indicated short-term effects of the phonics training and the strategy training on the efficiency of a broad range of word-level and sentence-level processes. None of the treatment effects persisted over the long term, indicating the need for instructional efforts to regularly practice the acquired skills after the actual training.     

The perceptions,policy, and practice of educating twins: A review

Twin births are increasing at a rapid rate, yet many educational personnel throughout the world are unaware of the needs of twins in school. Stereotypical views of twins as pathologically close or excessively dependent pervade educational policies and practices, despite the fact that twins do not differ from singletons on rates of psychological disturbance. Research shows that many schools implement strict policies for classroom placement of twins, yet the effects of such policies on twins' development and educational attainment are currently unknown. Recommendations for school personnel and future directions for this formative area are discussed. © 2003 Wiley Periodicals, Inc. Psychol Schs 40: 429–438, 2003.