Statistical biases in Information Retrieval metrics for recommender systems

There is an increasing consensus in the Recommender Systems community that the dominant error-based evaluation metrics are insufficient, and mostly inadequate, to properly assess the practical effectiveness of recommendations. Seeking to evaluate recommendation rankings—which largely determine the effective accuracy in matching user needs—rather than predicted rating values, Information Retrieval metrics have started to be applied for the evaluation of recommender systems. In this paper we analyse the main issues and potential divergences in the application of Information Retrieval methodologies to recommender system evaluation, and provide a systematic characterisation of experimental design alternatives for this adaptation. We lay out an experimental configuration framework upon which we identify and analyse specific statistical biases arising in the adaptation of Information Retrieval metrics to recommendation tasks, namely sparsity and popularity biases. These biases considerably distort the empirical measurements, hindering the interpretation and comparison of results across experiments. We develop a formal characterisation and analysis of the biases upon which we analyse their causes and main factors, as well as their impact on evaluation metrics under different experimental configurations, illustrating the theoretical findings with empirical evidence. We propose two experimental design approaches that effectively neutralise such biases to a large extent. We report experiments validating our proposed experimental variants, and comparing them to alternative approaches and metrics that have been defined in the literature with similar or related purposes.     

意大利实心粉年(节选)

村上春树的作品总是承载着幽幽的孤独、脉脉的温情和淡淡的忧伤,流淌着一种孤独之美。该文亦如是。女主人翁与男朋友分手后,一个人孤独地怀念,孤独地怨恨,孤独地冥想,孤独地和自己的心灵对话。那日复一日吃意大利实心粉的执著,那大得可以装下德国牧羊人的铝锅以及捏成德国牧羊人的实心粉,那如零碎的记忆般的拜访者,那印象模糊的电话那端的女孩……这一切都将女主人翁的内心世界展露无遗。村上春树笔端细腻,刻画人物内心情感丝丝入扣,震撼人心。     

How do firms’ knowledge bases affect intra-industry heterogeneity? : An analysis of the Spanish pharmaceutical industry

This paper aims to contribute to the analysis of within industry inter-firm variety. The paper develops two themes: (i) the analysis of intra-industry heterogeneity, and (ii) the extent to which higher performance is associated with the capacity of firms to expand their knowledge base. The main contribution of the paper is empirical, based on a data source consisting of information on documents published in international scientific journals by Spanish pharmaceutical firms. The empirical results support the argument that the firm's knowledge base is a main driver of persistent heterogeneity within industries. We find systematic variety in terms of how firms articulate their research activities, and positive correlation between firms’ knowledge diversification and performance.     

Circadian Rhythms in Cognitive Processes: Implications for School Learning

Circadian variations have been found in cognitive processes, such as attention, working memory, and executive functions, which may explain oscillations in the performance of many tasks. These cognitive processes improve during the day and decrease during the night and early hours of the morning. Sleep deprivation further decreases these cognitive processes during daytime hours. This review analyses the recent findings on circadian rhythms in cognitive processes, as well as the implications of these rhythms for school learning.     

Does generating examples aid proof production?

Many mathematics education researchers have suggested that asking learners to generate examples of mathematical concepts is an effective way of learning about novel concepts. To date, however, this suggestion has limited empirical support. We asked undergraduate students to study a novel concept by either tackling example generation tasks or reading worked solutions to these tasks. Contrary to suggestions in the literature, we found no advantage for the example generation group on subsequent proof production tasks. From a second study, we found that undergraduate students overwhelmingly adopt a trial and error approach to example generation and suggest that different example generation strategies may result in different learning gains. We conclude by arguing that the teaching strategy of example generation is not yet understood well enough to be a viable pedagogical recommendation.     

Preservice teachers’ knowledge of proof by mathematical induction

There is a growing effort to make proof central to all students’ mathematical experiences across all grades. Success in this goal depends highly on teachers’ knowledge of proof, but limited research has examined this knowledge. This paper contributes to this domain of research by investigating preservice elementary and secondary school mathematics teachers’ knowledge of proof by mathematical induction. This research can inform the knowledge about preservice teachers that mathematics teacher educators need in order to effectively teach proof to preservice teachers. Our analysis is based on written responses of 95 participants to specially developed tasks and on semi-structured interviews with 11 of them. The findings show that preservice teachers from both groups have difficulties that center around: (1) the essence of the base step of the induction method; (2) the meaning associated with the inductive step in proving the implication P(k) ⇒ P(k + 1) for an arbitrary k in the domain of discourse of P(n); and (3) the possibility of the truth set of a sentence in a statement proved by mathematical induction to include values outside its domain of discourse. The difficulties about the base and inductive steps are more salient among preservice elementary than secondary school teachers, but the difficulties about whether proofs by induction should be as encompassing as they could be are equally important for both groups. Implications for mathematics teacher education and future research are discussed in light of these findings.