On the interplay between normalisation,bias, and performance of paper impact metrics

We evaluate article-level metrics along two dimensions. Firstly, we analyse metrics’ ranking bias in terms of fields and time. Secondly, we evaluate their performance based on test data that consists of (1) papers that have won high-impact awards and (2) papers that have won prizes for outstanding quality. We consider different citation impact indicators and indirect ranking algorithms in combination with various normalisation approaches (mean-based, percentile-based, co-citation-based, and post hoc rescaling). We execute all experiments on two publication databases which use different field categorisation schemes (author-chosen concept categories and categories based on papers’ semantic information).In terms of bias, we find that citation counts are always less time biased but always more field biased compared to PageRank. Furthermore, rescaling paper scores by a constant number of similarly aged papers reduces time bias more effectively compared to normalising by calendar years. We also find that percentile citation scores are less field and time biased than mean-normalised citation counts.In terms of performance, we find that time-normalised metrics identify high-impact papers better shortly after their publication compared to their non-normalised variants. However, after 7 to 10 years, the non-normalised metrics perform better. A similar trend exists for the set of high-quality papers where these performance cross-over points occur after 5 to 10 years.Lastly, we also find that personalising PageRank with papers’ citation counts reduces time bias but increases field bias. Similarly, using papers’ associated journal impact factors to personalise PageRank increases its field bias. In terms of performance, PageRank should always be personalised with papers’ citation counts and time-rescaled for citation windows smaller than 7 to 10 years.     

Cross‐Validated Prediction of Academic Performance of First‐Year University Students: Identifying Risk Factors in a Nonselective Environment

Early prediction of academic performance is important for student support. The authors explored, in a multivariate approach, whether pre‐entry data (e.g., high school study results, preparative activities, expectations, capabilities, motivation, and attitude) could predict university students’ first‐year academic performance. Preregistered applicants for a bachelor's program filled out an intake questionnaire before study entry. Outcome data (first‐year grade point average, course credits, and attrition) were obtained 1 year later. Prediction accuracy was assessed by cross‐validation. Students who performed better in preparatory education, followed a conventional educational path before entering, and expected to spend more time on a program‐related organization performed better during their first year at university. Concrete preuniversity behaviors were more predictive than psychological attributions such as self‐efficacy. Students with a “love of learning” performed better than leisure‐oriented students. The intake questionnaire may be used for identifying up front who may need additional support, but is not suitable for student selection.     

Echoes of Early Life: Recent Insights From Mathematical Modeling

In the last decades, developmental origins of health and disease (DOHaD) has emerged as a central framework for studying early‐life effects, that is, the impact of fetal and early postnatal experience on adult functioning. Apace with empirical progress, theoreticians have built mathematical models that provide novel insights for DOHaD. This article focuses on three of these insights, which show the power of environmental noise (i.e., imperfect indicators of current and future conditions) in shaping development. Such noise can produce: (a) detrimental outcomes even in ontogenetically stable environments, (b) individual differences in sensitive periods, and (c) early‐life effects tailored to predicted future somatic states. We argue that these insights extend DOHaD and offer new research directions.     

Alternative approaches to structural modeling of ordinal data: A Monte Carlo study

In practice, several measures of association are used when analyzing structural equation models with ordinal variables: ordinary Pearson correlations (PE approach), polychoric and polyserial correlations (PO approach), and conditional polychoric correlations (CPO approach). In the case of structural equation models without latent variables, the literature has shown that the PE approach is outperformed by the alternatives. In this article we report a Monte Carlo study showing the comparative performance of the aforementioned alternative approaches under deviations from their respective assumptions in the case of structural equation models with latent variables when attention is restricted to point estimates of model parameters. The CPO approach is shown to be the most robust against nonnormality. It is also robust to randomness of the exogenous variables, but not to the existence of measurement errors in them. The PO approach lacks robustness against nonnormality. The PE approach lacks robustness against transformation errors but otherwise it can perform about as well as the alternative approaches.     

Selection to the Master’s Phase at the Binary Divide,a Dutch Case Study

Tertiary Education and Management - This article presents a case study of selection for graduate study at the Vrije Universiteit (VU) in the Netherlands. The VU developed and implemented an...     

How useful is the concept of motivation for program evaluation?

European Journal of Psychology of Education -