How robust are journal rankings based on the impact factor? Evidence from the economic sciences

It is well-known that the distribution of citations to articles in a journal is skewed. We ask whether journal rankings based on the impact factor are robust with respect to this fact. We exclude the most cited paper, the top 5 and 10 cited papers for 100 economics journals and recalculate the impact factor. Afterwards we compare the resulting rankings with the original ones from 2012. Our results show that the rankings are relatively robust. This holds both for the 2-year and the 5-year impact factor.     

Rankings,Diversity and the Power of Renewal in Science. A Comparison between Germany,the UK and the US

Rankings are part and parcel of the neoliberal agenda in science aiming at increasing the competitive allocation of funds among universities. This article focuses on the decreasing power of renewal in science as a result of this agenda particularly because of its increasingly consolidated stratification of the academic system into élite and mass institutions. A comparison based on data from the Academic Ranking of World Universities of the Shanghai Jiao Tong University between the less stratified German system, the more strongly stratified British system, and the most strongly stratified, though at its heart still diverse US-American system provides a first test of the hypotheses.     

How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management

This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management Schools (BMS) in the UK. On the basis of various mappings and metrics, this study shows that: (i) Innovation Studies units are consistently more interdisciplinary in their research than Business & Management Schools; (ii) the top journals in the Association of Business Schools’ rankings span a less diverse set of disciplines than lower-ranked journals; (iii) this results in a more favourable assessment of the performance of Business & Management Schools, which are more disciplinary-focused. This citation-based analysis challenges the journal ranking-based assessment. In short, the investigation illustrates how ostensibly ‘excellence-based’ journal rankings exhibit a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how the bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.     

从世界排名的变化看乒乓球技术的发展方向

在国际乒坛涌现出的三名年轻优秀男子乒乓球运动员的世界排名攀升过程中,分析了11分制实施前和实施后这些优秀运动员世界排名的升降特点和技术风格,从中简要预测今后乒乓球技术的发展方向和趋势.     

世界一流大学与一流学科关系探析——基于软科2019年排行榜的分析

以软科公布的2019年世界大学学术排名与世界一流学科排名为依据,对世界一流大学拥有的世界一流学科数量以及世界一流学科在各大学和各学科领域的分布情况进行统计。统计表明:世界一流大学通常在多个学科领域拥有较多的世界一流学科;世界一流学科也大多分布在世界一流大学中。近年来,我国高校无论在世界大学学术排行榜上,还是在世界一流学科排行榜上都取得了显著进步。整体上,我国拥有世界一流学科的高校较多,世界一流大学较少;我国的世界一流大学拥有的世界一流学科在数量上有一定优势,但学科领域的分布不均衡,世界一流学科主要集中在工学领域。“双一流”建设后续进程中应适当调整一流大学建设与一流学科建设的关系,强化学科发展的均衡性,从重点关注一流学科“点”的建设到更加关注大学的综合能力建设。     

专业搜索引擎的排序算法研究

探讨影响搜索引擎排序的一般性因素：词频和词位置信息、用户行为信息、网页之间的链接信息等，在此基础上针对专业搜索引擎的排序算法，提出主题相关度并结合基础教育搜索引擎进行实验。实验结果表明，专业搜索引擎中主题相关度的适当应用能明显改善排序结果。     

Evaluating university performance using reference point based composite indicators

In this paper, we propose the application of a novel methodology to build composite indicators, in order to evaluate university performance. We analyse separately the three basic dimensions of our university system (research, teaching and technology transfer), because we are interested in getting a more accurate vision of each of them. In order to build the composite indicators, we use a multi-criteria analysis technique, based on the double reference point method. One advantage of this technique is the possibility to use reference levels, in such a way that the results obtained are easily interpreted in terms of the performance of the university with respect to these levels. Besides, aggregations for different compensation degrees are provided. In order to illustrate the advantages of this method, it has been applied to evaluate the performance of the public universities of the Spanish region of Andalucía, for year 2008. The results show that the performance of the Andalusian public universities in the teaching block is better than in the research and technology transfer blocks. The application lets us conclude that the methodology offers a warning system to assist in strategic decision making, and the values of the indicators allow us to find fields of improvement in all areas.     

中国大学综合实力排行榜排名方法的比较分析——基于高等教育机构排名柏林原则的视角

高等教育机构排名的柏林原则是国际大学排名机构的基本评价准则。以16项柏林原则为基准，以武书连榜、校友会榜、邱均平榜、软科榜的相关排名方法资料为比较分析对象，逐项对照、分析比较，给予五级等距吻合度评判。研究结果显示，武书连榜获4A、3D，校友会榜获3A、2D，邱均平榜获2A、3D，软科榜获4A、3D。基于对各排行榜的评判和对获得A级和D级的分析，对排名机构提出了两方面的建议：一是应参照柏林原则，自查自纠，改进排名方法，提高排名的规范性；二是应使用《IREG排名审计手册》开展自我审计，有条件者可向IREG学术排名和卓越协会申请排名审计。     

Examining drug and side effect relation using author–entity pair bipartite networks

The current study has two objectives. First, we explore the characteristics of biological entities, such as drugs, and their side effects using an author–entity pair bipartite network. Second, we use the constructed network to examine whether there are outstanding features of relations between drugs and side effects. We extracted drug and side effect names from 169,766 PubMed abstracts published between 2010 to 2014 and constructed author–entity pair bipartite networks after ambiguous author names were processed. We propose a new ranking algorithm that takes into consideration the characteristics of bipartite networks to identify top-ranked biological drug and side effect pairs. To investigate the relationship between a particular drug and a side effect, we compared the drug and side effect pairs obtained from the network containing both drug and side effect with those observed in SIDER, a human expert-curated database. The results of this study indicate that our approach was able to identify a wide range of patterns of drug–side effect relations from the perspective of authors’ research interests. Further, our approach also identified the unique characteristics of the relation of biomedical entities obtained using an author–entity pair bipartite network.     

Globalised vs averaged: Bias and ranking performance on the author level

We analyse the difference between the averaged (average of ratios) and globalised (ratio of averages) author-level aggregation approaches based on various paper-level metrics. We evaluate the aggregation variants in terms of (1) their field bias on the author-level and (2) their ranking performance based on test data that comprises researchers that have received fellowship status or won prestigious awards for their long-lasting and high-impact research contributions to their fields. We consider various direct and indirect paper-level metrics with different normalisation approaches (mean-based, percentile-based, co-citation-based) and focus on the bias and performance differences between the two aggregation variants of each metric. We execute all experiments on two publication databases which use different field categorisation schemes. The first uses author-chosen concept categories and covers the computer science literature. The second covers all disciplines and categorises papers by keywords based on their contents. In terms of bias, we find relatively little difference between the averaged and globalised variants. For mean-normalised citation counts we find no significant difference between the two approaches. However, the percentile-based metric shows less bias with the globalised approach, except for citation windows smaller than four years. On the multi-disciplinary database, PageRank has the overall least bias but shows no significant difference between the two aggregation variants. The averaged variants of most metrics have less bias for small citation windows. For larger citation windows the differences are smaller and are mostly insignificant.In terms of ranking the well-established researchers who have received accolades for their high-impact contributions, we find that the globalised variant of the percentile-based metric performs better. Again we find no significant differences between the globalised and averaged variants based on citation counts and PageRank scores.