An in-depth study of similarity predicate committee

In the last decades, many similarity measures are proposed, such as Jaccard coefficient, cosine similarity, BM25, language model, etc. Despite the effectiveness of the existing similarity measures, we observe that none of them can consistently outperform the others in most typical situations. Choosing which similarity predicate to use is usually treated as an empirical question by evaluating a particular task with a number of different similarity predicates, which is not computationally efficient and the obtained results are not portable. In this paper, we propose a novel approach to combine different similarity predicates together to form a committee so that we do not need to worry about choosing which of them to use. Empirically, we can obtain a better result than any individual similarity predicate, which is quite meaningful in practice. Specifically, our method models the problem of committee generation as a 0–1 integer programming problem based on the confidence of similarity predicates and the reliability of attributes. We demonstrate the effectiveness of our model by applying it on three datasets with controlled errors. Experimental results demonstrate that our similarity predicate committee is more robust and superior over existing individual similarity predicates.     

基于直觉模糊信息的多属性排序问题的软集决策方法

针对属性信息具有模糊性的一类多属性方案排序问题,提出一种基于直觉模糊软集的决策方法。直觉模糊软集结合软集和直觉模糊集的优势,对对象近似描述时没有任何限制,并且能够更客观地表达事物的模糊性本质。首先定义直觉模糊软集的综合精确度、综合犹豫度和综合得分值,以及直觉模糊软集水平集的选择值和接受水平。在此基础上,提出关于多属性方案排序的选择值准则、精确度准则、犹豫度准则以及得分值准则。然后,提出一种组合使用各排序准则的多属性方案排序方法。最后,通过数值算例及讨论证实该方法的可行性和有效性。这种方法充分考虑决策者的直觉信息及主观偏好,能够有效完成多属性方案的优选和排序决策。     

以评估定级为契机促进图书馆事业发展--对省级公共图书馆评估标准的分析与思考

在阐述第三次全国公共图书馆评估工作的意义、作用以及比较分析评估标准有关项目的基础上．提出了对评估标准的补充修改意见。     

Catching up with Harvard: results from regression analysis of world universities league tables

This paper uses regression analysis to test if the universities performing less well according to Shanghai Jiao Tong University’s world universities league tables are able to catch up with the top performers, and to identify national and institutional factors that could affect this catching up process. We have constructed a dataset of 461 universities across 41 countries. We found consistent evidence of a moderate degree of catching up, especially amongst non-US universities. Larger universities as well as universities located in English speaking countries not only perform better on average, but also catch up more over 2003–2009. Universities located in lower income countries are also catching up more. The performance of private universities, as compared to that of public universities, varies substantially between the US and the other countries.     

On the interplay between normalisation,bias, and performance of paper impact metrics

We evaluate article-level metrics along two dimensions. Firstly, we analyse metrics’ ranking bias in terms of fields and time. Secondly, we evaluate their performance based on test data that consists of (1) papers that have won high-impact awards and (2) papers that have won prizes for outstanding quality. We consider different citation impact indicators and indirect ranking algorithms in combination with various normalisation approaches (mean-based, percentile-based, co-citation-based, and post hoc rescaling). We execute all experiments on two publication databases which use different field categorisation schemes (author-chosen concept categories and categories based on papers’ semantic information).In terms of bias, we find that citation counts are always less time biased but always more field biased compared to PageRank. Furthermore, rescaling paper scores by a constant number of similarly aged papers reduces time bias more effectively compared to normalising by calendar years. We also find that percentile citation scores are less field and time biased than mean-normalised citation counts.In terms of performance, we find that time-normalised metrics identify high-impact papers better shortly after their publication compared to their non-normalised variants. However, after 7 to 10 years, the non-normalised metrics perform better. A similar trend exists for the set of high-quality papers where these performance cross-over points occur after 5 to 10 years.Lastly, we also find that personalising PageRank with papers’ citation counts reduces time bias but increases field bias. Similarly, using papers’ associated journal impact factors to personalise PageRank increases its field bias. In terms of performance, PageRank should always be personalised with papers’ citation counts and time-rescaled for citation windows smaller than 7 to 10 years.     

Using the structure of overlap between search results to rank retrieval systems without relevance judgments

This paper addresses the problem of how to rank retrieval systems without the need for human relevance judgments, which are very resource intensive to obtain. Using TREC 3, 6, 7 and 8 data, it is shown how the overlap structure between the search results of multiple systems can be used to infer relative performance differences. In particular, the overlap structures for random groupings of five systems are computed, so that each system is selected an equal number of times. It is shown that the average percentage of a system’s documents that are only found by it and no other systems is strongly and negatively correlated with its retrieval performance effectiveness, such as its mean average precision or precision at 1000. The presented method uses the degree of consensus or agreement a retrieval system can generate to infer its quality. This paper also addresses the question of how many documents in a ranked list need to be examined to be able to rank the systems. It is shown that the overlap structure of the top 50 documents can be used to rank the systems, often producing the best results. The presented method significantly improves upon previous attempts to rank retrieval systems without the need for human relevance judgments. This “structure of overlap” method can be of value to communities that need to identify the best experts or rank them, but do not have the resources to evaluate the experts’ recommendations, since it does not require knowledge about the domain being searched or the information being requested.     

国内部分师范类院校学科建设水平现状研究——基于2017年全国学科评估的数据

以2017 年全国学科评估的数据为基础,对我国师范类院校学科评估的情况进行多角度的排名分析,并选取其中位居前列的院校作进一步研究,划分为若干个集团,对处于不同集团师范院校的学科实力进行比较,包括比较各自获A+ 学科的情况,及在人文类学科、艺术类学科、理学门类、工学门类、医学门类、管理学门类中的表现等等,进一步从得分、高校属性、高校所在的地域探讨了这些院校学科建设水平的现状与格局。对获得不同层次成绩的学科,如在学科评估中获得较好成绩（如获得A 类等级）的学科、仍有较大进步空间的学科特别是弱势学科、仍未参评的学科,提出了不同的学科建设方面的建议。最后特别针对大部分地方师范类院校,提出“教育学科+ 特色学科”的学科建设模式。     

Dealing with textual noise for robust and effective BERT re-ranking

The pre-trained language models (PLMs), such as BERT, have been successfully employed in two-phases ranking pipeline for information retrieval (IR). Meanwhile, recent studies have reported that BERT model is vulnerable to imperceptible textual perturbations on quite a few natural language processing (NLP) tasks. As for IR tasks, current established BERT re-ranker is mainly trained on large-scale and relatively clean dataset, such as MS MARCO, but actually noisy text is more common in real-world scenarios, such as web search. In addition, the impact of within-document textual noises (perturbations) on retrieval effectiveness remains to be investigated, especially on the ranking quality of BERT re-ranker, considering its contextualized nature. To mitigate this gap, we carry out exploratory experiments on the MS MARCO dataset in this work to examine whether BERT re-ranker can still perform well when ranking text with noise. Unfortunately, we observe non-negligible effectiveness degradation of BERT re-ranker over a total of ten different types of synthetic within-document textual noise. Furthermore, to address the effectiveness losses over textual noise, we propose a novel noise-tolerant model, De-Ranker, which is learned by minimizing the distance between the noisy text and its original clean version. Our evaluation on the MS MARCO and TREC 2019–2020 DL datasets demonstrates that De-Ranker can deal with synthetic textual noise more effectively, with 3%–4% performance improvement over vanilla BERT re-ranker. Meanwhile, extensive zero-shot transfer experiments on a total of 18 widely-used IR datasets show that De-Ranker can not only tackle natural noise in real-world text, but also achieve 1.32% improvement on average in terms of cross-domain generalization ability on the BEIR benchmark.     

企业网站搜索引擎优化策略研究

搜索引擎优化对企业推广有着重要的作用。在分析影响网站搜索引擎排名的因素的基础上,总结出有利于提高企业网站排名的搜索引擎优化策略。     

A statistical analysis of the construction of the United Nations E-Government Development Index

There is much debate among scholars about what constitutes e-government success, what method is best for measuring it, and which variables best describe it. This lack of consensus naturally leads to disagreement about the best approach for ranking the e-government programs of different countries. This article evaluates the current standard in e-government ranking, the United Nations E-government Development Index, by performing a confirmatory factor analysis on the raw data reported in the “Data tables” section of the “Statistical annex” in the 2010 United Nations E-Government Survey. The results identify several technical issues with the index and point to the need for a statistical tool that can be used to evaluate and guide the development of e-government ranking systems. The article recommends that researchers develop a new paradigm for quantitative e-government ranking that uses factor analysis to aid in the selection of variables, the organization of aggregate constructs, and the determination of weights.