构建符合中国国情的“学校声誉”指标体系的探讨

"学校声誉"排名已成为我国大陆大学排行榜的评价指标之一,但目前进行的"学校声誉"排名不符合中国国情,其实质就是"社会声誉"排名。符合中国国情的"学校声誉"包括"社会声誉"、"学术声誉"、"国家声誉"三个方面。本文选取2004年在网大、中国校友会和中国管理科学研究院三家评价机构的学校综合排名中,进入前50 名的并集,共有65所大学为标本,进行学校声誉排名研究,并给出2004年中国大陆50强"学校声誉"排行榜。     

自然科学期刊自引对影响因子的"调控"

本文利用《中国科技期刊引证报告》,重新计算了其中几个学科的一些期刊除去自引后的影响因子,并对去除前和去除后的影响因子与期刊排名进行了对比,以考察期刊自引对影响因子和期刊排名的影响。调查发现目前个别期刊过度自引已经使期刊排名发生了失真。最后对如何遏制这种现象提出了一些建议。     

科技评价中标准化方法对评价结果的影响研究

对科技评价中多属性评价方法数据标准化问题进行系统梳理,认为无论是正向指标还是反向指标,两极标准化评价结果肯定小于可调标准化;可调标准化对一些评价分值较低的期刊有一种"鼓励作用",可以变相提高它们的评价得分;无论是正向指标还是反向指标,线性标准化方法的不同只会影响评价结果值,但不会影响评价结果的排序。但是反向指标经典标准化方法可能会影响评价结果的排序;提出一种新的线性反向指标标准化方法,并且提出应该淘汰反向指标经典标准化方法;评价机构应该公布标准化方法以保证评价公平。     

基于SCI-E收录论文期刊分区的奖励可行性分析研究

文摘针对SCI-E收录论文期刊分区中,中国科学院文献情报中心与W eb ofKnowledge平台中的《期刊引文报告》(Journal C itation Reports,简称JCR)采用方法不同。本文以东华大学为案例,分析研究了SCI-E收录论文分区奖励的可行性,研究表明:对于以纺织服装学科为特色的东华大学,可以采用在重点奖励中国科学院文献情报中心公布的1区和2区论文基础上,鼓励特色学科位于JCR中的Q1区论文。     

Measuring engagement on twitter using a composite index: An application to social media influencers

Engagement on social networks is a complex concept, in which many interconnected, difficult-to-assess components interact. It is precisely this complexity which motivated this work, which proposes a composite index as a tool to measure engagement. Using TOPSIS, a multicriteria method that bases its ranking on minimizing the distance to the ideal point and maximizing the distance to the anti-ideal, a mix of indicators based on two approaches is used: the tweet approach and the follower approach. The former reflects engagement based on user production, and the latter measures engagement by popularity. This index was applied to a group of Social Media Influencers and a general ranking was obtained, as well as a ranking by each approach to measuring engagement. A comparison of the rankings generated by the different approaches shows the suitability and pertinence of both, as it is confirmed that they measure different aspects, and that both are needed to offer a holistic view of the engagement generated by a user on Twitter; this is a new finding compared to prior studies, which only focused on one approach or the other.     

《复印报刊资料》图书馆学情报学期刊排名的实证研究

以《复印报刊资料》(下称资料)研究论文定量分析和2000—2003年图书馆学情报学期刊全文转载排名为基础，通过比较其综合排名与《中文核心期刊要目总览》、《中文社会科学引文索引》、《中国人文社会科学核心期刊要览》的差异和CNKI、《中文科技期刊引文数据库》收录期刊基金论文和被引频次验证排名合理性，结果表明《资料》期刊排名有一定的合理性，基金论文和被引频次是衡量期刊排名的客观指标。     

Entity ranking in Wikipedia: utilising categories,links and topic difficulty prediction

Entity ranking has recently emerged as a research field that aims at retrieving entities as answers to a query. Unlike entity extraction where the goal is to tag names of entities in documents, entity ranking is primarily focused on returning a ranked list of relevant entity names for the query. Many approaches to entity ranking have been proposed, and most of them were evaluated on the INEX Wikipedia test collection. In this paper, we describe a system we developed for ranking Wikipedia entities in answer to a query. The entity ranking approach implemented in our system utilises the known categories, the link structure of Wikipedia, as well as the link co-occurrences with the entity examples (when provided) to retrieve relevant entities as answers to the query. We also extend our entity ranking approach by utilising the knowledge of predicted classes of topic difficulty. To predict the topic difficulty, we generate a classifier that uses features extracted from an INEX topic definition to classify the topic into an experimentally pre-determined class. This knowledge is then utilised to dynamically set the optimal values for the retrieval parameters of our entity ranking system. Our experiments demonstrate that the use of categories and the link structure of Wikipedia can significantly improve entity ranking effectiveness, and that topic difficulty prediction is a promising approach that could also be exploited to further improve the entity ranking performance.     

Learning to rank query suggestions for adhoc and diversity search

Query suggestions have become pervasive in modern web search, as a mechanism to guide users towards a better representation of their information need. In this article, we propose a ranking approach for producing effective query suggestions. In particular, we devise a structured representation of candidate suggestions mined from a query log that leverages evidence from other queries with a common session or a common click. This enriched representation not only helps overcome data sparsity for long-tail queries, but also leads to multiple ranking criteria, which we integrate as features for learning to rank query suggestions. To validate our approach, we build upon existing efforts for web search evaluation and propose a novel framework for the quantitative assessment of query suggestion effectiveness. Thorough experiments using publicly available data from the TREC Web track show that our approach provides effective suggestions for adhoc and diversity search.     

Ranking scientific articles based on bibliometric networks with a weighting scheme

As the volume of scientific articles has grown rapidly over the last decades, evaluating their impact becomes critical for tracing valuable and significant research output. Many studies have proposed various ranking methods to estimate the prestige of academic papers using bibliometric methods. However, the weight of the links in bibliometric networks has been rarely considered for article ranking in existing literature. Such incomplete investigation in bibliometric methods could lead to biased ranking results. Therefore, a novel scientific article ranking algorithm, W-Rank, is introduced in this study proposing a weighting scheme. The scheme assigns weight to the links of citation network and authorship network by measuring citation relevance and author contribution. Combining the weighted bibliometric networks and a propagation algorithm, W-Rank is able to obtain article ranking results that are more reasonable than existing PageRank-based methods. Experiments are conducted on both arXiv hep-th and Microsoft Academic Graph datasets to verify the W-Rank and compare it with three renowned article ranking algorithms. Experimental results prove that the proposed weighting scheme assists the W-Rank in obtaining ranking results of higher accuracy and, in certain perspectives, outperforming the other algorithms.     

How to evaluate rankings of academic entities using test data

In the field of scientometrics, impact indicators and ranking algorithms are frequently evaluated using unlabelled test data comprising relevant entities (e.g., papers, authors, or institutions) that are considered important. The rationale is that the higher some algorithm ranks these entities, the better its performance. To compute a performance score for an algorithm, an evaluation measure is required to translate the rank distribution of the relevant entities into a single-value performance score. Until recently, it was simply assumed that taking the average rank (of the relevant entities) is an appropriate evaluation measure when comparing ranking algorithms or fine-tuning algorithm parameters.With this paper we propose a framework for evaluating the evaluation measures themselves. Using this framework the following questions can now be answered: (1) which evaluation measure should be chosen for an experiment, and (2) given an evaluation measure and corresponding performance scores for the algorithms under investigation, how significant are the observed performance differences?Using two publication databases and four test data sets we demonstrate the functionality of the framework and analyse the stability and discriminative power of the most common information retrieval evaluation measures. We find that there is no clear winner and that the performance of the evaluation measures is highly dependent on the underlying data. Our results show that the average rank is indeed an adequate and stable measure. However, we also show that relatively large performance differences are required to confidently determine if one ranking algorithm is significantly superior to another. Lastly, we list alternative measures that also yield stable results and highlight measures that should not be used in this context.     

A survey of approaches for ranking on the web of data

Ranking information resources is a task that usually happens within more complex workflows and that typically occurs in any form of information retrieval, being commonly implemented by Web search engines. By filtering and rating data, ranking strategies guide the navigation of users when exploring large volumes of information items. There exist a considerable number of ranking algorithms that follow different approaches focusing on different aspects of the complex nature of the problem, and reflecting the variety of strategies that are possible to apply. With the growth of the web of linked data, a new problem space for ranking algorithms has emerged, as the nature of the information items to be ranked is very different from the case of Web pages. As a consequence, existing ranking algorithms have been adapted to the case of Linked Data and some specific strategies have started to be proposed and implemented. Researchers and organizations deploying Linked Data solutions thus require an understanding of the applicability, characteristics and state of evaluation of ranking strategies and algorithms as applied to Linked Data. We present a classification that formalizes and contextualizes under a common terminology the problem of ranking Linked Data. In addition, an analysis and contrast of the similarities, differences and applicability of the different approaches is provided. We aim this work to be useful when comparing different approaches to ranking Linked Data and when implementing new algorithms.     

Application of the Rasch model to measuring the impact of scientific journals

This paper focuses on a fresh and fair way to determine a ranking of science journals according to the “number of citations-to and articles published,” data used by SCI Journal Citation Reports of ISI to determine journal ranking by “impact factor.” Impact is considered a latent variable defined by a set of items (citations and articles published). The theoretical background is Item Response Theory, which suggests that, if we can understand how each item in a set of items operates with an object, then we can estimate a measure for the object. The Rasch model is the most common formulation of that theory. This technique is here applied to the citations and articles published of 62 medical journals (objects) to provide a Rasch measure for these journals which is compared with the current “impact factor” computation.     

国际大学H指数与综合指标排名的比较研究

引进排名基准距,对国际大学h指数排名、上海交通大学的Academic Ranking of World Universities和武汉大学的"世界大学科研竞争力排行榜"进行了比较,分析表明三个排名体系的排名基准距之间存在正相关性。     

Which used product is more sellable? A time-aware approach

A number of online marketplaces enable customers to buy or sell used products, which raises the need for ranking tools to help them find desirable items among a huge pool of choices. To the best of our knowledge, no prior work in the literature has investigated the task of used product ranking which has its unique characteristics compared with regular product ranking. While there exist a few ranking metrics (e.g., price, conversion probability) that measure the “goodness” of a product, they do not consider the time factor, which is crucial in used product trading due to the fact that each used product is often unique while new products are usually abundant in supply or quantity. In this paper, we introduce a novel time-aware metric—“sellability”, which is defined as the time duration for a used item to be traded, to quantify the value of it. In order to estimate the “sellability” values for newly generated used products and to present users with a ranked list of the most relevant results, we propose a combined Poisson regression and listwise ranking model. The model has a good property in fitting the distribution of “sellability”. In addition, the model is designed to optimize loss functions for regression and ranking simultaneously, which is different from previous approaches that are conventionally learned with a single cost function, i.e., regression or ranking. We evaluate our approach in the domain of used vehicles. Experimental results show that the proposed model can improve both regression and ranking performance compared with non-machine learning and machine learning baselines.     

Learning to rank with (a lot of) word features

In this article we present Supervised Semantic Indexing which defines a class of nonlinear (quadratic) models that are discriminatively trained to directly map from the word content in a query-document or document-document pair to a ranking score. Like Latent Semantic Indexing (LSI), our models take account of correlations between words (synonymy, polysemy). However, unlike LSI our models are trained from a supervised signal directly on the ranking task of interest, which we argue is the reason for our superior results. As the query and target texts are modeled separately, our approach is easily generalized to different retrieval tasks, such as cross-language retrieval or online advertising placement. Dealing with models on all pairs of words features is computationally challenging. We propose several improvements to our basic model for addressing this issue, including low rank (but diagonal preserving) representations, correlated feature hashing and sparsification. We provide an empirical study of all these methods on retrieval tasks based on Wikipedia documents as well as an Internet advertisement task. We obtain state-of-the-art performance while providing realistically scalable methods.     

Rating Pakistani medical journals using the principles of citation analysis

OBJECTIVE: To quantify the impact of Pakistani Medical Journals using the principles of citation analysis. METHODS: References of articles published in 2006 in three selected Pakistani medical journals were collected and examined. The number of citations for each Pakistani medical journal was totalled. The first ranking of journals was based on the total number of citations; second ranking was based on impact factor 2006 and third ranking was based on the 5-year impact factor. Self-citations were excluded in all the three ratings. RESULTS: A total of 9079 citations in 567 articles were examined. Forty-nine separate Pakistani medical journals were cited. The Journal of the Pakistan Medical Association remains on the top in all three rankings, while Journal of College of Physicians and Surgeons-Pakistan attains second position in the ranking based on the total number of citations. The Pakistan Journal of Medical Sciences moves to second position in the ranking based on the impact factor 2006. The Journal of Ayub Medical College, Abbottabad moves to second position in the ranking based on the 5-year impact factor. CONCLUSION: This study examined the citation pattern of Pakistani medical journals. The impact factor, despite its limitations, is a valid indicator of quality for journals.     

时间加权z指数探测热点关键词研究

[目的/意义]构建关键词热度h值模型,探究近年图书情报领域热点与发展趋势。[方法/过程]基于学者Prathp的z指数模型,对关键词词频和文章被引值进行赋权,引入前人时间加权思想,多维角度归一化处理,计算热度值与年度排名,获得趋势。[结果/结论]比较加权前后绝对词频、z指数和关键词热度h值,排名结果差异明显;热度h值模型可提升热点关键词排名,拉低非热点关键词排名,排名情况验证h值的有效性,效果优良。     

A GP-adaptive web ranking discovery framework based on combinative content and context features

The problem of ranking is a crucial task in the web information retrieval systems. The dynamic nature of information resources as well as the continuous changes in the information demands of the users has made it very difficult to provide effective methods for data mining and document ranking. Regarding these challenges, in this paper an adaptive ranking algorithm is proposed named GPRank. This algorithm which is a function discovery framework, utilizes the relatively simple features of web documents to provide suitable rankings using a multi-layer/multi-population genetic programming architecture. Experiments done, illustrate that GPRank has better performance in comparison with well-known ranking techniques and also against its full mode edition.     

Contents and time sensitive document ranking of scientific literature

A new link-based document ranking framework is devised with at its heart, a contents and time sensitive random literature explorer designed to more accurately model the behaviour of readers of scientific documents. In particular, our ranking framework dynamically adjusts its random walk parameters according to both contents and age of encountered documents, thus incorporating the diversity of topics and how they evolve over time into the score of a scientific publication. Our random walk framework results in a ranking of scientific documents which is shown to be more effective in facilitating literature exploration than PageRank measured against a proxy gold standard based on papers’ potential usefulness in facilitating later research. One of its many strengths lies in its practical value in reliably retrieving and placing promisingly useful papers at the top of its ranking.     

First in,best dressed: The presence of order-effect bias in journal ranking surveys

The purpose of this study is to test for the presence of order-effect bias in journal ranking surveys. Data were obtained from 379 active knowledge management and intellectual capital researchers who rated 25 journals on a 7-point scale. Five different versions of the survey instrument were utilized. Consistent with the cognitive elaboration model, the satisficing theory, and the Gricean maxim of orderliness, order-effect bias was observed in journal ranking surveys. Journals that appear in the beginning of the ranking list delivered to survey respondents consistently receive higher scores than journals at the end of the list. Overall, the position of the journal in the list explains over 10% of its score. Therefore, authors of journal ranking studies are recommended to use multiple versions of the survey instrument with randomized journal orders.