The lognormal distribution explains the remarkable pattern documented by characteristic scores and scales in scientometrics

Characteristic scores and scales (CSS) – a well-established scientometric tool for the study of citation counts – have been used to document a striking phenomenon that characterizes citation distributions at high levels of aggregation: irrespective of scientific field and citation window empirical studies find a persistent pattern whereby about 70% of scientific papers belong to the class of poorly cited papers, about 21% belong to the class of fairly cited papers, 6% to that of remarkably cited papers and 3% to the class of outstandingly cited papers. This article aims to advance the understanding of this remarkable result by examining it in the context of the lognormal distribution, a popular model used to describe citation counts across scientific fields. The article shows that the application of the CSS method to lognormal distributions provides a very good fit to the 70–21–6–3% empirical pattern provided these distributions are characterized by a standard deviation parameter in the range of about 0.8–1.3. The CSS pattern is essentially explainable as an epiphenomenon of the lognormal functional form and, more generally, as a consequence of the skewness of science which is manifest in heavy-tailed citation distributions.     

Hidden scales in statistics of citation indicators

Scholarly citations – widely seen as tangible measures of the impact and significance of academic papers – guide critical decisions by research administrators and policy makers. The citation distributions form characteristic patterns that can be revealed by big-data analysis. However, the citation dynamics varies significantly among subject areas, countries etc. The problem is how to quantify those differences, separate global and local citation characteristics. Here, we carry out an extensive analysis of the power-law relationship between the total citation count and the h-index to detect a functional dependence among its parameters for different science domains. The results demonstrate that the statistical structure of the citation indicators admits representation by a global scale and a set of local exponents. The scale parameters are evaluated for different research actors – individual researchers and entire countries – employing subject- and affiliation-based divisions of science into domains. The results can inform research assessment and classification into subject areas; the proposed divide-and-conquer approach can be applied to hidden scales in other power-law systems.     

The skewness of scientific productivity

This paper exploits a unique 2003–2011 large dataset, indexed by Thomson Reuters, consisting of 17.2 million disambiguated authors classified into 30 broad scientific fields, as well as the 48.2 million articles resulting from a multiplying strategy in which any article co-authored by two or more persons is wholly assigned as many times as necessary to each of them. The dataset is characterized by a large proportion of authors who have their oeuvre in several fields. We measure individual productivity in two ways that are uncorrelated: as the number of articles per person and as the mean citation per article per person in the 2003–2011 period. We analyze the shape of the two types of individual productivity distributions in each field using size- and scale-independent indicators. To assess the skewness of productivity distributions we use a robust index of skewness, as well as the Characteristic Scores and Scales approach. For productivity inequality, we use the coefficient of variation. In each field, we study two samples: the entire population, and what we call “successful authors”, namely, the subset of scientists whose productivity is above their field average. The main result is that, in spite of wide differences in production and citation practices across fields, the shape of field productivity distributions is very similar across fields. The parallelism of the results for the population as a whole and for the subset of successful authors, when productivity is measured as mean citation per article per person, reveals the fractal nature of the skewness of scientific productivity in this case. These results are essentially maintained when any article co-authored by two or more persons is fractionally assigned to each of them.     

The use of the Percentage Rank Position index for comparative evaluation of journals

In the present paper the Percentage Rank Position (PRP) index concluding from the principle of Similar Distribution of Information Impact in different fields of science (Vinkler, 2013), is suggested to assess journals in different research fields comparatively. The publications in the journals dedicated to a field are ranked by citation frequency, and the PRP-index of the papers in the elite set of the field is calculated. The PRP-index relates the citation rank number of the paper to the total number of papers in the corresponding set. The sum of the PRP-index of the elite papers in a journal, PRP(j,F) may represent the eminence of the journal in the field. The non-parametric and non-dimensional PRP(j,F) index of journals is believed to be comparable across fields.     

Impact maturity times and citation time windows: The 2-year maximum journal impact factor

Journal metrics are employed for the assessment of scientific scholar journals from a general bibliometric perspective. In this context, the Thomson Reuters journal impact factors (JIFs) are the citation-based indicators most used. The 2-year journal impact factor (2-JIF) counts citations to one and two year old articles, while the 5-year journal impact factor (5-JIF) counts citations from one to five year old articles. Nevertheless, these indicators are not comparable among fields of science for two reasons: (i) each field has a different impact maturity time, and (ii) because of systematic differences in publication and citation behavior across disciplines. In fact, the 5-JIF firstly appeared in the Journal Citation Reports (JCR) in 2007 with the purpose of making more comparable impacts in fields in which impact matures slowly. However, there is not an optimal fixed impact maturity time valid for all the fields. In some of them two years provides a good performance whereas in others three or more years are necessary. Therefore, there is a problem when comparing a journal from a field in which impact matures slowly with a journal from a field in which impact matures rapidly. In this work, we propose the 2-year maximum journal impact factor (2M-JIF), a new impact indicator that considers the 2-year rolling citation time window of maximum impact instead of the previous 2-year time window. Finally, an empirical application comparing 2-JIF, 5-JIF, and 2M-JIF shows that the maximum rolling target window reduces the between-group variance with respect to the within-group variance in a random sample of about six hundred journals from eight different fields.     

Research assessment by percentile-based double rank analysis

In the double rank analysis of research publications, the local rank position of a country or institution publication is expressed as a function of the world rank position. Excluding some highly or lowly cited publications, the double rank plot fits well with a power law, which can be explained because citations for local and world publications follow lognormal distributions. We report here that the distribution of the number of country or institution publications in world percentiles is a double rank distribution that can be fitted to a power law. Only the data points in high percentiles deviate from it when the local and world μ parameters of the lognormal distributions are very different. The likelihood of publishing very highly cited papers can be calculated from the power law that can be fitted either to the upper tail of the citation distribution or to the percentile-based double rank distribution. The great advantage of the latter method is that it has universal application, because it is based on all publications and not just on highly cited publications. Furthermore, this method extends the application of the well-established percentile approach to very low percentiles where breakthroughs are reported but paper counts cannot be performed.     

Characterizing in-text citations in scientific articles: A large-scale analysis

We report characteristics of in-text citations in over five million full text articles from two large databases – the PubMed Central Open Access subset and Elsevier journals – as functions of time, textual progression, and scientific field. The purpose of this study is to understand the characteristics of in-text citations in a detailed way prior to pursuing other studies focused on answering more substantive research questions. As such, we have analyzed in-text citations in several ways and report many findings here. Perhaps most significantly, we find that there are large field-level differences that are reflected in position within the text, citation interval (or reference age), and citation counts of references. In general, the fields of Biomedical and Health Sciences, Life and Earth Sciences, and Physical Sciences and Engineering have similar reference distributions, although they vary in their specifics. The two remaining fields, Mathematics and Computer Science and Social Science and Humanities, have different reference distributions from the other three fields and between themselves. We also show that in all fields the numbers of sentences, references, and in-text mentions per article have increased over time, and that there are field-level and temporal differences in the numbers of in-text mentions per reference. A final finding is that references mentioned only once tend to be much more highly cited than those mentioned multiple times.     

Individual and field citation distributions in 29 broad scientific fields

Using an initial dataset consisting of 18.5 million distinct authors and 15 million distinct articles published in the period 2000–2016, which are classified into 29 broad scientific fields, we search for regularities at the individual level for very productive authors with citation distributions of a certain size, and for the existence of a macro-micro relationship between the skewness of a scientific field citation distribution and the characteristics of the individual citation distributions of the authors belonging to the field. Our main results are the following three. Firstly, although the skewness of individual citation distributions varies greatly within each field, their average skewness is of a similar order of magnitude in all fields. Secondly, as in the previous literature, field citation distributions are highly skewed and the degree of skewness is very similar across fields. Thirdly, the skewness of field citation distributions is essentially explained in terms of the average skewness of individual authors, as well as individuals’ differences in mean citation rates and the number of publications per author. These results have important conceptual and practical consequences: to understand the skewness of field citation distributions at any aggregate level we must simply explain the skewness of the individual citation distributions of their very productive authors.     

Revisiting the scaling of citations for research assessment

Over the past decade, national research evaluation exercises, traditionally conducted using the peer review method, have begun opening to bibliometric indicators. The citations received by a publication are assumed as proxy for its quality, but they require standardization prior to use in comparative evaluation of organizations or individual scientists: the citation data must be standardized, due to the varying citation behavior across research fields. The objective of this paper is to compare the effectiveness of the different methods of normalizing citations, in order to provide useful indications to research assessment practitioners. Simulating a typical national research assessment exercise, he analysis is conducted for all subject categories in the hard sciences and is based on the Thomson Reuters Science Citation Index-Expanded^®. Comparisons show that the citations average is the most effective scaling parameter, when the average is based only on the publications actually cited.     

Comparison of two article-level,field-independent citation metrics: Field-Weighted Citation Impact (FWCI) and Relative Citation Ratio (RCR)

We reproduce the article-level, field-independent citation metric Relative Citation Ratio (RCR) using the Scopus database, and extend it beyond the biomedical field to all subject areas. We compare the RCR to the Field-Weighted Citation Impact (FWCI), also an article-level, field-normalised metric, and present the first results of correlations, distributions and application to research university benchmarking for both metrics. Our analyses demonstrate that FWCI and RCR of articles correlate with varying strengths across different areas of research. Additionally, we observe that both metrics are comparably stable across different subject areas of research. Moreover, at the level of universities, both metrics correlate strongly.     

Measuring contextual citation impact of scientific journals

This paper explores a new indicator of journal citation impact, denoted as source normalized impact per paper (SNIP). It measures a journal's contextual citation impact, taking into account characteristics of its properly defined subject field, especially the frequency at which authors cite other papers in their reference lists, the rapidity of maturing of citation impact, and the extent to which a database used for the assessment covers the field's literature. It further develops Eugene Garfield's notions of a field's ‘citation potential’ defined as the average length of references lists in a field and determining the probability of being cited, and the need in fair performance assessments to correct for differences between subject fields. A journal's subject field is defined as the set of papers citing that journal. SNIP is defined as the ratio of the journal's citation count per paper and the citation potential in its subject field. It aims to allow direct comparison of sources in different subject fields. Citation potential is shown to vary not only between journal subject categories – groupings of journals sharing a research field – or disciplines (e.g., journals in mathematics, engineering and social sciences tend to have lower values than titles in life sciences), but also between journals within the same subject category. For instance, basic journals tend to show higher citation potentials than applied or clinical journals, and journals covering emerging topics higher than periodicals in classical subjects or more general journals. SNIP corrects for such differences. Its strengths and limitations are critically discussed, and suggestions are made for further research. All empirical results are derived from Elsevier's Scopus.     

Double rank analysis for research assessment

Reliable methods for the assessment of research success are still in discussion. One method, which uses the likelihood of publishing very highly cited papers, has been validated in terms of Nobel prizes garnered. However, this method cannot be applied widely because it uses the fraction of publications in the upper tail of citation distribution that follows a power law, which includes a low number of publications in most countries and institutions. To achieve the same purpose without restrictions, we have developed the double rank analysis, in which publications that have a low number of citations are also included. By ranking publications by their number of citations from highest to lowest, publications from institutions or countries have two ranking numbers: one for their internal and another one for world positions; the internal ranking number can be expressed as a function of the world ranking number. In log–log double rank plots, a large number of publications fit a straight line; extrapolation allows estimating the likelihood of publishing the highest cited publication. The straight line derives from a power law behavior of the double rank that occurs because citations follow lognormal distributions with values of μ and σ that vary within narrow limits.     

Modeling a century of citation distributions

The prevalence of uncited papers or of highly cited papers, with respect to the bulk of publications, provides important clues as to the dynamics of scientific research. Using 25 million papers and 600 million references from the Web of Science over the 1900–2006 period, this paper proposes a simple model based on a random selection process to explain the “uncitedness” phenomenon and its decline over the years. We show that the proportion of cited papers is a function of (1) the number of articles available (the competing papers), (2) the number of citing papers and (3) the number of references they contain. Using uncitedness as a departure point, we demonstrate the utility of the stretched-exponential function and a form of the Tsallis q-exponential function to fit complete citation distributions over the 20th century. As opposed to simple power-law fits, for instance, both these approaches are shown to be empirically well-grounded and robust enough to better understand citation dynamics at the aggregate level. On the basis of these models, we provide quantitative evidence and provisional explanations for an important shift in citation practices around 1960. We also propose a revision of the “citation classic” category as a set of articles which is clearly distinguishable from the rest of the field.     

Comparison of the citation distribution and h-index between groups of different sizes

Evaluating the performance of institutions with different resources is not easy, any citation distribution comparisons are strongly affected by the differences in the number of articles published. The paper introduces a method for comparing citation distributions of research groups that differ in size. The citation distribution of a larger group is reduced by a certain factor and compared with the original distribution of a smaller group. Expected values and tolerance intervals of the reduced set of citations are calculated. A comparison of both distributions can be conveniently viewed in a graph. The size-independent reduced Hirsch index – a function of reducing factor that allows the comparison of groups within a scientific field – is calculated in the same way. The method can be used for comparing groups or units differing in full-time equivalent, funding or the number of researchers, for comparing countries by population, gross domestic product, etc. It is shown that for the calculation of the reduced Hirsch index, the upper part of the original citation distribution is sufficient. The method is illustrated through several case comparisons.     

引文曲线的分析框架研究——以诺贝尔奖得主的引文曲线为例

基于341位诺贝尔物理学、化学、生理学或医学、经济学奖获得者的引文曲线,借助曲线拟合方法构建引文曲线的分析框架,包括两种规则引文曲线——经典引文曲线、指数增长引文曲线,和三种不规则引文曲线——睡美人引文曲线、双峰引文曲线、波型引文曲线。以诺贝尔奖得主为例的实证分析表明:①引文曲线的分析框架是一种新的引文分析视角;②引文曲线的分析框架可用于将一组(有一定影响力的)论文或作者划分层次,也可用于分析不同学科论文或作者的引用差异。文章最后分析了该框架的适用性。图4。表1。参考文献32。     

PCSI:一种单篇论文被引频次标准化方法

[目的/意义] 文章的被引频次一直是量化评价一篇论文学术影响力的重要指标。但在不同学科不同年份发表的论文会因该领域研究论文数、引用滞后等因素呈现较大的差异。因此在对比两篇论文时，难以简单依据被引频次的绝对值来评判论文影响力大小。为此，本文设计了一个新的可计算数学模型，使得每篇论文可以有一个标准化的指标，以便对不同学科不同年份发表的论文的学术影响力进行直接比较。[方法/过程] 通过分析2006、2017两年中国科技类学术期刊各学科论文的被引频次分布规律，采用同学科论文被引频次的分布形态最接近对数正态分布的先设条件，提出一种被引频次标准化指数——Paper Citation Standardized Index （简称PCSI，中文"论文引证标准化指数"）。最后以中国科协优秀科技期刊论文评选结果为例，将它们与论文所属学科全部论文进行实证对比研究。[结果/结论] 结果证明，PCSI对不同年份、不同学科论文的被引频次进行了标准化，反映了被引频次的线性差距，是一种较为理想的单篇论文学术影响力比较评价工具。     

A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications

Combining different data sets with information on grant and fellowship applications submitted to two renowned funding agencies, we are able to compare their funding decisions (award and rejection) with scientometric performance indicators across two fields of science (life sciences and social sciences). The data sets involve 671 applications in social sciences and 668 applications in life sciences. In both fields, awarded applicants perform on average better than all rejected applicants. If only the most preeminent rejected applicants are considered in both fields, they score better than the awardees on citation impact. With regard to productivity we find differences between the fields. While the awardees in life sciences outperform on average the most preeminent rejected applicants, the situation is reversed in social sciences.     

PCSI:一种单篇论文被引频次标准化方法

[目的/意义]文章的被引频次一直是量化评价一篇论文学术影响力的重要指标。但在不同学科不同年份发表的论文会因该领域研究论文数、引用滞后等因素呈现较大的差异。因此在对比两篇论文时,难以简单依据被引频次的绝对值来评判论文影响力大小。为此,本文设计了一个新的可计算数学模型,使得每篇论文可以有一个标准化的指标,以便对不同学科不同年份发表的论文的学术影响力进行直接比较。[方法/过程]通过分析2006、2017两年中国科技类学术期刊各学科论文的被引频次分布规律,采用同学科论文被引频次的分布形态最接近对数正态分布的先设条件,提出一种被引频次标准化指数——Paper Citation Standardized Index (简称PCSI,中文"论文引证标准化指数")。最后以中国科协优秀科技期刊论文评选结果为例,将它们与论文所属学科全部论文进行实证对比研究。[结果/结论]结果证明,PCSI对不同年份、不同学科论文的被引频次进行了标准化,反映了被引频次的线性差距,是一种较为理想的单篇论文学术影响力比较评价工具。     

Scoring research output using statistical quantile plotting

In this paper, we propose two methods for scoring scientific output based on statistical quantile plotting. First, a rescaling of journal impact factors for scoring scientific output on a macro level is proposed. It is based on normal quantile plotting which allows to transform impact data over several subject categories to a standardized distribution. This can be used in comparing scientific output of larger entities such as departments working in quite different areas of research. Next, as an alternative to the Hirsch index [Hirsch, J.E. (2005). An index to quantify an individuals scientific research output. Proceedings of the National Academy of Sciences of the United States of America, 102(46), 16569–16572], the extreme value index is proposed as an indicator for assessment of the research performance of individual scientists. In case of Lotkaian–Zipf–Pareto behaviour of citation counts of an individual, the extreme value index can be interpreted as the slope in a Pareto–Zipf quantile plot. This index, in contrast to the Hirsch index, is not influenced by the number of publications but stresses the decay of the statistical tail of citation counts. It appears to be much less sensitive to the science field than the Hirsch index.     

Comparison of different mathematical functions for the analysis of citation distribution of papers of individual authors

The citation distribution of papers of selected individual authors was analyzed using five mathematical functions: power-law, stretched exponential, logarithmic, binomial and Langmuir-type. The former two functions have previously been proposed in the literature whereas the remaining three are novel and are derived following the concepts of growth kinetics of crystals in the presence of additives which act as inhibitors of growth. Analysis of the data of citation distribution of papers of the authors revealed that the value of the goodness-of-the-fit parameter R² was the highest for the empirical binomial relation, it was high and comparable for stretched exponential and Langmuir-type functions, relatively low for power law but it was the lowest for the logarithmic function. In the Langmuir-type function a parameter K, defined as Langmuir constant, characterizing the citation behavior of the authors has been identified. Based on the Langmuir-type function an expression for cumulative citations L relating the extrapolated value of citations l₀ corresponding to rank n = 0 for an author and his/her constant K and the number N of paper receiving citation l ≥ 1 is also proposed.