Estimating the accuracies of journal impact factor through bootstrap

The journal impact factor (JIF) reported in journal citation reports has been used to represent the influence and prestige of a journal. Whereas the consideration of the stochastic nature of a statistic is a prerequisite for statistical inference, the estimation of JIF uncertainty is necessary yet unavailable for comparing the impact among journals. Using journals in the Database of Research in Science Education (DoRISE), the current study proposes bootstrap methods to estimate the JIF variability. The paper also provides a comprehensive exposition of the sources of JIF variability. The collections of articles in the year of interest and in the preceding years both contribute to JIF variability. In addition, the variability estimate differs depending on the way a database selects its journals for inclusion. In the bootstrap process, the nested structure of articles in a journal was accounted for to ensure that each bootstrap replication reflects the actual citation characteristics of articles in the journal. In conclusion, the proposed point and interval estimates of the JIF statistic are obtained and more informative inferences on the impact of journals can be drawn.     

Publication delay adjusted impact factor: The effect of publication delay of articles on journal impact factor

The journal impact factor (JIF) has been questioned considerably during its development in the past half-century because of its inconsistency with scholarly reputation evaluations of scientific journals. This paper proposes a publication delay adjusted impact factor (PDAIF) which takes publication delay into consideration to reduce the negative effect on the quality of the impact factor determination. Based on citation data collected from Journal Citation Reports and publication delay data extracted from the journals’ official websites, the PDAIFs for journals from business-related disciplines are calculated. The results show that PDAIF values are, on average, more than 50% higher than JIF results. Furthermore, journal ranking based on PDAIF shows very high consistency with reputation-based journal rankings. Moreover, based on a case study of journals published by ELSEVIER and INFORMS, we find that PDAIF will bring a greater impact factor increase for journals with longer publication delay because of reducing that negative influence. Finally, insightful and practical suggestions to shorten the publication delay are provided.     

A comparison among citation-based journal indicators and their relative changes with time

This study compares the two-year impact factor (JIF2), JIF2 without journal self-citation (JIF2_noJSC), five-year impact factor (JIF5), eigenfactor score and article influence score (AIS) and investigates their relative changes with time. JIF2 increased faster than JIF5 overall. The relative change between JIF2 and JIF_noJSC shows that the control of JCR over journal self-citation is effective to some extent. JIF5 is more discriminative than JIF2. The correlation between JIF5 and AIS is stronger than that between JIF5 and the eigenfactor score. The relative change in journal rank according to different indicators varies with the ratio of the indicators and can be up to 60 % of the number of journals in a subject category. There is subject category discrepancy in the average AIS and its change over time. Through the screening of journals according to variations in the ratio of JIF2 to JIF5 for journals in individual subject categories, we found that journals in the same subject categories can have considerably different citation patterns. To provide a fair comparison of journals in individual subject categories, we argue that it is better to replace JIF2 with the ready-made JIF5 when ranking journals.     

Impact maturity times and citation time windows: The 2-year maximum journal impact factor

Journal metrics are employed for the assessment of scientific scholar journals from a general bibliometric perspective. In this context, the Thomson Reuters journal impact factors (JIFs) are the citation-based indicators most used. The 2-year journal impact factor (2-JIF) counts citations to one and two year old articles, while the 5-year journal impact factor (5-JIF) counts citations from one to five year old articles. Nevertheless, these indicators are not comparable among fields of science for two reasons: (i) each field has a different impact maturity time, and (ii) because of systematic differences in publication and citation behavior across disciplines. In fact, the 5-JIF firstly appeared in the Journal Citation Reports (JCR) in 2007 with the purpose of making more comparable impacts in fields in which impact matures slowly. However, there is not an optimal fixed impact maturity time valid for all the fields. In some of them two years provides a good performance whereas in others three or more years are necessary. Therefore, there is a problem when comparing a journal from a field in which impact matures slowly with a journal from a field in which impact matures rapidly. In this work, we propose the 2-year maximum journal impact factor (2M-JIF), a new impact indicator that considers the 2-year rolling citation time window of maximum impact instead of the previous 2-year time window. Finally, an empirical application comparing 2-JIF, 5-JIF, and 2M-JIF shows that the maximum rolling target window reduces the between-group variance with respect to the within-group variance in a random sample of about six hundred journals from eight different fields.     

Journal weighted impact factor: A proposal

The impact factor of a journal reflects the frequency with which the journal's articles are cited. It is the best available measure of journal quality. For calculation of impact factor, we just count the number of citations, no matter how prestigious the citing journal is. We think that impact factor as a measure of journal quality, may be improved if in its calculation, we not only take into account the number of citations, but also incorporate a factor reflecting the prestige of the citing journals relative to the cited journal. In calculation of this proposed “weighted impact factor,” each citation has a coefficient (weight) the value of which is 1 if the citing journal is as prestigious as the cited journal; is >1 if the citing journal is more prestigious than the cited journal; and is <1 if the citing journal has a lower standing than the cited journal. In this way, journals receiving many citations from prestigious journals are considered prestigious themselves and those cited by low-status journals seek little credit. By considering both the number of citations and the prestige of the citing journals, we expect the weighted impact factor be a better scientometrics measure of journal quality.     

期刊引用认同指标在期刊评价中的适用性分析

论文以CSSCI图书情报领域的18种期刊为例,以这些期刊在2009年全年登载论文的参考文献为研究对象,从CSSCI数据库中获取数据,统计分析各期刊的引用认同。结果显示:期刊引用认同指标(引文量、篇均引文量、英文引文比、期刊引用广度、自施引率、引用半衰期、期刊集中因子、认同期刊影响力等指标)与CSSCI来源期刊定量与定性评价指标并不明显相关,但这类指标可以反映期刊载文的内容特征与偏好、对国外科学文献和对其他学科文献的利用程度、期刊的办刊定位、学科的发展模式等等,在综合评价期刊方面具有一定意义。     

论文发表时滞与优先数字出版

将论文发表时滞分为审稿时滞与等待印刷时滞,解释了论文从投稿到发表的过程中各个环节所产生的时滞及其所产生的负面影响。分析优先数字出版在大幅缩短论文发表时滞方面的功能与意义,统计表明,优先数字出版能将期刊影响因子提高约15％。提出了优先数字出版中值得讨论的问题。     

Skewed citation distributions and bias factors: Solutions to two core problems with the journal impact factor

The journal impact factor (JIF) proposed by Garfield in the year 1955 is one of the most prominent and common measures of the prestige, position, and importance of a scientific journal. The JIF may profit from its comprehensibility, robustness, methodological reproducibility, simplicity, and rapid availability, but it is at the expense of serious technical and methodological flaws. The paper discusses two core problems with the JIF: first, citations of documents are generally not normally distributed, and, furthermore, the distribution is affected by outliers, which has serious consequences for the use of the mean value in the JIF calculation. Second, the JIF is affected by bias factors that have nothing to do with the prestige or quality of a journal (e.g., document type). For solving these two problems, we suggest using McCall's area transformation and the Rubin Causal Model. Citation data for documents of all journals in the ISI Subject Category “Psychology, Mathematical” (Journal Citation Report) are used to illustrate the proposal.     

Some comments on the journal weighted impact factor proposed by Habibzadeh and Yadollahie

In a recent paper in the Journal of Informetrics, Habibzadeh and Yadollahie [Habibzadeh, F., & Yadollahie, M. (2008). Journal weighted impact factor: A proposal. Journal of Informetrics, 2(2), 164–172] propose a journal weighted impact factor (WIF). Unlike the ordinary impact factor, the WIF of a journal takes into account the prestige or the influence of citing journals. In this communication, we show that the way in which Habibzadeh and Yadollahie calculate the WIF of a journal has some serious problems. Due to these problems, a ranking of journals based on WIF can be misleading. We also indicate how the problems can be solved by changing the way in which the WIF of a journal is calculated.     

Evaluating research and researchers by the journal impact factor: Is it better than coin flipping?

The journal impact factor (JIF) is the average of the number of citations of the papers published in a journal, calculated according to a specific formula; it is extensively used for the evaluation of research and researchers. The method assumes that all papers in a journal have the same scientific merit, which is measured by the JIF of the publishing journal. This implies that the number of citations measures scientific merits but the JIF does not evaluate each individual paper by its own number of citations. Therefore, in the comparative evaluation of two papers, the use of the JIF implies a risk of failure, which occurs when a paper in the journal with the lower JIF is compared to another with fewer citations in the journal with the higher JIF. To quantify this risk of failure, this study calculates the failure probabilities, taking advantage of the lognormal distribution of citations. In two journals whose JIFs are ten-fold different, the failure probability is low. However, in most cases when two papers are compared, the JIFs of the journals are not so different. Then, the failure probability can be close to 0.5, which is equivalent to evaluating by coin flipping.     

Measuring contextual citation impact of scientific journals

This paper explores a new indicator of journal citation impact, denoted as source normalized impact per paper (SNIP). It measures a journal's contextual citation impact, taking into account characteristics of its properly defined subject field, especially the frequency at which authors cite other papers in their reference lists, the rapidity of maturing of citation impact, and the extent to which a database used for the assessment covers the field's literature. It further develops Eugene Garfield's notions of a field's ‘citation potential’ defined as the average length of references lists in a field and determining the probability of being cited, and the need in fair performance assessments to correct for differences between subject fields. A journal's subject field is defined as the set of papers citing that journal. SNIP is defined as the ratio of the journal's citation count per paper and the citation potential in its subject field. It aims to allow direct comparison of sources in different subject fields. Citation potential is shown to vary not only between journal subject categories – groupings of journals sharing a research field – or disciplines (e.g., journals in mathematics, engineering and social sciences tend to have lower values than titles in life sciences), but also between journals within the same subject category. For instance, basic journals tend to show higher citation potentials than applied or clinical journals, and journals covering emerging topics higher than periodicals in classical subjects or more general journals. SNIP corrects for such differences. Its strengths and limitations are critically discussed, and suggestions are made for further research. All empirical results are derived from Elsevier's Scopus.     

“影响因子百分位”指标的特点研究

[目的/意义] 分析JCR最新公布的"影响因子百分位"指标的特点以及其对期刊评价的影响。[方法/过程] 从影响因子百分位的内涵分析入手,比较其与影响因子的统计学特征,在因子分析的基础上,采用多元回归和分位数回归研究影响因子百分位与其他文献计量指标之间的关系,同时采用Spearman相关系数分析其与其他文献计量指标的关系。[结果/结论] 研究结果表明,影响因子百分位改变了影响因子的统计学特征;影响因子较低的一些期刊,转换成影响因子百分位后相对值更低,其他期刊转换成影响因子百分位后相对值更高;影响因子百分位与其他文献计量指标相关程度中等;当影响因子百分位较低时,影响因子相关指标的弹性系数较高,当影响因子百分位处于中等时,总量指标及特征因子指标的回归系数较低;影响因子百分位并不适宜作为其中一个指标用于期刊多属性评价。     

科技期刊影响力数据稳定性的数学描述

参照统计学中的置信限的概念,定义了影响力数据的置信限和稳定度。这一定义不仅综合考虑了影响因子、特征因子、总被引频次、载文量等文献计量指标,而且具有计算简便、可操作性较强的特点。通过对工程技术类79种期刊的实例计算与分析,进一步论证了有关影响因子的置信限与稳定度在研究评价和期刊评估中的实用性。     

影响科技期刊影响因子的期刊因素分析与思考 ——以SCI收录的眼科类期刊为例

统计了SCI收录的16种眼科类高影响因子期刊2007—2011年影响因子的变化,通过分析16种眼科期刊2005—2010年办刊策略的变化,从办刊策略层面寻找影响期刊影响因子的相关因素。研究发现,期刊影响力、栏目设置、期刊载文量、网站建设和出版周期等是影响期刊影响因子的重要因素。     

Online journals: impact on print journal usage

PURPOSE: The research sought to determine the impact of online journals on the use of print journals and interlibrary loan (ILL). SETTING: The Library of the Health Sciences-Peoria is a regional site of the University of Illinois at Chicago (UIC) Library with a print journal collection of approximately 400 titles. Since 1999, UIC site licenses have given students and faculty affiliated with UIC-Peoria access to more than 4,000 online full-text journal titles through the Internet. METHODOLOGY: The Library of the Health Sciences-Peoria has conducted a journal-use study over an extended period of time. The information collected from this study was used to assess the impact of 104 online journals, added to the collection in January 1999, on the use of print journals. RESULTS: Results of the statistical analysis showed print journal usage decreased significantly since the introduction of online journals (F(1,147) = 12.10, P < 0.001). This decrease occurred regardless of whether a journal was available only in print or both online and in print. Interlibrary loan requests have also significantly decreased since the introduction of online journals (F(2,30) = 4.46, P < 0.02). CONCLUSIONS: The decrease in use of the print collection suggests that many patrons prefer to access journals online. The negative impact the online journals have had on the use of the journal titles available only in print suggests users may be compromising quality for convenience when selecting journal articles. Possible implications for collection development are discussed.     

Update on Journal Impact Ratings in Communication: 2006–2008

Bibliometric data indexed through the Institute for Scientific Information were analyzed for 45 communication journals. Several measures were included to identify the most widely cited journals in the field, including (a) journal impact factor, (b) five-year journal impact, (c) article influence, and (d) journal relatedness. Results serve to expand on findings by Feeley (2008 Feeley , T. H. ( 2008 ). A bibliometric analysis of communication journals from 2002 to 2005 . Human Communication Research , 34 , 505 – 520 .[Crossref], [Web of Science ®] , [Google Scholar]) with respect to overall and within-field influence of communication journals whose analysis covered 2002 through 2005 and 19 journals. Results indicate stability in journal impact ratings over time and several journals (e.g., Communication Research, Human Communication Research, Journal of Communication, Communication Monographs, and Communication Theory) are highly central in the communication journal citation network.     

高影响力论文是名牌医学期刊的特征

在某一学术领域内,某一篇论文的被引用次数在某种程度上可以衡量该论文在某一时期内的学术影响.<中国高影响力医学期刊论文计量报告>一书详细而科学、客观地揭示了中国高影响力医学论文的被引频次和专业分类.本文在此基础上进一步分析被引80次以上的299篇中国高影响力医学论文的来源期刊的情况,认为高影响力论文是名牌医学期刊的特征.     

The Reference Return Ratio

The paper introduces a new journal impact measure called The Reference Return Ratio (3R). Unlike the traditional Journal Impact Factor (JIF), which is based on calculations of publications and citations, the new measure is based on calculations of bibliographic investments (references) and returns (citations). A comparative study of the two measures shows a strong relationship between the 3R and the JIF. Yet, the 3R appears to correct for citation habits, citation dynamics, and composition of document types – problems that typically are raised against the JIF. In addition, contrary to traditional impact measures, the 3R cannot be manipulated ad infinitum through journal self-citations.     

自引对科技期刊影响因子作用的量化研究

以中国科技论文统计源数据库收录的70种中华医学会系列期刊为样本,通过计算得出这些期刊去除自引后的影响因子,并对去除前后期刊的影响因子及期刊排名进行比较.通过量化统计分析发现,自引对期刊的影响因子和排名影响显著(P＜0.01),进而影响对科技期刊的评价.     

论期刊影响因子人为操纵的识别

近年来,期刊影响因子的人为操纵受到学术界广泛关注,期刊过度自引成为人为操纵影响因子的重要手段.期刊自引率是识别影响因子人为操纵的敏感指标,但对小集团内部期刊互引和其他形式人为操纵的识别却无能为力.在该研究中,深度挖掘了自引率、扩散因子、被引半衰期、开放因子和互引指数在期刊影响因子人为操纵识别中的应用.