How accurate are cited references in Iranian peer‐reviewed journals?

The references cited in scientific articles are as important as any other part of the paper, because of their usefulness to the scientific community and to abstracting and indexing services and citation databases. I studied inaccuracies in references and in‐text citations in sample of 97 of the 519 peer‐reviewed journals accredited by the Iranian National Commission for Journal Accreditation Policy (Ministry of Research, Science and Technology). The target journals published 2,980 articles with 74,577 cited references and 108,151 in‐text citations. The results showed 36.6% as the average percentage error rate (range 5.6% to 61.3%). The mean number of errors in cited reference and in‐text citations was 2.7 per article, and the mean number of errors per journal was 690. For the entire sample of articles, 4,369 in‐text citations did not match any source in the list of references (4%), and 8,683 cited references did not match any in‐text citation (11.6%). The stakeholders in scholarly communication, especially authors, pay insufficient attention to the accuracy of bibliographic references. Peer‐reviewed journals should encourage the use of standardized journal policies and quality‐control measures regarding peer review, data quality and accuracy.     

Should citations be counted separately from each originating section?

Articles are cited for different purposes and differentiating between reasons when counting citations may therefore give finer-grained citation count information. Although identifying and aggregating the individual reasons for each citation may be impractical, recording the number of citations that originate from different article sections might illuminate the general reasons behind a citation count (e.g., 110 citations = 10 Introduction citations + 100 Methods citations). To help investigate whether this could be a practical and universal solution, this article compares 19 million citations with DOIs from six different standard sections in 799,055 PubMed Central open access articles across 21 out of 22 fields. There are apparently non-systematic differences between fields in the most citing sections and the extent to which citations from one section overlap with citations from another, with some degree of overlap in most cases. Thus, at a science-wide level, section headings are partly unreliable indicators of citation context, even if they are more standard within individual fields. They may still be used within fields to help identify individual highly cited articles that have had one type of impact, especially methodological (Methods) or context setting (Introduction), but expert judgement is needed to validate the results.     

Do open access articles have greater citation impact?: A critical review of the literature

The last few years have seen the emergence of several open access options in scholarly communication which can broadly be grouped into two areas referred to as ‘gold’ and ‘green’ open access (OA). In this article we review the literature examining the relationship between OA status and citation counts of scholarly articles. Early studies showed a correlation between the free online availability or OA status of articles and higher citation counts, and implied causality without due consideration of potential confounding factors. More recent investigations have dissected the nature of the relationship between article OA status and citations. Three non-exclusive postulates have been proposed to account for the observed citation differences between OA and non-OA articles: an open access postulate, a selection bias postulate, and an early view postulate. The most rigorous study to date (in condensed matter physics) showed that, after controlling for the early view postulate, the remaining difference in citation counts between OA and non-OA articles is explained by the selection bias postulate. No evidence was found to support the OA postulate per se; i.e. article OA status alone has little or no effect on citations. Further studies using a similarly rigorous approach are required to determine the generality of this finding.     

同行评审意见类型识别及其在不同被引频次下的分布研究

[目的/意义]识别学术论文同行评审意见类型、分析不同被引频次下同行评审意见类型在同行评审报告中的分布情况,有助于加深对同行评议机制的认识,为评估论文学术质量、量化评审专家贡献提供新思路。[方法/过程]首先,将同行评审意见类型划分为正面评价、负面评价、要求/建议(主、次要方面)、问题/疑问、陈述六个类别,经人工标注、获取训练、测试语料后,对比分析传统机器学习模型、深度学习模型在同行评审意见类型自动识别上的效果;其次,将同行评审报告涉及的学术论文进行主题聚类,进而对被引频次进行标准化处理;最后,使用Spearman相关系数、累积分布、K-S检验、负二项回归分析不同被引频次学术论文对应的同行评审报告中同行评审意见类型的分布情况。[结果/结论]SciBert模型识别效果最佳;在基于Spearman的相关性分析中,评审报告中正面评价的分布占比与被引频次具有显著的弱正相关,负面评价的分布占比与被引频次具有显著的弱负相关;通过累计分布发现,多数情况下,当累积概率相同时,高被引分区中正面评价的分布占比大于低被引分区、负面评价的分布占比小于低被引分区,K-S检验能够检测到这种差异;在负二项回归分析中,正面评价分布占比、负面评价分布占比分别对被引频次有显著的正向影响、负向影响。研究结果表明,同行评审报告中正面评价、负面评价的分布情况与其对应论文的被引频次存在相关性,被引频次一定程度上能够反映论文的学术质量。     

Text Readability and the Impact of Academic Articles: An Empiri-cal Study Based on Library and Information Science北大核心

哪些因素会影响学术论文的被引次数是文献计量学领域的一个经典研究议题。目前的研究主要关注论文的内容特征和形式特征与被引次数之间的关系,鲜有研究从文本可读性视角切入这一议题。文本可读性影响读者对文本内容的理解和知识吸收,是一个关乎知识传播效率和研究成果认可度的重要因素。本研究在控制论文知识品质和权威性的基础上,使用文本可读性R值等五个变量研究论文的文本可读性对被引次数的影响。以中文图书情报学知名期刊发表于2016—2020年的论文为研究样本,研究发现论文的文本可读性R值、是否采用复合式标题、是否使用公式和表格对被引次数有显著影响,而是否使用图对被引次数没有显著影响。研究验证了中文情境下文本可读性对论文影响力的实质性作用,研究结果对科研人员改善自身的中文学术写作以及提高研究成果影响力具有重要参考价值。     

Article’s scientific prestige: Measuring the impact of individual articles in the web of science

We performed a citation analysis on the Web of Science publications consisting of more than 63 million articles and over a billion citations on 254 subjects from 1981 to 2020. We proposed the Article’s Scientific Prestige (ASP) metric and compared this metric to number of citations (#Cit) and journal grade in measuring the scientific impact of individual articles in the large-scale hierarchical and multi-disciplined citation network. In contrast to #Cit, ASP, that is computed based on the eigenvector centrality, considers both direct and indirect citations, and provides steady-state evaluation cross different disciplines. We found that ASP and #Cit are not aligned for most articles, with a growing mismatch amongst the less cited articles. While both metrics are reliable for evaluating the prestige of articles such as Nobel Prize winning articles, ASP tends to provide more persuasive rankings than #Cit when the articles are not highly cited. The journal grade, that is eventually determined by a few highly cited articles, is unable to properly reflect the scientific impact of individual articles. The number of references and coauthors are less relevant to scientific impact, but subjects do make a difference.     

Can Microsoft Academic assess the early citation impact of in-press articles? A multi-discipline exploratory analysis

Many journals post accepted articles online before they are formally published in an issue. Early citation impact evidence for these articles could be helpful for timely research evaluation and to identify potentially important articles that quickly attract many citations. This article investigates whether Microsoft Academic can help with this task. For over 65,000 Scopus in-press articles from 2016 and 2017 across 26 fields, Microsoft Academic found 2–5 times as many citations as Scopus, depending on year and field. From manual checks of 1122 Microsoft Academic citations not found in Scopus, Microsoft Academic’s citation indexing was faster but not much wider than Scopus for journals. It achieved this by associating citations to preprints with their subsequent in-press versions and by extracting citations from in-press articles. In some fields its coverage of scholarly digital libraries, such as arXiv.org, was also an advantage. Thus, Microsoft Academic seems to be a more comprehensive automatic source of citation counts for in-press articles than Scopus.     

Bibliography of Bioethics and Index Medicus: comparison of coverage, publication delay, and ease of recall for journal articles on bioethics

Citations selected from the bibliographies of recent texts, a specialized subject bibliography, and review articles were checked in both Cumulated Index Medicus (IM) and the Bibliography of Bioethics (BB) to compare coverage, publication delay, probable causes of indexing and retrieval failure, and the ease with which relevant citations were retrieved. The study also attempted to determine whether BB included appropriate articles from the MEDLINE database in a timely and systematic manner. While 98% of the IM citations appeared within a year of publication, 79% of the BB citations appeared two to three years after their publication dates. The average citation appeared twice as frequently in IM as in BB.     

对某课题组所发表系列论文中“假引”现象的思考

通过对某课题组所发表系列论文中“假引”现象的分析,阐述以期刊影响因子作为单篇论文学术质量评价指标的局限性及正确认识“假引”现象对单篇论文学术质量评价的意义。认为对于单篇论文应运用期刊影响因子、总被引频次并结合同行评议及论文的社会价值等进行综合评价,并应采取相应的措施减少甚至杜绝“假引”问题。     

The hyperlinking pattern of open-access journals in library and information science: A cited citing reference study

Using 17 open-access journals published without interruption between 2000 and 2004 in the field of library and information science, this study compares the pattern of cited/citing hyperlinked references of Web-based scholarly electronic articles under various citation ranges in terms of language, file format, source and top-level domain. While the patterns of cited references were manually examined by counting the live hyperlinked-cited references, the patterns of citing references were examined by using the cited by tag in Google Scholar. The analysis indicates that although language, top-level domain, and file format of citations did not differ significantly for articles under different citation ranges, sources of citation differed significantly for articles in different citation ranges. Articles with fewer citations mostly cite less-scholarly sources such as Web pages, whereas articles with a higher number of citations mostly cite scholarly sources such as journal articles, etc. The findings suggest that 8 out of 17 OA journals in LIS have significant research impact in the scholarly communication process.     

Where are citations located in the body of scientific articles? A study of the distributions of citation locations

We address issues concerning what one may learn from how citation instances are distributed in scientific articles. We visualize and analyze patterns of citation distributions in the full text of 350 articles published in the Journal of Informetrics. In particular, we visualize and analyze the distributions of citations in articles that are organized in a commonly seen four-section structure, namely, introduction, method, results, and conclusions (IMRC). We examine the locations of citations to the groundbreaking h-index paper by Hirsch in 2005 and how patterns associated with citation locations evolve over time. The results show that citations are highly concentrated in the first section of an article. The density of citations in the first section is about three times higher than that in subsequent sections. The distributions of citations to highly cited papers are even more uneven.     

Evaluating the influence of a medical journal using Google Scholar

This study shows how a journal's influence can be calculated by using citations obtained from Google Scholar and other methods even though the journal is not covered by any citation databases. Influence is measured in terms of foreign contributions, ‘equivalent’ immediacy scores of recent articles, and the calculation of citations and ‘equivalent’ impact factor. A total of 580 articles published in the Medical Journal of Malaysia (MJM) between 2004 and 2008 served as the sample. Very few foreign authors contributed to MJM (12.5%), implying its low regional acceptance as a channel for research communication. Immediacy scores for each year indicate citations were received by recently published articles. A total of 1,164 citations were received by 446 of the 580 articles and the main citing sources were journals (1,083) with reasonable h index and impact factor. Yearly impact scores ranged between 0.367 and 0.616. Higher impact factor scores were obtained by older articles.     

The Citation Cloud of a biomedical article: a free,public, web-based tool enabling citation analysis

Background:An article''s citations are useful for finding related articles that may not be readily found by keyword searches or textual similarity. Citation analysis is also important for analyzing scientific innovation and the structure of the biomedical literature. We wanted to facilitate citation analysis for the broad community by providing a user-friendly interface for accessing and analyzing citation data for biomedical articles.Case Presentation:We seeded the Citation Cloud dataset with over 465 million open access citations culled from six different sources: PubMed Central, Microsoft Academic Graph, ArnetMiner, Semantic Scholar, Open Citations, and the NIH iCite dataset. We implemented a free, public extension to PubMed that allows any user to visualize and analyze the entire citation cloud around any paper of interest A: the set of articles cited by A, those which cite A, those which are co-cited with A, and those which are bibliographically coupled to A.Conclusions:Citation Cloud greatly enables the study of citations by the scientific community, including relatively advanced analyses (co-citations and bibliographic coupling) that cannot be undertaken using other available tools. The tool can be accessed by running any PubMed query on the Anne O''Tate value-added search interface and clicking on the Citations button next to any retrieved article.     

An empirical approach based on quantile regression for estimating citation ageing

An aspect of citation behavior, which has received longstanding attention in research, is how articles’ received citations evolve as time passes since their publication (i.e., citation ageing). Citation ageing has been studied mainly by the formulation and fit of mathematical models of diverse complexity. Commonly, these models restrict the shape of citation ageing functions and explicitly take into account factors known to influence citation ageing. An alternative—and less studied—approach is to estimate citation ageing functions using data-driven strategies. However, research following the latter approach has not been consistent in taking into account those factors known to influence citation ageing. In this article, we propose a model-free approach for estimating citation ageing functions which combines quantile regression with a non-parametric specification able to capture citation inflation. The proposed strategy allows taking into account field of research effects, impact level effects, citation inflation effects and skewness in the distribution of cites effects. To test our methodology, we collected a large dataset consisting of more than five million citations to 59,707 research articles spanning 12 dissimilar fields of research and, with this data in hand, tested the proposed strategy.     

Contributing factors to long‐term citation count in marine and freshwater biology articles

Researchers have investigated factors thought to affect the total number of citations in various academic disciplines, and some general trends have emerged. However, there are still limited data for many fields, including aquatic sciences. Using papers published in 2003–2005 (n = 785), we investigated marine and freshwater biology articles to identify factors that may contribute to the probability of citation and for cumulative citation counts over 10 years. We found no relationships with probability of citation; however, we found evidence that for those that were cited at least once, cumulative citations were related to several factors. Articles cited by books received more citations than those never cited by books, which we hypothesized to be indicative of the impact an article may have in the field. We also found that articles first cited within 2 years of publication received more cumulative citations than those first cited after 2 years. We found no evidence that self‐citation (as the first citation) had a significant effect on total citations. Our findings were compared with previous studies in other disciplines, and it was found that aquatic science citation patterns are comparable to fields in science and technology but less so to humanities and social sciences.     

How research data is cited in scholarly literature: A case study of HINTS

This study investigated how Health Information National Trends Survey (HINTS) data from the U.S. National Cancer Institute (NCI) was cited in the scholarly literature. It addressed the following research questions: (1) What patterns of citations exist among authors of research articles using HINTS data? and (2) How is the citation format of HINTS data characterized? We collected scholarly articles that used HINTS data as primary or secondary data for analysis from Web of Science databases and HINTS publications on the NCI website. Among the resulting 250 articles, we identified citations to HINTS data themselves (data citations) and those to HINTS‐related documents (data‐related citations). Among the 250 articles, 156 articles (62.4%) cited HINTS data or HINTS‐related documents; only 29 articles (11.6%) cited HINTS data, while 127 (50.8%) cited HINTS‐related documents. Both data and data‐related citations increased over time. Data citation format varied, and 13 different compositions of citation elements were identified. Author, Title, and Location (URL) were common elements. The frequent use of URLs is undesirable due to URL instability. Furthermore, the data citations showed not only various compositions of citation elements but also ill‐defined element formats. Standardized citation formats are therefore needed.     

科学评价体系中同行评议与文献计量的实证分析

通过对比F1000因子与被引频次、F1000因子与期刊评价指标,并对主要指标进行相关性分析,来验证同行评议与引文分析间的相关性。结果表明,F1000因子与被引频次呈正相关性,即专家打分与被引频次变动方向相同;但也存在专家打分高的论文被引频次低,而专家打分低的论文被引频次高的事实。相关性分析的结果表明：在特征因子、SNIP等主要指标中,SJR、IF与专家评议值相关度最大。