The effectiveness of peer review in identifying issues leading to retractions

Retractions are necessary to remove flawed research from citable literature but cannot offset the negative impact those publications have on science advances and public trust. The editorial peer-review process is intended to prevent flawed research from being published. However, there is limited empirical evidence of its effectiveness in identifying issues that lead to retractions. This study analyzed the peer-review comments (provided by Clarivate Analytics) for a sample of retracted publications (provided by Retraction Watch) to investigate how the peer-review process effectively detects the areas where the retraction causes lie and whether reviewer characteristics are related to the effectiveness. We found that a small proportion of peer reviews suggested rejections during the peer review stage, while about half suggested acceptance or minor revision for those later retracted papers. The peer-review process was more effective in identifying retraction causes related to data, methods, and results than those related to text plagiarism and references. Additionally, factors such as reviewer seniority and the level of match between reviewers’ expertise and the submission were significant in determining the possibility of peer reviews identifying suspicious areas in submissions. We discussed potential insights from these findings and called for collective efforts to prevent retractions.     

注册式研究报告的同行评议机制研究

[目的/意义]近年来频发的"学术丑闻"对我国的科研评议机制提出新的挑战。而在开放科学运动中兴起的注册式研究报告因其独特的同行评议机制,能有效地提高研究过程、评估环节的透明度,减少审稿过程中的出版偏见,确保学术严谨和科研质量,最大程度地减少学术造假行为。对注册式研究报告的同行评议机制现状和特点进行分析,以期为我国同行评议的创新发展和科学完善提供参考。[方法/过程]综合运用网络调研法和内容分析法,从注册式研究报告同行评议机制的评议流程、评议形式、评议效率、评议道德指南、同行评议专家库建设等方面进行分析,探讨注册式研究报告同行评议机制中作者、评议专家、编辑三者之间关系以及相关权利,总结注册式研究报告的同行评议机制的特征。[结果/结论]注册式研究报告同行评议机制创新性特点主要表现在：①审稿流程与标准的优化：注册式研究报告实行两次同行评议的新模式,不再仅以专家主观判断为标准,同时评议专家选择和专家意见处理等流程科学合理;②审稿匿名性和交互性的改进：同行评议的形式多样化,在保持基本的制衡关系中追求最大的灵活性;③审稿效率的提升：并行式的评议信息传递方式、明文化规定和系统化监惩机制促成高效率的同行评议。注册式研究报告同行评议机制的先进性特点主要表现在：①建立严格且细致的评议专家道德规范体系;②重视同行评议专家库的建设,形成完善的评议专家激励机制。     

Current and Evolving Models of Peer Review

New models of scientific publishing and new ways of practicing peer review have injected a recent dynamism into the scholarly communication system. In this article, we delineate the context of the traditional peer-review model, reflect on some of the first experiences with open peer review, and forecast some of the challenges that new models for peer review will have to meet. Our findings suggest that the peer-review function has the potential to be divorced from the journal system, so that the responsibility to judge the significance of a paper may no longer fall exclusively to formal reviewers, but may be assessed by the whole readership community.     

典型学术丑闻折射下同行评审的关键问题及其应对策略

基于发生在权威期刊的几桩典型学术丑闻,探讨同行评审如何系统性地应对学术论文不端行为。回顾性提炼丑闻事件的重要节点,从速度、数量、质量和评价标准4个维度来综合分析同行评审存在的关键问题;提出应在战略和技术层面上重新构建学术出版中的同行评审体系:明确同行评审中的各方责任,利用网络信息技术收集并开放数据。一个明确、健全和开放的同行评审体系有助于系统性地防范学术不端,提高同行评审质量。     

The evaluation of peer-review quality

This paper discusses the role of peer review in the decision-making processes of scientific journals. It outlines the objectives of peer review and some of the problems. It then goes on to describe research into the quality of peer review, in particular the BMJ's programme of research to date, the results obtained, and consequent changes in practice. It also looks briefly at future possibilities for research into the peer-review process.     

Interactive journal concept for improved scientific publishing and quality assurance

Many scientific publications are careless, useless or false, and inhibit scholarly communication and scientific progress. This is caused by the failure of traditional journal publishing and peer review to provide efficient scientific exchange and quality assurance in today's highly diverse world of science. The most promising way to improve matters is a two‐stage (or multi‐stage) publication processes with interactive peer review and public discussion in new and traditional scientific journals. A concept for such interactive scientific journals is outlined, and its applicability is demonstrated by the open access journal Atmospheric Chemistry and Physics.     

国外开放获取期刊的同行评议方式研究

通过网络调研和文献梳理的方式,结合具体案例,如Elsevier、Nature、PLoS、F1000 Research等,对国外开放获取期刊的同行评议方式进行研究.认为结构化同行评议、发表后开放式同行评议以及第三方独立同行评议,各有利弊,应该取长补短,优化评议方式,更好地发挥科技期刊作为学术质量把关者和过滤器的作用.     

科技期刊审稿专家的职责及实现保证——以《中国机械工程》为例

结合《中国机械工程》工作实践,阐述审稿专家负有判断内容是否达标、保障审稿流程快捷运行及协助编辑部做好相关工作的职责。介绍保障审稿专家队伍长期、健康地履行其职责的经验,如使审稿专家正确认识、理解其职责成为高素质的审稿人,提供友好的审稿平台,开展专家审稿培训,尊重专家的劳动,开展多种形式的沟通。     

Data exchange standards to support and acknowledge peer‐review activity

• Peer review is used to evaluate research, including publications, scientific awards, and grant proposals, and there is a continuum of at least six approaches to review from completely closed, double‐blind review to fully‐open and citable peer review.
• It is getting harder to find suitable experts to serve as reviewers so publishers and others are experimenting with methods to incentivize researcher participation, with a growing interest in enabling citation of peer‐review activity as a component.
• A Working Group on Peer Review Service, facilitated by CASRAI, was created to develop a data model and citation standard for peer‐review activity that can be used to support both existing and new review models.
• Standardized citation structures for reviews can enable the inclusion of peer‐review activity in personal recognition and evaluation, as well the ability to refer to reviews as part of the scholarly literature.

     

客观评价审稿贡献 消除同行评议瓶颈

为了应对专业期刊审稿工作的困境,对国外平台审稿工作的奖励方法进行分析,旨在使审稿人的审稿工作得到经济或学术的认可.认为应建立合理的度量审稿工作的指标,寻找有效的方法来激励审稿人积极参与期刊的审稿工作,以调动审稿人审稿的积极性,从而促进期刊学术水平的提高.     

数据密集、语义、可视化与互动出版:全球科技出版发展趋势研究

探讨全球科技出版领域呈现的数据密集型出版、语义出版、可视化出版与互动出版等趋势.指出数据密集型出版从宏观层面构建海量数据的框架蓝图,为科学范式的转变提供基础环境;语义出版通过对各种文献进行标引关联,从微观层面铺设结构化数据的基础设施,解决数据和信息的机器理解问题;可视化出版作为重要的直观表达工具,从文献组成部分、文献架构、文献网络三个层面最大限度地提升科研人员获取信息的效率和效果;互动出版目前以学术维基出版物和全新的同行评议为主要特色,既是“无形学院”在网络 环境下的延伸,也渗透于正式科学交流中,从读者和用户层面打破科学信息开放、共享、交流的主要障碍.     

学术期刊专家审稿进度控制的优化方法研究

同行评议时间控制应贯穿学术期刊专家审稿前、中、后全过程,而“审稿中”的时间控制是同行评议全程管理中易忽视的薄弱环节。对于该阶段的进度控制,编辑部可通过扩充专家邀请数量、提前设置增审、压缩增审周期来影响审稿进度。本文将围绕这3种方法调研其实施现状,以反映审稿进度控制现状并发现问题。结果显示,大多数期刊首次邀请专家人数仅等于要求回收意见的份数,审稿异常稿件需消耗1个审稿周期后才会被设置增审,审稿周期较长期刊的增审周期也未适当缩短,这些反映编辑部对处在“审稿中”稿件的时间管理处于近乎“虚空”的低干预状态。本文基于调研结果提出,期刊首次邀请专家人数宜大于要求回收意见份数但不是越多越好;分类确定提前增审对象及时间点;增审周期可控制在≤21 d;挖掘采编系统智能化辅助功能,减轻人力负担。多措并举,优化审稿进度控制效果,提升审稿运行效率,助力我国世界一流学术期刊建设。     

学术论文双盲评审的优势与可行性

就双盲评审与其他2种方式作对比,阐述其优势和可行性.单盲评审的不平衡,容易导致审稿人责任的缺失以及个人偏见导致的不公正判断.开放评审相对公开透明,被国外一些期刊所尝试,但其还不成熟,不适应国内当前的实际情况,不能被广泛推广.双盲评审能弥补单盲的缺陷,保证审稿更加客观公正,受到多数研究人员的认可,并且有利于提高审稿质量,提高期刊的被引用率.在采用双盲评审方式时,编辑人员应发挥良好的桥梁作用.     

The peer-review process

The recent literature about peer review of scholarly articles is reviewed, with particular emphasis on the cost of the peer-review process. Possible impacts of electronic scholarly publishing upon peer reviewing are discussed. Opinion among academics in their roles as authors, editors and referees seems likely to insist upon preservation of a pre-publication refereeing system in most disciplines. As the administration of any such system seems to have a cost of about $400 per published article, any scholarly publishing system will need to locate financial support to at least that extent, and a system of lump-sum payment by the authors' funders is best placed to cover this cost while providing universal free access to scholarly material.     

医学编辑中具有代表性的2种审稿方式

以美国<神经外科杂志>和<美国外科医师学会杂志>为例,分析顺序审稿制与同时审稿制的不同特点.重点介绍医学期刊中非常独特的顺序审稿制.     

How good is a model based on bibliometric indicators in predicting the final decisions made by peers?

This paper shows how bibliometric models can be used to assist peers in selecting candidates for academic openings.Several studies have demonstrated that a relationship exists between results from peer-review evaluations and results obtained with certain bibliometric indicators. However, very little has been done to analyse the predictive power of models based on bibliometric indicators. Indicators with high predictive power will be seen as good instruments to support peer evaluations. The goal of this study is to assess the predictive power of a model based on bibliometric indicators for the results of academic openings at the level of Associado and Catedrático at Portuguese universities. Our results suggest that the model can predict the results of peer-review at this level with a reasonable degree of accuracy. This predictive power is better when only the scientific performance is assessed by peers.     

Peer Review and Scientific Publishing in Times of Web 2.0

The second half of the twentieth century brought major changes in the society and consequently in the different areas of the sciences. The growing number of universities after the second world war, the post-war industrial growth and later digitalization transformed the sciences present until then, enlarged the scientific community and the number of scientific publications. In the last couple of decades, the creation of web 2.0 brought new possibilities for knowledge co-production, interaction and exchange between all interested parties in research. The goal of this paper is to explore the possibilities for (extended) peer review and quality control on the internet, primarily blogs and social media, which could contribute to the standard peer review process and open the sciences to a wider audience. We are wondering if these processes could also raise the quality in science and lead to the democratization of knowledge production. We argue that changes in science also have an impact in reshaping the society and bringing democratization in knowledge production.     

编委送审制在国内学术期刊中的应用

研究编委送审制的意义及流程,阐述编委送审制在创办新刊、老刊转变审理流程、创办专刊及编委推荐中的实践模式,同时给出了应用中的建议。编委送审制在期刊中的应用,保证了期刊的学术质量,提高了期刊的学术水平,有助于期刊影响力及影响因子的提高。     

论科学论文学术价值评审中的有限性

论述了科学论文学术价值评审的有限性与可能性。指出：审者评审论文学术价值，不可避免地要受到诸多因素的限制，其审稿意见必然具有局限性。因此，审者对论文学术价值的评审，只具有可能性，即认为该论文可能具有怎样的价值，而不是认为该论文肯定具有怎样的价值。认识到审稿中的有限性与可能性，对寻求科学合理的审稿方法，具有重要意义。