Comments on “Ranking researchers: Evidence from Indonesia” by Fry et al. (2023)

The Fry et al. article suggests that productivity can be improved by ranking researchers on a platform, but this assumption overlooks the possibility that previous resource allocation may not have been inefficient. The proposed solution simply shifts resources towards measurable metrics, such as SINTA, without a clear understanding of its long-term effects. Therefore, it is important to consider potential trade-offs and be mindful of whether such policies are beneficial in the future.The authors suggest that Fry et al. should broaden their sources and engage with stakeholders to gain a nuanced understanding of the local context and challenges confronting researchers in Indonesia. They also emphasize the importance of prioritizing research integrity over traditional metrics and rankings, aligning research funding allocation with national development priorities, and considering potential trade-offs when implementing policies such as the SINTA platform. This is the true cost of implementing rank-based and ingratiation-driven research curation system.     

The impact of national research funds: A regression discontinuity approach to the Chilean FONDECYT

This paper analyses the role of national research funds in promoting scientific production in emerging economies. The study focuses on the impact of the Chilean National Science and Technology Research Fund (FONDECYT). The analysis uses data drawn from international sources of bibliometric information combined with the administrative records of the program's executing unit. To measure the program's impact, we implement a regression discontinuity (RD) design on principal researchers who applied for funding between 1988 and 1997 considering as outcomes both quantity (publications up to 2002) and quality (citations up to 2005) of their scientific production. Our results show significant and positive impact in terms of publications, but no impact in terms of quality of scientific production in the proximity of the program's threshold ranking.     

科研人员省际流动网络分析及演化过程研究

   本文基于“科学家在线”平台数据,运用复杂网络分析方法,按照年份分别构建2010—2017年间全国34个省份之间科研人员流动网络模型,测度并分析8年间科研人员流动的复杂性网络整体特征和随时间变化的趋势。研究发现：（1）8年间科研人员流动呈现整体较为明显的“无标度”特性,即北京、上海、江苏等经济科技发达地区不仅流入科研人员的规模较大,流出科研人员的规模也不小,但是全国大部分省份流动科研人员规模较为有限;（2）除了小部分省份之外其余省份的人员流入流出相对较为均衡,一些经济科技发达省份在一定程度上也是全国或本区域科研人员流动的中心节点;（3）从发展态势来看,8年间全国省级科技人员流动态势相对平稳,但是省份之间科研人员流动规模差距越来越明显,经济科技发达省份与其他省份科研人员流动的规模相比差距越来越大;（4）最为突出的变化是流动人员规模显著增加,人员流动更加便利。     

Management system for a scientific research institute based on the assessment of scientific publications

This study depicts ihe management system of a research institute based on input-output principles. The elements of the system are the scientific departments. The main categories of the controlling system arc the planning and supervision or research activities, evaluation of scientific and economic results and rewarding of researchers. The evaluation of scientific results focuses on the analysis of publications. The quality of scientific papers is characterized by a scoring system based on impact factors of periodicals and by citations received. Possible reasons for the lack of citations are discussed. The results of mission-oriented research (MOR) are evaluated on the basis of revenues and earnings. The amount of bonus given to researchers is based on a scoring system taking publication and MOR activity into account. A marked difference in the number of publications and citations among individual departments was observed.     

New journal classification methods based on the global h-index

In this work we develop new journal classification methods based on the h-index. The introduction of the h-index for research evaluation has attracted much attention in the bibliometric study and research quality evaluation. The main purpose of using an h-index is to compare the index for different research units (e.g. researchers, journals, etc.) to differentiate their research performance. However the h-index is defined by only comparing citations counts of one’s own publications, it is doubtful that the h index alone should be used for reliable comparisons among different research units, like researchers or journals. In this paper we propose a new global h-index (Gh-index), where the publications in the core are selected in comparison with all the publications of the units to be evaluated. Furthermore, we introduce some variants of the Gh-index to address the issue of discrimination power. We show that together with the original h-index, they can be used to evaluate and classify academic journals with some distinct advantages, in particular that they can produce an automatic classification into a number of categories without arbitrary cut-off points. We then carry out an empirical study for classification of operations research and management science (OR/MS) journals using this index, and compare it with other well-known journal ranking results such as the Association of Business Schools (ABS) Journal Quality Guide and the Committee of Professors in OR (COPIOR) ranking lists.     

University licensing and the flow of scientific knowledge

As university involvement in technology transfer and entrepreneurship has increased, concerns over the patenting and licensing of scientific discoveries have grown. This paper examines the effect that the licensing of academic patents has on journal citations to academic publications covering the same scientific research. We analyze data on invention disclosures, patents, and licenses from the University of California, a leading U.S. academic patenter and licensor, between 1997 and 2007. We also develop a novel “inventor-based” maximum-likelihood matching technique to automate and generalize Murray's (2002) “patent-paper pairs” methodology. We use this methodology to identify the scientific publications associated with University of California patents and licenses.Based on a “difference-in-differences” analysis, we find that within our sample of patented academic discoveries, citations to licensed patent-linked publications are higher in the three years after the license, although this difference is not statistically significant. We then disaggregate our sample into (a) patented discoveries that are likely to be used as “research tools” by other researchers (based on the presence of material transfer agreements (MTAs) that cover them) and (b) patented discoveries not covered by MTAs. Citations to publications linked to licensed patents in the latter subset (not covered by MTAs) are higher for publications linked to licensed patents, and this difference is statistically significant. In contrast, licensing of patented discoveries that are also research tools is associated with a reduction in citations to papers linked to these research advances, raising the possibility that licensing may restrict the flow of inputs to “follow-on” scientific research.     

Organization of biomedical data for collaborative scientific research: A research information management system

Biomedical researchers often work with massive, detailed and heterogeneous datasets. These datasets raise new challenges of information organization and management for scientific interpretation, as they demand much of the researchers’ time and attention. The current study investigated the nature of the problems that researchers face when dealing with such data. Four major problems identified with existing biomedical scientific information management methods were related to data organization, data sharing, collaboration, and publications. Therefore, there is a compelling need to develop an efficient and user-friendly information management system to handle the biomedical research data. This study evaluated the implementation of an information management system, which was introduced as part of the collaborative research to increase scientific productivity in a research laboratory. Laboratory members seemed to exhibit frustration during the implementation process. However, empirical findings revealed that they gained new knowledge and completed specified tasks while working together with the new system. Hence, researchers are urged to persist and persevere when dealing with any new technology, including an information management system in a research laboratory environment.     

Research addressing emerging technological ideas has greater scientific impact

This study empirically examines the association between the extent of emerging technological ideas in a scientific publication and its future scientific impact measured by number of citations. We analyze metadata of scientific publications in three scientific domains: Nano-Enabled Drug Delivery, Synthetic Biology, and Autonomous Vehicles. By employing a bibliometric indicator for identifying and quantifying emerging technological ideas – as derived terms from the titles and abstracts – we measure the extent to which the publication contains emerging technological ideas in each domain. Then, we statistically estimate the size and statistical significance of the relationship between the publication-level technological emergence score and the normalized number of citations accruing to the publication.Our analysis shows that the degree to which a paper contains technologically emerging ideas is positively and strongly associated with its future citation impact in each of the three domains. An additional analysis demonstrates that this relationship holds for citations from other publications, both in the same field as, and in different fields from, the scientific domain of the focal publication. A series of tests for validation further support our argument that the greater the extent to which scientific knowledge (a paper) contains emerging ideas, the bigger its scientific impact. Implications for academic researchers, research policymakers, and firms are discussed.     

改革开放以来我国科研经费管理政策的变迁、评介与走向——基于政策文本的内容分析

以我国改革开放以来的42项科研经费管理政策为样本,采用内容分析法对政策文本进行了细致和定量化的研究。研究发现,改革开放以来我国科研经费管理政策体系的建设经历了摸索、建立与健全三个阶段,与我国科技战略的调整与科研经费的增长具有较高的一致性;政策的制定显现出从政出多门向各相关部门间协同的趋势;管理原则与政策价值逐步向灵活的、自主的、更富激励性的宏观监控演进,近年来承诺机制与信用管理机制引入管理政策体现了注重科研工作者诚信自律的价值走向;政策工具也逐步从粗放式向精细化过渡。本研究结论有助于科研工作者了解和掌握我国科研经费管理制度的内核与关键,并为决策者制定科学、合理的科研经费管理政策提供了经验借鉴与理论支撑。     

A new perspective to automatically rank scientific conferences using digital libraries

Citation analysis is performed in order to evaluate authors and scientific collections, such as journals and conference proceedings. Currently, two major systems exist that perform citation analysis: Science Citation Index (SCI) by the Institute for Scientific Information (ISI) and CiteSeer by the NEC Research Institute. The SCI, mostly a manual system up until recently, is based on the notion of the ISI Impact Factor, which has been used extensively for citation analysis purposes. On the other hand the CiteSeer system is an automatically built digital library using agents technology, also based on the notion of ISI Impact Factor. In this paper, we investigate new alternative notions besides the ISI impact factor, in order to provide a novel approach aiming at ranking scientific collections. Furthermore, we present a web-based system that has been built by extracting data from the Databases and Logic Programming (DBLP) website of the University of Trier. Our system, by using the new citation metrics, emerges as a useful tool for ranking scientific collections. In this respect, some first remarks are presented, e.g. on ranking conferences related to databases.     

Funding model and creativity in science: Competitive versus block funding and status contingency effects

In many countries the scientific funding system is shifting from an internal block funding model toward a competitive project funding model. However, there is growing concern that the competitive project funding system favors relatively safe, conventional projects at the expense of risky, novel research. It is important to assess different funding models in order to design better funding systems for science. This paper empirically tests for differences in the novelty of funded outputs between internal block funding and competitive project funding, in the setting of Japan, where both funding models play a significant role. Combining survey data from a large sample of research projects in Japan and bibliometric information about the publications produced from these projects, we find that projects funded by competitive funds on average have higher novelty compared to those funded by internal block funds. However, such positive effects only hold for researchers with high status, such as senior and male researchers. In contrast, compared to internal block funding, competitive project funding has a negative relation to novelty for low status scientists (especially junior and female researchers). The findings suggest that the competitive project selection procedure is less receptive to novel ideas from researchers with low academic status and therefore discourages their novel research. These findings can serve as a warning about potential biases in competitive funding allocation procedures and suggest the importance of secure stable funding for allowing researchers with low status to pursue their novel ideas.     

The impact of funding on research collaboration: Evidence from a developing country

In this paper we evaluate the impact of research grants on the amount of collaboration among scientific researchers in Argentina. We find a positive and significant impact of funding on collaboration which is measured in terms of the number of co-authors for publications in peer-reviewed journals. Our identification strategy is based on comparing collaboration indicators for researchers with financially supported projects with those of a control group of researchers who submitted projects that were accepted in terms of quality, but not supported because of shortage of funds. We obtain consistent results by using different non-experimental techniques including difference-in-differences models combined with propensity score matching algorithms.     

科学论文语义标注框架的设计与应用

[目的/意义]文章的研究目的是探索科学论文的语义标注模式以满足科研人员快速获取细粒度科学知识的需求。[方法/过程]结合已有的概念模型,提出了一个聚焦于描述科学论文论证过程的语义标注框架,并利用该框架对一篇认知心理学科学论文进行了语义标注实验以验证其有效性。[结果/结论]以上述科学论文为例,利用该框架对其物理结构、论证过程、科学结论和实验过程进行了语义标注,以此构建细粒度的语义出版物,并在此基础上实现了对科学论文的细粒度内容的检索。[局限]如何对科学论文中的图片、表格和数据集进行语义标注,需要进一步研究。     

Science or graduates: How do firms benefit from the proximity of universities?

This paper examines the impact of universities on the technological performance of adjacent firms. We extend existing research by jointly analyzing, and comparing, the effects of education (graduates) and scientific research (publications) activities of universities on firms’ technological performance. Adopting the knowledge production framework, our study is conducted at the level of 101 Italian territorial areas (provinces) and four industries. Overall, fixed-effect panel data models reveal a positive effect of both university graduates and scientific publications on the technological performance of firms. At the same time, considerable industry differences are observed. While the provision of university graduates positively affects firm performance in all industries under study, additional effects for scientific research are only observed in electrical and pharmaceutical industries that are science-intensive and where the scientific knowledge base is changing rapidly over time. The observation that spillovers from academia into the industrial texture of provinces rely on education and research in an industry-specific manner is relevant to the design of appropriate research and innovation policies.     

Industry funding and university professors’ research performance

University research is to an increasing extent funded by industry, and the share of basic funding is decreasing. In the literature, there are optimistic and pessimistic views on the implication of this development. Based on data from a questionnaire study among all tenured university professors in Norway (N = 1967) we find that there is a significant relationship between industry funding and research performance: professors with industrial funding describe their research as applied to a greater extent, they collaborate more with other researchers both in academia and in industry, and they report more scientific publications as well as more frequent entrepreneurial results. There is neither a positive nor negative relationship between academic publishing and entrepreneurial outputs.     

高级研究人员自我激励因素研究

高级研究人员是从事知识生产、创造、扩展和应用,以科研为主要劳动并具备一定管理才能的知识型员工.他们广泛工作在科研院所、高等院校、科技型企业等,因其工作性质而具备高独立自主性、自我价值感强、渴望成就与能力提升等个体特征.通过对116名不同高校、不同学科领域的长江学者特聘教授和讲座教授进行背景分析和学术成果调查,剖析高级研究人员的个体特征,提出成就与挑战、个体成长、工作自治、认可和目标导向等五个高级研究人员自我激励因素,并构建了基于自我激励因素的个体激励模型.     

2004年度诺贝尔生理或医学奖获得者科学论文测度及思考

本文利用国际权威数据库，根据科学计量学方法，从发表论文数量 统计和引文分析两个方面，通过对2004年诺贝尔生理或医学奖获得者科技论文进行测度，对 顶级医 学科学家发表论文数量与质量、医学科学活动规律、项目研究过程和相互合作等进行了辩证 的分析与思考，以为我国杰出医学科学家的培养和早出诺贝尔奖级研究成果提供参考。     

Why do firms publish? A systematic literature review and a conceptual framework

In this paper, we address the question of ‘Why do firms publish scientific papers?’. Research examining the competitive advantages that firms accrue from investment in R&D has provided evidence that such efforts can be associated with the voluntary disclosure of research findings in scientific publications. This form of scientific openness occurs despite potentially undermining the value-capturing process by generating knowledge spillovers and hindering the use of other instruments for protecting intellectual property (patents and secrecy). Our understanding of what leads firms to engage in scientific publishing remains relatively limited, however. We address this gap by presenting a systematic review of 164 studies examining firm publishing. We then develop a conceptual framework that outlines five incentives for firms to engage in publishing: (i) accessing external knowledge and resources; (ii) attracting and retaining researchers; (iii) supporting IP strategies; (iv) building the firm's reputation; and (v) supporting commercialization strategies. Mechanisms that relate incentives to publish to firms' major stakeholders – i.e. academia, industry, investors, users, and institutions – are also outlined in the framework. We conclude by setting out an agenda for future research.     

学术和商业激励作用下的高校R&D活动影响研究

随着传统型大学向创业型大学的转变，高校兼具知识创造和知识应用的双重特性。科研人员在R&D活动中追求学术价值和商业价值双重目标。本文基于一个简单模型来研究学术和商业激励两种方式对R&D活动时间配置以及项目选择的影响，基于此，进一步分析了组织最优激励契约设计。研究结果表明：高的商业激励会减少科研人员研究活动，但却激励科研人员选择高风险的基础研究，从而会带来突破性创新。组织层面高校应加大商业激励鼓励研究偏好人员，对偏好研究活动的人员，激励其进行开发活动；对于不偏好研究活动的人员，激励其增加努力程度。     

基于科学计量学的我国科学论文产出分析

运用科学计量学方法,统计美国科学信息所（ISI）提供的数据,对我国的科学研究成果进行定量分析和国际比较,从而来评价我国当前的科学研究现状．结果表明,我国的SCI、SSCI论文呈指数增长,高引文数量虽有增长,但增长速度远远不及论文数量增长的速度,而且中国内地高校院所入围Highlycited．CON高引研究人员名单的数目仅为3人,这从一个侧面反映了我国科学研究机制在经费投入、政策导向、人才培养方面存在着不足．