Gender and the evaluation of research

This study examines if and how gender relates to research evaluation via panel assessment and journal ratings lists. Using data from UK business schools we find no evidence that the proportion of women in a submission for panel assessment affected the score received by the submitting institution. However, we do find that women on average receive lower scores according to some journal ratings lists. There are important differences in the rated quality of journals that men and women publish in across the sub-disciplines with men publishing significantly more research in the highest rated accountancy, information management and strategy journals. In addition, women who are able to utilise networks to co-author with individuals outside their institution are able to publish in higher-rated journals, although the same is not true for men; women who are attributed with “individual staff circumstances” (e.g. maternity leave or part-time working) have lower scores according to journal ratings lists.     

Retraction statements and research malpractice in economics

We draw on rational crime theory to help analyse 55 articles that have been retracted from 734 peer-reviewed journals in the field of economics. We highlight and discuss what these findings indicate regarding the nature and pattern of research malpractice in that discipline. Particular attention is given to exploring “no reason” retractions and the policy guidelines of publishers regarding retracted papers. We conclude that the frequent vagueness of retraction statements, and a reluctance to signal research malpractice, generally results in little damage to the reputation of caught, and known, offenders. Thus, a key deterrent to engaging in research malpractice is lacking. To reduce the incidence of research malpractice, we offer several recommendations for publishers and journal editors.     

基于CNKI的我国数字期刊研究状况分析

摘 要 【目的】揭示我国数字期刊研究的演进历程、研究水平和未来走向。【方法】 检索中国知网（CNKI）关于数字期刊的相关文献,运用文献计量学的方法进行统计与分析。【结果】年度发文量、作者、文献下载次数和被引频次、作者单位来源、文献来源期刊以及关键词出现频次等几个方面的数据,反映出当前我国数字期刊的研究日益深化,体现出理论与实践相结合的特点。【结论】构建合理的研究队伍,平衡出版流程研究的着力点,加强研究视角的广阔性和前瞻性才能更好地促进数字期刊的发展。     

基于CNKI的我国数字期刊研究状况分析

【目的】揭示我国数字期刊研究的演进历程、研究水平和未来走向。【方法】 检索中国知网（CNKI）关于数字期刊的相关文献,运用文献计量学的方法进行统计与分析。【结果】年度发文量、作者、文献下载次数和被引频次、作者单位来源、文献来源期刊以及关键词出现频次等几个方面的数据,反映出当前我国数字期刊的研究日益深化,体现出理论与实践相结合的特点。【结论】构建合理的研究队伍,平衡出版流程研究的着力点,加强研究视角的广阔性和前瞻性才能更好地促进数字期刊的发展。     

Peering inside research networks: Some observations on the effect of the intensity of collaboration on the variability of research quality

Although there is some general agreement that increasing levels of collaboration amongst academics produce research papers that receive more citations, and that larger numbers of citations often imply higher quality, the issue of collaboration and its effect upon research output remains a controversial area with a wide range of views of what role collaboration plays and its general implications for quality. This paper re-examines the process of collaboration within research networks. It considers the role of collaboration and its effect on quality by studying the relationship between the level of interaction within research networks (collaboration) and the extent of variability of quality within those research networks. Twenty-two scientific networks from Austria are examined. The findings of the analysis are that increasing levels of collaboration are strongly associated with lower levels of variability of quality within each network. The conclusion is therefore drawn that collaboration at the level of the research network acts upon research quality qua peer review and that this peer review effect is inherent throughout the research process.     

The research core of the knowledge management literature

A bibliometric analysis and a content analysis were conducted to explore the nature of the knowledge management literature. For the bibliometric analysis, three levels of Bradford analysis were used to examine the shape of the knowledge management literature based on 21,596 references from 2771 source publications. Each of the three analyses conformed to the typical curve of the Bradford distribution. For the content analysis, the texts of 630 knowledge management articles were analyzed to address the question of what research methodologies are used in the knowledge management literature. It was found that 27.8 percent of knowledge management-related articles in knowledge management journals used no identifiable research method. Of the remaining 455 refereed articles, 60 percent employed mainstream social sciences research methodologies. The remaining 40 percent of the articles using an identifiable methodology were characterized by the use of “provisional methods” that appeared to substitute for more formally defined or scientifically based research methodologies.     

The relation between research priorities and societal demands: The case of rice

To what extent is scientific research related to societal needs? To answer this crucial question systematically we need to contrast indicators of research priorities with indicators of societal needs. We focus on rice research and technology between 1983 and 2012. We combine quantitative methods that allow investigation of the relation between ‘revealed’ research priorities and ‘revealed’ societal demands, measured respectively by research output (publications) and national accounts of rice use and farmers’ and consumers’ rice-related needs. We employ new bibliometric data, methods and indicators to identify countries’ main rice research topics (priorities) from publications. For a panel of countries, we estimate the relation between revealed research priorities and revealed demands. We find that, across countries and time, societal demands explain a country's research trajectory to a limited extent. Some research priorities are nicely aligned to societal demands, confirming that science is partly related to societal needs. However, we find a relevant number of misalignments between the focus of rice research and revealed demands, crucially related to human consumption and nutrition. We discuss some implications for research policy.     

The determinants of research output and impact: A study of Mexican researchers

This paper uses a unique data set of Mexican researchers to explore the determinants of research output and impact. Our findings confirm a quadratic relationship between age and the number of published papers. However, publishing peaks when researchers are approximately 53 years old, 5 or 10 years later than what prior studies have shown. Overall, the results suggest that age does not have a substantial influence on research output and impact. We also find that reputation matters for the number of citations but not publications. Results also show important heterogeneity across areas of knowledge. Interpretations of other aspects, such as gender, country of PhD, and cohort effect, among others, are also discussed.     

Multi-authorship and its impact on the visibility of research from Puerto Rico

The impact of the existence and nature of multi-authorship on the visibility of research results is a relevant issue in the framework of the monitoring and evaluation of scientific performance. Multi-authorship involving researchers from different institutions is a growing trend typical of today’s social, economic and political development and an expression of the so-called “internationalization of science”. This paper analyzes how the establishment of scientific relationships and the local or international nature of such relationships affect the visibility of the research results published by the community of researchers affiliated with Puerto Rican institutions. Multi-dimensional indicators and multivariate analysis techniques, specifically Factorial Correspondence Analysis (FCA), were used to analyze and represent the visibility of the papers published in mainstream scientific journals. The results of the study show that the establishment and furtherance of local and international co-authorship favour the visibility of the papers published and consequently can be regarded to be a valid strategy in the context of the research and development effort in Puerto Rico.     

科学评价的规范体系研究

科学评价规范体系是科学评价管理与监督的基础,科学评价管理与监督需要科学评价机制、政策制度、法律法规和行业规范作为保障,只有形成了科学合理、健康完善的评价规范体系,才能实现对科学评价活动的有效管理和监督。科学评价规范体系可以分为四个层次:第一层次为机制与制度;第二层次为政策与法律法规;第三层次为行业规范;第四层次为科学评价管理与监督。但各国科学评价发展水平不一,科学评价规范程度也各不相同,因而形成各具特色的科学评价规范体系。     

Navigating the impact-innovation double hurdle: The case of a climate change research fund

This paper analyses how the funding for research grants was allocated from a specific research fund which aimed to support innovative research projects with the potential to have research impact by reducing carbon emissions. The fund received a total of 106 proposals, of which 27 were successful at obtaining financial support. Our aims were to test which factors influenced the funding decision and to discover whether or not and to what extent the fund met its intended objectives through the allocation of monies. The allocation process and its outcomes were analysed using correlation, logistical and linear regression to test our research hypotheses. Using this research funding process as a single study, we found that trying to clear the impact-innovation double hurdle in a single funding initiative ultimately compromises both goals. This paper therefore contributes to our understanding of innovation management within the context of carbon emission reduction and explains which factors influenced success in securing research monies through the funding process.     

How journal rankings can suppress interdisciplinary research: A comparison between Innovation Studies and Business & Management

This study provides quantitative evidence on how the use of journal rankings can disadvantage interdisciplinary research in research evaluations. Using publication and citation data, it compares the degree of interdisciplinarity and the research performance of a number of Innovation Studies units with that of leading Business & Management Schools (BMS) in the UK. On the basis of various mappings and metrics, this study shows that: (i) Innovation Studies units are consistently more interdisciplinary in their research than Business & Management Schools; (ii) the top journals in the Association of Business Schools’ rankings span a less diverse set of disciplines than lower-ranked journals; (iii) this results in a more favourable assessment of the performance of Business & Management Schools, which are more disciplinary-focused. This citation-based analysis challenges the journal ranking-based assessment. In short, the investigation illustrates how ostensibly ‘excellence-based’ journal rankings exhibit a systematic bias in favour of mono-disciplinary research. The paper concludes with a discussion of implications of these phenomena, in particular how the bias is likely to affect negatively the evaluation and associated financial resourcing of interdisciplinary research organisations, and may result in researchers becoming more compliant with disciplinary authority over time.     

大型科研院所的综合竞争力评价研究

大型科研院所的竞争环境在转制后发生了质的变化,如何科学评价其综合竞争力具有重要的理论意义和实践价值。本文构建了大型科研院所的竞争力评价指标体系,设计了兼顾"功能性"和"协调性"的综合性评价方法,最后运用于煤炭科学研究总院下辖某分院的综合竞争力评价,并根据评价结果对其市场竞争态势进行了分析。     

Influencing scientists’ collaboration and productivity patterns through new institutions: University research centers and scientific and technical human capital

This paper analyzes the effect of university research centers on the productivity and collaboration patterns of university faculty. University research centers are an important subject for policy analysis insofar that they have become the predominant policy response to scientific and technical demands that have not been met by extant institutions, including academic departments, private firms, and government laboratories. Specifically, these centers aim to organize researchers from across the disciplines and sectors which, collectively as a research unit, possess the scientific and technical capacity relevant to scientific and technical goals of the sponsoring agencies. In this paper, we measure the productivity and collaboration patterns of university researchers affiliated with a relatively large-scale and “mature” university research center to discern the effects, if any, of the center mechanism on individual scientists and engineers. Based on an analysis of longitudinal bibliometric data, the results from this case study demonstrate affiliation with the center to be effective at enhancing overall productivity as well as at facilitating cross-discipline, cross-sector, and inter-institutional productivity and collaborations.     

The diffusion of ideas in the academy: A quantitative illustration from economics

It has long been recognised in policy discussion that the influence of new ideas depends not just on their generation but also on the way in which they diffuse through society. However, while the spread of new ideas in industry has been much studied, there are no similar quantitative studies on diffusion within the academy. This paper addresses that gap by providing quantitative evidence on the adoption of two techniques for empirically testing hypotheses in economics. On the basis of this evidence, and contrary to a common expectation, the spread of ideas seems neither straightforward nor especially rapid in the academy.     

Towards a taxonomy of research misconduct: The case of business school research

This paper examines the growing pressures and incentives encouraging research misconduct, along with the consequences, as illustrated by the case of business school research. Drawing on a review of the literature on different theoretical approaches to analysing organizational misconduct, we develop a formal taxonomy distinguishing appropriate conduct from blatantly inappropriate misconduct but with a specific focus on the ‘grey’ areas between these extremes in the form of questionable and inappropriate behaviour. We identify various sources of research misbehaviour and different categories of those affected. The aim is to provide a clearer understanding of what research behaviour is deemed appropriate or not, which stakeholders it affects, and the pressures and incentives likely to exacerbate such misconduct. We conclude with a discussion of how the taxonomy can help shape future good research practice (thereby setting a better example to students), and offer some propositions for future research.     

基于文献计量的国内外碳排放权交易研究现状分析

基于文献计量学方法,利用VOS viewer、CiteSpace V等软件,对Web of Science(WoS)核心合集10643篇文献及中国知网(CNKI)核心期刊5576篇碳排放权交易相关文献进行计量分析与对比,探索该领域研究现状及发展趋势。结果表明,我国在该领域的研究能力不断提升；国家之间的合作研究呈现实力不均衡特征,我国研究机构与美英研究机构合作较为紧密；与国际文献相比,国内文献多从政策、市场机制、市场体系等方面展开研究,对能源消费与碳足迹追踪等角度的研究较少。最后,总结了我国现阶段碳排放权交易研究方面的短板与问题,以期为未来在该领域开展研究的学者提供指导与帮助。     

Funding model and creativity in science: Competitive versus block funding and status contingency effects

In many countries the scientific funding system is shifting from an internal block funding model toward a competitive project funding model. However, there is growing concern that the competitive project funding system favors relatively safe, conventional projects at the expense of risky, novel research. It is important to assess different funding models in order to design better funding systems for science. This paper empirically tests for differences in the novelty of funded outputs between internal block funding and competitive project funding, in the setting of Japan, where both funding models play a significant role. Combining survey data from a large sample of research projects in Japan and bibliometric information about the publications produced from these projects, we find that projects funded by competitive funds on average have higher novelty compared to those funded by internal block funds. However, such positive effects only hold for researchers with high status, such as senior and male researchers. In contrast, compared to internal block funding, competitive project funding has a negative relation to novelty for low status scientists (especially junior and female researchers). The findings suggest that the competitive project selection procedure is less receptive to novel ideas from researchers with low academic status and therefore discourages their novel research. These findings can serve as a warning about potential biases in competitive funding allocation procedures and suggest the importance of secure stable funding for allowing researchers with low status to pursue their novel ideas.     

案例研究数据科学性的评价体系——基于不同数据源案例研究样本论文的实证分析

案例研究成为国内学者探索中国情境下基于企业实践的管理理论的重要研究方法,本文基于现有案例研究方法相关理论,从数据处理的角度提出一个评价案例研究数据科学性的评价体系,以全面评估案例研究者处理数据问题的严谨性和规范性,为研究者运用不同类型的数据进行科学的案例研究提供理论依据.该体系包括由六项指标构成的数据收集策略评价要素(数据选择理由、多种数据来源、数据来源说明、数据补充清理、数据偏差说明、建立案例研究数据库)和由两项指标构成的数据分析策略评价要素(构念与数据的契合性说明、构念三角验证).以上述评价体系中的各项指标为基础设计测量题项,并以2007年—2011年已发表在国内领先管理期刊的122篇案例研究论文为样本,将基于一手数据的案例研究论文样本与基于二手数据的案例研究论文样本进行对比,分析国内案例研究者在处理数据科学性问题上的实践和不足.最后,给出提高案例研究数据科学性的研究策略,以期推动中国的管理学案例研究的发展.