Using the Web for research evaluation: The Integrated Online Impact indicator

Previous research has shown that citation data from different types of Web sources can potentially be used for research evaluation. Here we introduce a new combined Integrated Online Impact (IOI) indicator. For a case study, we selected research articles published in the Journal of the American Society for Information Science & Technology (JASIST) and Scientometrics in 2003. We compared the citation counts from Web of Science (WoS) and Scopus with five online sources of citation data including Google Scholar, Google Books, Google Blogs, PowerPoint presentations and course reading lists. The mean and median IOI was nearly twice as high as both WoS and Scopus, confirming that online citations are sufficiently numerous to be useful for the impact assessment of research. We also found significant correlations between conventional and online impact indicators, confirming that both assess something similar in scholarly communication. Further analysis showed that the overall percentage for unique Google Scholar citations outside the WoS were 73% and 60% for the articles published in JASIST and Scientometrics, respectively. An important conclusion is that in subject areas where wider types of intellectual impact indicators outside the WoS and Scopus databases are needed for research evaluation, IOI can be used to help monitor research performance.     

数字图书馆可用性评价方法研究

本文介绍了数字图书馆可用性的含义和数字图书馆可用性评价的内容,从可用性检查、可用性测试、可用性调查三个方面分析了数字图书馆可用性评价的主要方法。     

Using evaluation research methods to improve quality

相似文献   

论生物样本库研究类论文伦理问题的编辑审查

参照GB 3100～ 3102-1993《量和单位》和关于计算机流程图的相关规范,分析科技期刊中计算机流程图表达中存在的问题,且给出规范的表达形式.认为:流程图中,开始与结束标志符号用椭圆框表示,数据输入和输出标志符号用平行四边形框表示,判定标志符号用菱形框表示,活动标志符号用矩形框表示;判定的2种选择可用"是"或"否","Y"或"N","Yes"或"No"表示,建议全期期刊表达一致,首选"Y"或"N"表示;图形中的文字、变量符号、体例、格式可参考GB 3100～3102-1993等相关标准进行规范.     

The status of evaluation research in communication training programs

The purpose of this study was to determine the frequency and types of evaluation research methods used in communication training programs. A random sample of internal or in‐house human resource (HR) trainers were surveyed and asked to respond to a series of questions concerning evaluation practices used in planning, implementing, and assessing communication training programs. The findings suggest that: (1) HR trainers, for the most part, do not conduct needs analyses while planning communication training programs; (2) the majority of HR trainers collect data on trainees’ attitudes toward communication training programs; (3) the majority of HR trainers do not measure changes in cognitive, affective and psychomotor behaviors or job performance that may occur as a result of participation in communication training programs; (4) experimental and quasi‐experimental designs are used by only a very small percentage of HR trainers; and (5) cost benefit analyses are seldom used to assess the economic effects of communication training programs. The findings of this study suggest that although communication training programs have significantly increased in the past decade, the use of systematic and controlled evaluation research is presently not being undertaken to plan, implement, and assess communication training programs.     

Publication counting methods for a national research evaluation exercise

In this paper, we investigate the effects of using four methods of publication counting (complete, whole, fractional, square root fractional) and limiting the number of publications (at researcher and institution levels) on the results of a national research evaluation exercise across fields using Polish data. We use bibliographic information on 0.58 million publications from the 2013–2016 period. Our analysis reveals that the largest effects are in those fields within which a variety publication and cooperation patterns can be observed (e.g. in Physical sciences or History and archeology). We argue that selecting the publication counting method for national evaluation purposes needs to take into account the current situation in the given country in terms of the excellence of research outcomes, level of internal, external and international collaboration, and publication patterns in the various fields of sciences. Our findings show that the social sciences and humanities are not significantly influenced by the different publication counting methods and limiting the number of publications included in the evaluation, as publication patterns in these fields are quite different from those observed in the so-called hard sciences. When discussing the goals of any national research evaluation system, we should be aware that the ways of achieving these goals are closely related to the publication counting method, which can serve as incentives for certain publication practices.     

从学术研究的共性看学术期刊的编辑初审

做好稿件的初审工作是办好学术期刊的关键.编辑如何才能把好学术论文的质量初审关,一般是从2个方面来讨论的:一是认为要提高编辑素质,主张编辑学者化;二是从方法的角度出发,阐述初审时可以采用的一些方法.这里提出初审的另类视角,即从学术研究的共性出发来探讨编辑把好稿件初审关的问题.     

Rooted citation graphs density metrics for research papers influence evaluation

Traditionally, citation count has served as the main evaluation measure for a paper's importance and influence. In turn, many evaluations of authors, institutions and journals are based on aggregations upon papers (e.g. h-index). In this work, we explore measures defined on the citation graph that offer a more intuitive insight into the impact of a paper than the superficial count of citations. Our main argument is focused on the identification of influence as an expression of the citation density in the subgraph of citations built for each paper. We propose two measures that capitalize on the notion of density providing researchers alternative evaluations of their work. While the general idea of impact for a paper can be viewed as how many researchers have shown interest to a piece of work, the proposed measures are based on the hypothesis that a piece of work may have influenced some papers even if they do not contain references to that piece of work. The proposed measures are also extended to researchers and journals.     

国内外电子资源使用与绩效评估研究述评

文章系统地介绍了自20世纪90年代以来,国内外开展的电子资源及其服务绩效评估研究项目,制定的相关指南、标准。公开发表的相关文献,并对国内外研究现状进行了分析、比较。     

基于受众评价的广告社会效果研究

本研究从受众的角度,基于6个城镇的1736份问卷调查,对广告的社会效果及其影响因素做了比较全面的考察.研究结果表明,从整体来看,受众对广告的社会效果的评价不高,人们对广告产生的社会效果持不太满意的态度.不同居住地、年龄、受教育程度和收入水平的受众对广告社会效果的评价上有较显著差异.在对基于受众评价的广告可信度考察中,发现广告传播存在严重的信任危机.并在讨论中,推导出广告信任的评价与研究,乃是平衡广告商业效益与社会效益的中间值.     

Recent research on the prognosis and evaluation of teaching effectiveness

A discussion of ethical proof and the preacher—from five points of view.     

电子资源、服务的测度和评估研究

随着电子出版、网络出版和图书馆数字化的发展，电子资源逐渐成为图书馆馆藏资源的重要组成部分。电子资源和服务的科学统计和计量将成为图书馆馆藏资源建设不可或缺的组成部分。国外先进的计量指标和标准、研究方法是值得我们借鉴的。确立科学的评估指标和采用综合的统计方法是电子资源评估的关键所在。     

What makes or breaks competitive research proposals? A mixed-methods analysis of research grant evaluation reports

The evaluation of grant proposals is an essential aspect of competitive research funding. Funding bodies and agencies rely in many instances on external peer reviewers for grant assessment. Most of the research available is about quantitative aspects of this assessment, and there is little evidence from qualitative studies. We used a combination of machine learning and qualitative analysis methods to analyse the reviewers' comments in evaluation reports from 3667 grant applications to the Initial Training Networks (ITN) of the Marie Curie Actions under the Seventh Framework Programme (FP7). Our results show that the reviewers' comments for each evaluation criterion were aligned with the Action's prespecified criteria and that the evaluation outcome was more influenced by the proposals’ weaknesses than by their strengths.