Benchmarking: the Search for Peers. A Hybrid Library's Experience

ABSTRACT

Although there are established measurements and metricsfor academic libraries, there are few, if any, for presidential libraries, especially hybrid presidential libraries. The presidential library system, part of the National Archives and Records Administration (NARA), administers and defines metrics for 13 libraries documenting the lives and work of U.S. presidents since Herbert Hoover (1929–1933). There is no equivalent system for the non-NARA Presidential Libraries, such as the ALPLM, that documents the lives of earlier presidents. Can the non-NARA Libraries be considered a peer group for benchmarking? This column examines benchmarking and the attempts of the ALPLM to define a peer group.     

Academic sell-out: how an obsession with metrics and rankings is damaging academia

Increasingly, academics have to demonstrate that their research has academic impact. Universities normally use journal rankings and journal impact factors to assess the research impact of individual academics. More recently, citation counts for individual articles and the h-index have also been used to measure the academic impact of academics. There are, however, several serious problems with relying on journal rankings, journal impact factors and citation counts. For example, articles without any impact may be published in highly ranked journals or journals with high impact factor, whereas articles with high impact could be published in lower ranked journals or journals with low impact factor. Citation counts can also be easily gamed and manipulated, and the h-index disadvantages early career academics. This paper discusses these and several other problems and suggests alternatives such as post-publication peer review and open-access journals.     

基于UML类图的软件度量工具研究

就UML类图质量模型和软件度量方法进行研究,基于传统的CK度量集,设计并实现了基于UML类图的软件度量工具JCKMT．通过对UML类图进行度量分析,实现软件开发过程控制,为软件开发人员客观地分析、评估软件的设计、生产和使用状况提供了支持．     

MULTIPLICITY JUSTIFIES CORPORATE STRATEGY

This study conducts an analysis of the relationship between strategic theory, industry- or market-wide practices, valuation metrics, and justification rhetoric in performing strategic practice. In doing this, we refer to Michel Callon on the performativity of economics and Boltanski and Thévenot on the justification of strategic action. The paper introduces an analytical framework for studying the corporate strategy pragmatics of a forest industry company – Stora Enso – over the 1990–2008 period. The authors argue that Stora Enso's corporate strategy is justified by and represents the outcome of multiple performances that coexist and interact with each other. These multiple performances, ranging from general strategic management conceptual theories to industry-wide practices and valuation metrics, may lead to conflict when creating successful businesses.     

学科知识结构揭示模型构建

[目的/意义] 为完善学科知识结构的揭示过程,将隐含的、抽象的学科知识结构具象化,形成较为严谨的方法规范和理论范式。[方法/过程] 对揭示学科知识结构所调用的相关理论、方法、技术等进行系统分析,将"三计学"理论（文献计量学、科学计量学和信息计量学）、社会网络分析方法、学科知识结构理论和知识图谱技术进行集成优化和协同整合。[结果/结论] 构建学科知识结构揭示模型,即以"三计学"模块为基础、社会网络分析模块为支撑、学科知识结构模块为牵引、知识图谱模块为输出、四大模块交叉融合的"233-1理论模型",为实证研究提供理论支撑,也为相关研究提供方法学参考和依据。     

开放政府数据评估体系述评:特点分析

[目的/意义]随着开放政府数据运动的快速发展,开放政府数据评估体系也在研究和实践中不断出现,对这些评估体系的特点进行分析,有利于采取更科学合理的方法对开放政府数据进行准确评估,推动开放政府数据的可持续发展。[方法/过程]对研究与实践中现有的开放政府数据评估体系进行系统梳理,归纳出现有评估体系的评估动因、评估内容和评估方法及相应特点和适用性。[结果/结论]研究发现现有评估体系在城市层面、特定行业和开放数据成熟度评估方面存在局限,提出应根据不同评估动因、因地制宜地选择评估内容和评估方法的建议,并构建了开放政府数据评估动因、评估内容和评估方法的循环迭代检验机制。     

On information retrieval metrics designed for evaluation with incomplete relevance assessments

Modern information retrieval (IR) test collections have grown in size, but the available manpower for relevance assessments has more or less remained constant. Hence, how to reliably evaluate and compare IR systems using incomplete relevance data, where many documents exist that were never examined by the relevance assessors, is receiving a lot of attention. This article compares the robustness of IR metrics to incomplete relevance assessments, using four different sets of graded-relevance test collections with submitted runs—the TREC 2003 and 2004 robust track data and the NTCIR-6 Japanese and Chinese IR data from the crosslingual task. Following previous work, we artificially reduce the original relevance data to simulate IR evaluation environments with extremely incomplete relevance data. We then investigate the effect of this reduction on discriminative power, which we define as the proportion of system pairs with a statistically significant difference for a given probability of Type I Error, and on Kendall’s rank correlation, which reflects the overall resemblance of two system rankings according to two different metrics or two different relevance data sets. According to these experiments, Q′, nDCG′ and AP′ proposed by Sakai are superior to bpref proposed by Buckley and Voorhees and to Rank-Biased Precision proposed by Moffat and Zobel. We also point out some weaknesses of bpref and Rank-Biased Precision by examining their formal definitions.

Author-level metrics in the new academic profile platforms: The online behaviour of the Bibliometrics community

The new web-based academic communication platforms do not only enable researchers to better advertise their academic outputs, making them more visible than ever before, but they also provide a wide supply of metrics to help authors better understand the impact their work is making. This study has three objectives: a) to analyse the uptake of some of the most popular platforms (Google Scholar Citations, ResearcherID, ResearchGate, Mendeley and Twitter) by a specific scientific community (bibliometrics, scientometrics, informetrics, webometrics, and altmetrics); b) to compare the metrics available from each platform; and c) to determine the meaning of all these new metrics. To do this, the data available in these platforms about a sample of 811 authors (researchers in bibliometrics for whom a public profile Google Scholar Citations was found) were extracted. A total of 31 metrics were analysed. The results show that a high number of the analysed researchers only had a profile in Google Scholar Citations (159), or only in Google Scholar Citations and ResearchGate (142). Lastly, we find two kinds of metrics of online impact. First, metrics related to connectivity (followers), and second, all metrics associated to academic impact. This second group can further be divided into usage metrics (reads, views), and citation metrics. The results suggest that Google Scholar Citations is the source that provides more comprehensive citation-related data, whereas Twitter stands out in connectivity-related metrics.     

Fuzzy—区间分布数与区间拟概率度量

本文引入Fuzzy—区间分布数及其扩张加法运算,并在此基础上,研究区间拟概率及量和伪度量之间的关系,从而为进一步讨论普通拟概率度量空间提供另一有效方法。     

Unpacking ontological security: A decolonial reading of scholarly impact

Despite the growing debate about scholarly impact, an analysis of the onto-epistemic grammar underlying impact has remained absent. By taking a different analytical approach to examining impact, we interrogate the concept through the lens of decolonial thought. We offer an empathetic review of the impact scholarship and illuminate the limits of the modern imaginary that circumscribe critiques of impact in the literature, making visible the Eurocentric and provincial horizons of modern reason underlying these critiques and impact in general. Drawing on ?ūnyatā ontological perspective, we seek to articulate from modernity imaginary’s edges and suggest imagining and being otherwise. We argue that the question of scholarly impact is intimately structured by and connected to the modern subject’s desire for ontological security.