评价情报检索系统性能的新方法--加权中值法

查全率和查准率被大多数情报检索系统作为评价系统性能的方法,但人们已逐渐认识到它们的局限性.本文通过对三组情报检索系统用户相关性判断分布规律实验的数据分析,使用了一种基于用户相关性判断分布规律的方法--加权中值法,对情报检索系统的性能进行评价,并通过倾斜度的计算分析用户相关性判断的态度.     

网络环境下情报检索系统性能评价研究

分析网络环境下情报检索系统的特点,据此提出评价网络情报检索系统性能的指标体系,最后提出改善网络环境下情报检索系统性能的建议。     

人类信息行为研究的几个主要问题

阐述人类信息行为研究的主要问题,包括用户认知、用户信息行为及其与系统交互的研究现状.首先从情报检索的两大研究范式出发,分析情报检索认知范式的特点,进而介绍认知情报检索的研究,认为这一领域对丰富情报学的基础理论和深入认识用户具有重要的基础作用.其次,介绍国外用户信息行为研究取得的进展,包括美国学者等建立的一系列行为模型所包含的要素,即目标与任务、情境、情感和可信度等.最后讨论相关性和情报检索评价中的非效率效果因素.     

情报检索系统用户相关性判断的层次分析模型

利用层次分析法对情报检索系统用户相关性进行判断,从而可对检索结果进行定量取舍,也可用于对不同的检索系统进行比较和评价     

人类信息行为研究进展

本文评述了人类信息行为研究中的几个主要问题的研究状况,包括对用户的认知、行为和与系统交互的研究之现状。从情报检索的两大研究范式出发,首先讨论了情报检索认知范式的特点,进而介绍认知情报检索的研究。这一领域对丰富情报学的基础理论和深入认识用户起到了重要的基础作用。用户行为研究以美国的学者为代表,建立了一系列的行为模型,为了解用户提供了更为深厚的理论和实证基础。目标与任务、情境、情感和可信度等则是这些模型之中的关键要素。本文最后讨论相关性和情报检索评价中的非效率效果因素。本文是对信息行为研究的梳理,也是对情报检索的前沿问题探讨的补充。     

WWW�������������о��ͷ�չ�Բߵ�̽��

本文运用情报检索语言的基本原理和基本方法,在网上数据调查和检索测试实验的基础上,分析目前网络搜索引擎的性能现状和存在的问题,并从情报检索语言理论和新技术的引入和应用等方面探讨改进网络搜索引擎性能的发展对策。 首先,分析搜索引擎研究的现状,并从搜索引擎性能研究的角度出发,界定搜索引擎概念的范畴和类型划分,将搜索引擎分为分类目录式搜索引擎、机器人式搜索引擎和元搜索引擎3种类型。在此基础上,从结构层面、技术层面、检索层面3个角度出发,详细阐述了搜索引擎的工作原理。 其次,在借鉴传统情报检索系统的评价标准和分析搜索引擎性能因素的基础上,从数据库规模和特点、信息资源组织特点、检索功能、检索效果和用户服务特点这5个方面出发,建立了一个搜索引擎评估指标体系;并以此评估体系为评价标准,选     

现代情报检索模型理论比较与发展研究

关于相关性的计量一直是情报检索的核心问题，为此，人们提出了一系列检索模型。本文从比较与发展的角度，按时间顺序介绍了布尔模型、向量空间模型、概率模型、模糊模型、逻辑模型、概念模型、网络模型等，并在分析、比较与评价的基础上，对情报检索模型研究发展的未来趋势做了一些预测。     

情报检索系统评价研究:回顾与展望

总结情报检索系统评价研究的历史和现状,分析目前存在的问题,展望评价研究的发 展趋势。     

基于检索相关性转移的本体论检索系统

在讨论当前情报检索系统的相关性判断基础上,指出在相关性判断的多维度多层次的复杂概念中用户关心的是语用相关,但多数检索系统仅能提供形式相关.建议运用本体论的基本原理,建立本体论的情报检索系统,从而实现检索的语用相关判断.     

情报检索发展的几个前沿问题

本文介绍了现阶段情报检索研究中的几个前沿问题,包括自然语言检索、跨语言检索、智能信息检索、基于语义的图像和视频检索及检索系统评价研究的最新发展情况。语言、智能和语义等问题相互关联,近年来又一同推动着信息检索的发展。检索评价研究也有针对性地引导着情报检索的发展方向。结语部分论述了这些问题在情报检索前沿发展中的本质联系及未来的发展方向。     

Predicting relevance based on assessor disagreement: analysis and practical applications for search evaluation

Evaluation of search engines relies on assessments of search results for selected test queries, from which we would ideally like to draw conclusions in terms of relevance of the results for general (e.g., future, unknown) users. In practice however, most evaluation scenarios only allow us to conclusively determine the relevance towards the particular assessor that provided the judgments. A factor that cannot be ignored when extending conclusions made from assessors towards users, is the possible disagreement on relevance, assuming that a single gold truth label does not exist. This paper presents and analyzes the predicted relevance model (PRM), which allows predicting a particular result’s relevance for a random user, based on an observed assessment and knowledge on the average disagreement between assessors. With the PRM, existing evaluation metrics designed to measure binary assessor relevance, can be transformed into more robust and effectively graded measures that evaluate relevance towards a random user. It also leads to a principled way of quantifying multiple graded or categorical relevance levels for use as gains in established graded relevance measures, such as normalized discounted cumulative gain, which nowadays often use heuristic and data-independent gain values. Given a set of test topics with graded relevance judgments, the PRM allows evaluating systems on different scenarios, such as their capability of retrieving top results, or how well they are able to filter out non-relevant ones. Its use in actual evaluation scenarios is illustrated on several information retrieval test collections.     

A Task-Oriented Non-Interactive Evaluation Methodology for Information Retrieval Systems

Past research has identified many different types of relevance in information retrieval (IR). So far, however, most evaluation of IR systems has been through batch experiments conducted with test collections containing only expert, topical relevance judgements. Recently, there has been some movement away from this traditional approach towards interactive, more user-centred methods of evaluation. However, these are expensive for evaluators in terms both of time and of resources. This paper describes a new evaluation methodology, using a task-oriented test collection, which combines the advantages of traditional non-interactive testing with a more user-centred emphasis. The main features of a task-oriented test collection are the adoption of the task, rather than the query, as the primary unit of evaluation and the naturalistic character of the relevance judgements.     

科技项目及其成果文献的相关性评估研究

科技项目后评估对项目的完成质量与效益进行评判,是科技管理全流程中的核心环节。现有科技项目后评估研究更多的是关注如何构建评估体系和量化评价指标,鲜有基于内容来探讨项目与成果的相关性。本文使用科技报告作为科技项目的补充信息,构建基于BERT (bidirectional encoder representations from transformers)架构的相似度计算模型,通过文本语义匹配程度对科技项目-成果文献的相关性评估进行探索。研究结果表明,构建的融合模型可以有效地评估科技项目-成果文献的相关性。此外,本文对项目-成果相关性较低的情况以及科技报告监管过程中存在的问题进行了分析,旨在建设优质科技资源,完善科技项目后评估体系,综合提升科技情报服务效率。     

中国近代公共图书馆运动起讫年研究

文章通过对中国近代公共图书馆运动研究现状分析,认为目前这一方向的研究十分薄弱。从已有研究成果中反映出对这一问题研究的代表是程焕文教授。并指出这一研究领域存在着许多空白,文中对公共图书馆运动的起讫年进行探讨,认为分别是1906年和1917年。参考文献31。     

Low-cost evaluation techniques for information retrieval systems: A review

For a system-based information retrieval evaluation, test collection model still remains as a costly task. Producing relevance judgments is an expensive, time consuming task which has to be performed by human assessors. It is not viable to assess the relevancy of every single document in a corpus against each topic for a large collection. In an experimental-based environment, partial judgment on the basis of a pooling method is created to substitute a complete assessment of documents for relevancy. Due to the increasing number of documents, topics, and retrieval systems, the need to perform low-cost evaluations while obtaining reliable results is essential. Researchers are seeking techniques to reduce the costs of experimental IR evaluation process by the means of reducing the number of relevance judgments to be performed or even eliminating them while still obtaining reliable results. In this paper, various state-of-the-art approaches in performing low-cost retrieval evaluation are discussed under each of the following categories; selecting the best sets of documents to be judged; calculating evaluation measures, both, robust to incomplete judgments; statistical inference of evaluation metrics; inference of judgments on relevance, query selection; techniques to test the reliability of the evaluation and reusability of the constructed collections; and other alternative methods to pooling. This paper is intended to link the reader to the corpus of ‘must read’ papers in the area of low-cost evaluation of IR systems.     

Evaluating the effectiveness of relevance feedback based on a user simulation model: effects of a user scenario on cumulated gain value

We propose a method for performing evaluation of relevance feedback based on simulating real users. The user simulation applies a model defining the user’s relevance threshold to accept individual documents as feedback in a graded relevance environment; user’s patience to browse the initial list of retrieved documents; and his/her effort in providing the feedback. We evaluate the result by using cumulated gain-based evaluation together with freezing all documents seen by the user in order to simulate the point of view of a user who is browsing the documents during the retrieval process. We demonstrate the method by performing a simulation in the laboratory setting and present the “branching” curve sets characteristic for the presented evaluation method. Both the average and topic-by-topic results indicate that if the freezing approach is adopted, giving feedback of mixed quality makes sense for various usage scenarios even though the modeled users prefer finding especially the most relevant documents.     

政府信息服务评价体系的可持续性研究

[目的/意义]当前,建立评价常态化机制已成为各级政府提高其信息服务水平的一种有效管理策略,但当今社会所呈现出的复杂性、多样性、发展性却不断对现有评价体系的应用效果产生冲击,这就迫切需要人们对评价体系的可持续性展开相关研究。[方法/过程]创建一种评价体系的可持续性模型（S-PPF模型）,该模型在理论层面体现了评价指标集合的差异性、评价目标的关联性以及基于过去、现在、将来划分原则的发展性,并通过提出评价连续体概念以及相关实用性原则实现了在应用层面的可操作性。[结果/结论]研究发现,S-PPF模型能同时解决评价体系的针对性、关联性和时效性等难题,是一种有效的评价体系可持续性解决方案。     

我国图书馆学研究与图书馆工作现状的脱节及思考

论文指出图书馆学研究与图书馆实践之间存在脱节的倾向,主要表现在图书馆学研究的对象与工作主体不符,图书馆学对应用研究存在错位思考,以及图书馆学教育不能满足社会的需要.论文同时对这些现象提出自己的看法和建议.     

Pooling for a Large-Scale Test Collection: An Analysis of the Search Results from the First NTCIR Workshop

We have conducted a study to: (1) verify the exhaustiveness of pooling for the purpose of constructing a large-scale test collection, and (2) examine whether a difference in the number of pool documents can affect the relative evaluation of IR systems. We carried out the experiments using search topics, their relevance assessments, and the search results that were submitted for both the pre-test and test of the first NTCIR Workshop.Our results verified the efficiency and the effectiveness of the pooling method, the exhaustiveness of the relevance assessments, and the reliability of the evaluation using the test collection based on the pooling method.     

基于主成分分析法的科技期刊评价方法

针对目前科技期刊评价方法中存在的指标之间相关性问题和指标权重选取问题,提出了基于主成分分析的科技期刊评价方法.该评价方法通过相关系数矩阵的特征向量将评价指标线性变化成彼此独立的主成分,根据主成分累计贡献值确定主成分的取用维数,由主成分方差确定权重.其优点是:可以消除由于指标间的相关性带来的偏差,降低计算维数,从而降低指标选择的难度,提高评价结果的可信度;此外,可以消除人为确定指标权重引起的弊病,使评价结果更具客观性和准确性.