User relevance criteria choices and the information search process

Relevance judgments occur within an information search process, where time, context and situation can impact the judgments. The determination of relevance is dependent on a number of factors and variables which include the criteria used to determine relevance. The relevance judgment process and the criteria used to make those judgments are manifestations of the cognitive changes which occur during the information search process.Understanding why these relevance criteria choices are made, and how they vary over the information search process can provide important information about the dynamic relevance judgment process. This information can be used to guide the development of more adaptive information retrieval systems which respond to the cognitive changes of users during the information search process.The research data analyzed here was collected in two separate studies which examined a subject’s relevance judgment over an information search process. Statistical analysis was used to examine these results and determine if there were relationships between criteria selections, relevance judgments, and the subject’s progression through the information search process. Findings confirm and extend findings of previous studies, providing strong statistical evidence of an association between the information search process and the choices of relevance criteria by users, and identifying specific changes in the user preferences for specific criteria over the course of the information search process.     

Uncertainty in information seeking and retrieval: A study in an academic environment

This research focuses specifically on uncertainty and information seeking in a digital environment. In this research we argue that different types of uncertainty are associated with the information seeking process and that, with the proliferation of new and different search tools, sources and channels, uncertainty, positive/desirable or negative/undesirable, continues to be a significant factor in the search process. Users may feel uncertain at any stage of the information search and retrieval process and uncertainty may remain even after completion of the process resulting in what may be called persistent uncertainty. An online questionnaire was used to collect data from users in the higher education sector. There were three parts to the questionnaire focusing on: information seeking activities, information seeking problems, and access to specific information channels or sources. Quantitative analysis was carried out on the data collected through the online questionnaire. A total of 668 responses were returned from the chosen user categories of academic staff, research staff and research students. This research has shown that there are some information seeking activities and information seeking problems that are the most common causes of uncertainty among significant number of users from different disciplines, age, gender, ICT skills, etc. This is also the case with respect to access to and use of specific information sources/channels, although the degrees of uncertainty in relation are relatively small. Possible implications of this study and further research issues are indicated.     

Complex adaptive filtering user profile using graphical models

This article explores how to develop complex data driven user models that go beyond the bag of words model and topical relevance. We propose to learn from rich user specific information and to satisfy complex user criteria under the graphical modelling framework. We carried out a user study with a web based personal news filtering system, and collected extensive user information, including explicit user feedback, implicit user feedback and some contextual information. Experimental results on the data set collected demonstrate that the graphical modelling approach helps us to better understand the complex domain. The results also show that the complex data driven user modelling approach can improve the adaptive information filtering performance. We also discuss some practical issues while learning complex user models, including how to handle data noise and the missing data problem.     

Toward a reconceptualization of information seeking research: focus on the exchange of meaning

This paper reports the results of a preliminary study of interpersonal information seeking interactions between a user and a human information source. The study showed that users specify their information needs (uncertainty) largely in terms of what they know (certainty) during the interaction. The articulated certainty and uncertainty in the interaction can be classified as utterances focusing on either the topic (what the user is talking about) or comment (how that topic fits in with the user's situation or problem). We suggest that user studies in information seeking research should conceptually realign from an emphasis on user's uncertainty- and topic-based matching to the inclusion of user's certainty and comment dimensions in order to develop a more linguistically robust, multi-dimensional approach to matching for information retrieval.     

A test of genetic algorithms in relevance feedback

There have been recent applications of genetic algorithms to information retrieval, mostly with respect to relevance feedback. Nevertheless, they are yet to be evaluated in a way that allows them to be compared with each other and with other relevance feedback techniques. We here implement the different genetic algorithms that have been applied in the literature together with some of our own variations, and evaluate them using the residual collection method described by Salton in 1990 for the evaluation of relevance feedback techniques. We compare the results with those of the Ide dec-hi method, which is one of the traditional methods that yields the best results.     

Information culture and records management: a suitable match? Conceptualizations of information culture and their application on records management

Good information and records management is assumed to promote organizational efficiency. Despite established management regimes and available technology, many organizations still consider information and records management challenging. The reason may be cultural factors. This study based on a literature review, aims to explore the academic discourse on information culture and to discuss its relevance for records management. The findings show that the concept information culture is used in various ways: as an explanatory framework; as an analytical and evaluative tool; or as normative standard. The research on information culture addresses several areas: business performance, systems implementation, the manifestation of information culture in different organizations, and a few concerns records management practices. The research settings and the objects of study varied, why general conclusions are difficult to draw, but often a positive correlation between culture and performance is assumed. The focus has been on how information is used, shared and disseminated, while the production and management, that is the vital object of records management, has with few exceptions been neglected. If information culture should fully function as an analytical framework concerning records management, a widened and more inclusive conceptualization is required, which also will enrich information culture as a theoretical concept.     

A relevance feedback mechanism for cluster-based retrieval

Contextual document clustering is a novel approach which uses information theoretic measures to cluster semantically related documents bound together by an implicit set of concepts or themes of narrow specificity. It facilitates cluster-based retrieval by assessing the similarity between a query and the cluster themes’ probability distribution. In this paper, we assess a relevance feedback mechanism, based on query refinement, that modifies the query’s probability distribution using a small number of documents that have been judged relevant to the query. We demonstrate that by providing only one relevance judgment, a performance improvement of 33% was obtained.     

Weak information work in scientific discovery

Scientists continually work with information to move their research projects forward, but the activities involved in finding and using information and their impact on discovery are poorly understood. In the Information and Discovery in Neuroscience (IDN) project we investigated the information work involved as researchers make progress and confront problems in the practice of brain research. Through case studies of recent neuroscience projects, we found that the most difficult and time-consuming information activities had parallels with Simon’s explication of weak methods in scientific problem solving. But, while Simon’s weak/strong distinction is an effective device for interpreting information work, his general conception of how discovery takes place is artificially constrained. We present cross-case and case-based results from the IDN project to illustrate how the conditions of problem solving Simon associated with weak methods relate to information work and to identify additional weak aspects of the research process not considered by Simon. Our analysis both extends Simon’s framework of what constitutes the discovery process and further elaborates how weak approaches influence the conduct of research.     

Temporal modeling of group information foraging: An application to emergency response

This work seeks to uncover and explain the dynamics of information foraging within small groups. The focus of this work is collaborative information foraging within multidisciplinary emergency response teams during the response to a simulated emergency. The study investigates how such groups distribute their effort between the activities of information seeking and handling (i.e., processing) for information that is unique (i.e., initially held by one member) versus common (i.e., initially held by multiple members). Temporal analysis is applied to the data from a laboratory study of three such groups. The results suggest that temporal analysis may be used to model distribution of effort between seeking and handling, but not how this effort is divided between common versus unique information sources. Opportunities for future research along these lines are identified and briefly discussed.     

Contextual factors affecting the utility of surrogates within exploratory search

In this paper we investigate how information surrogates might be useful in exploratory search and what information it is useful for a surrogate to contain. By comparing assessments based on artificially created information surrogates, we investigate the effect of the source of information, the quality of an information source and the date of information upon the assessment process. We also investigate how varying levels of topical knowledge, assessor confidence and prior expectation affect the assessment of information surrogates. We show that both types of contextual information affect how the information surrogates are judged and what actions are performed as a result of the surrogates.     

信息检索的相关性

在传统的信息检索中，我们常用检全率、检准率来评价信息检索系统性能和检索效果.随着信息总量的急剧增加，信息载体形式的不断变化，信息交流速度的日益加快，相关性这一信息检索中的关键性概念日益受到人们的重视.本文分析探讨了信息检索中相关性的概念、构成及评价指标，阐述了提高信息检索相关性的措施。     

互联网知识共享平台信息过载效应与弱化机制——基于知乎的案例研究

【目的/意义】信息过载效应成为了制约互联网知识共享平台发展的一大阻碍,本研究致力于探索弱化信息 过载的有效机制。【方法/过程】采用单案例纵向研究方法,选取知乎这一知识共享平台为研究对象,并基于其社区 阶段和平台阶段的动态演变加以研究。【结果/结论】研究发现了互联网知识共享平台弱化信息过载效应的系统机 制--高质信息“生成-选择-传播”机制,并基于知乎发展的不同阶段,发现了上述机制的动态演变性,丰富了信息 过载弱化机制的系统视角,推进了知识共享相关研究在互联网情境下的发展,为互联网知识共享平台提供了切实 可行的管理启示。     

A review of ontology based query expansion

This paper examines the meaning of context in relation to ontology based query expansion and contains a review of query expansion approaches. The various query expansion approaches include relevance feedback, corpus dependent knowledge models and corpus independent knowledge models. Case studies detailing query expansion using domain-specific and domain-independent ontologies are also included. The penultimate section attempts to synthesise the information obtained from the review and provide success factors in using an ontology for query expansion. Finally the area of further research in applying context from an ontology to query expansion within a newswire domain is described.     

Enhancing query translation with relevance feedback in translingual information retrieval

As an effective technique for improving retrieval effectiveness, relevance feedback (RF) has been widely studied in both monolingual and translingual information retrieval (TLIR). The studies of RF in TLIR have been focused on query expansion (QE), in which queries are reformulated before and/or after they are translated. However, RF in TLIR actually not only can help select better query terms, but also can enhance query translation by adjusting translation probabilities and even resolving some out-of-vocabulary terms. In this paper, we propose a novel relevance feedback method called translation enhancement (TE), which uses the extracted translation relationships from relevant documents to revise the translation probabilities of query terms and to identify extra available translation alternatives so that the translated queries are more tuned to the current search. We studied TE using pseudo-relevance feedback (PRF) and interactive relevance feedback (IRF). Our results show that TE can significantly improve TLIR with both types of relevance feedback methods, and that the improvement is comparable to that of query expansion. More importantly, the effects of translation enhancement and query expansion are complementary. Their integration can produce further improvement, and makes TLIR more robust for a variety of queries.     

Relevance feedback and cross-language information retrieval

This paper presents a study of relevance feedback in a cross-language information retrieval environment. We have performed an experiment in which Portuguese speakers are asked to judge the relevance of English documents; documents hand-translated to Portuguese and documents automatically translated to Portuguese. The goals of the experiment were to answer two questions (i) how well can native Portuguese searchers recognise relevant documents written in English, compared to documents that are hand translated and automatically translated to Portuguese; and (ii) what is the impact of misjudged documents on the performance improvement that can be achieved by relevance feedback. Surprisingly, the results show that machine translation is as effective as hand translation in aiding users to assess relevance in the experiment. In addition, the impact of misjudged documents on the performance of RF is overall just moderate, and varies greatly for different query topics.     

Expertise seeking: A review

Expertise seeking is the activity of selecting people as sources for consultation about an information need. This review of 72 expertise-seeking papers shows that across a range of tasks and contexts people, in particular work-group colleagues and other strong ties, are among the most frequently used sources. Studies repeatedly show the influence of the social network – of friendships and personal dislikes – on the expertise-seeking network of organisations. In addition, people are no less prominent than documentary sources, in work contexts as well as daily-life contexts. The relative influence of source quality and source accessibility on source selection varies across studies. Overall, expertise seekers appear to aim for sufficient quality, composed of reliability and relevance, while also attending to accessibility, composed of access to the source and access to the source information. Earlier claims that seekers disregard quality to minimise effort receive little support. Source selection is also affected by task-related, seeker-related, and contextual factors. For example, task complexity has been found to increase the use of information sources whereas task importance has been found to amplify the influence of quality on source selection. Finally, the reviewed studies identify a number of barriers to expertise seeking.     

核心技术、互补资产与后发企业的超越追赶

核心技术与互补资产是后发追赶过程中两种不同的主导策略,后发企业往往面临两种策略如何选择与平衡的难题。本文通过海天塑机集团2001~2017年在全电动注塑机技术领域追赶的案例研究,发现后发企业由后二次创新向一次创新的超越追赶动态过程中,主导追赶策略经历了从核心技术到互补资产再到二者并重的演化;从组织双元视角看,在该过程中,企业经历了从探索与利用序列交替的间断型平衡模式向探索与利用共时进行的双元型平衡模式的转变。本文研究发现不仅揭示了后发企业从后二次创新向一次创新的超越追赶机制,同时深化了我们对后发追赶动态性的理解。     

Aspects of ‘relevance’ in the alignment of curriculum with educational standards

With over 60,000 US K-12 science and mathematics education standards and a rapid proliferation of Web-enabled curriculum, retrieving curriculum that aligns with the standards to which teachers must teach is a key objective for educational digital libraries. However, previous studies of such alignment use single-dimensional and binary measures of the alignment concept. As a consequence, they suffer from low inter-rater reliability (IRR), with experts agreeing about alignments only some 20–40% of the time. We present the results of an experiment in which the alignment variable was operationalized using the Saracevic model of relevance ‘clues’ taken from the everyday practice of K-12 teaching. Results show high IRR across all clues with IRR on several specific alignment dimensions significantly higher than on overall alignment. In addition, a model of overall alignment is derived and estimated. The structure and explanatory power of the model as well as the relationships between alignment clues differ significantly between alignments of curriculum found by users themselves and curriculum found by others. These results illustrate the usefulness of clue-based relevance measures for information retrieval and have important consequences for both the formulation of automated retrieval mechanisms and the construction of a gold standard or benchmark set of standard-curriculum alignments.     

Query expansion and dimensionality reduction: Notions of optimality in Rocchio relevance feedback and latent semantic indexing

Rocchio relevance feedback and latent semantic indexing (LSI) are well-known extensions of the vector space model for information retrieval (IR). This paper analyzes the statistical relationship between these extensions. The analysis focuses on each method’s basis in least-squares optimization. Noting that LSI and Rocchio relevance feedback both alter the vector space model in a way that is in some sense least-squares optimal, we ask: what is the relationship between LSI’s and Rocchio’s notions of optimality? What does this relationship imply for IR? Using an analytical approach, we argue that Rocchio relevance feedback is optimal if we understand retrieval as a simplified classification problem. On the other hand, LSI’s motivation comes to the fore if we understand it as a biased regression technique, where projection onto a low-dimensional orthogonal subspace of the documents reduces model variance.