Investigating the role of in-situ user expectations in Web search

Pre-adoption expectations often serve as an implicit reference point in users’ evaluation of information systems and are closely associated with their goals of interactions, behaviors, and overall satisfaction. Despite the empirically confirmed impacts, users’ search expectations and their connections to tasks, users, search experiences, and behaviors have been scarcely studied in the context of online information search. To address the gap, we collected 116 sessions from 60 participants in a controlled-lab Web search study and gathered direct feedback on their in-situ expected information gains (e.g., number of useful pages) and expected search efforts (e.g., clicks and dwell time) under each query during search sessions. Our study aims to examine (1) how users’ pre-search experience, task characteristics, and in-session experience affect their current expectations and (2) how user expectations are correlated with search behaviors and satisfaction. Our results with both quantitative and qualitative evidence demonstrate that: (1) user expectation is significantly affected by task characteristics, previous and in-situ search experience; (2) user expectation is closely associated with users’ browsing behaviors and search satisfaction. The knowledge learned about user expectation advances our understanding of users’ search behavioral patterns and their evaluations of interaction experience and will also facilitate the design, implementation, and evaluation of expectation-aware user models, metrics, and information retrieval (IR) systems.     

Adapting information retrieval systems to user queries

Users enter queries that are short as well as long. The aim of this work is to evaluate techniques that can enable information retrieval (IR) systems to automatically adapt to perform better on such queries. By adaptation we refer to (1) modifications to the queries via user interaction, and (2) detecting that the original query is not a good candidate for modification. We show that the former has the potential to improve mean average precision (MAP) of long and short queries by 40% and 30% respectively, and that simple user interaction can help towards this goal. We observed that after inspecting the options presented to them, users frequently did not select any. We present techniques in this paper to determine beforehand the utility of user interaction to avoid this waste of time and effort. We show that our techniques can provide IR systems with the ability to detect and avoid interaction for unpromising queries without a significant drop in overall performance.     

Adaptive visualization for exploratory information retrieval

As the volume and breadth of online information is rapidly increasing, ad hoc search systems become less and less efficient to answer information needs of modern users. To support the growing complexity of search tasks, researchers in the field of information developed and explored a range of approaches that extend the traditional ad hoc retrieval paradigm. Among these approaches, personalized search systems and exploratory search systems attracted many followers. Personalized search explored the power of artificial intelligence techniques to provide tailored search results according to different user interests, contexts, and tasks. In contrast, exploratory search capitalized on the power of human intelligence by providing users with more powerful interfaces to support the search process. As these approaches are not contradictory, we believe that they can re-enforce each other. We argue that the effectiveness of personalized search systems may be increased by allowing users to interact with the system and learn/investigate the problem in order to reach the final goal. We also suggest that an interactive visualization approach could offer a good ground to combine the strong sides of personalized and exploratory search approaches. This paper proposes a specific way to integrate interactive visualization and personalized search and introduces an adaptive visualization based search system Adaptive VIBE that implements it. We tested the effectiveness of Adaptive VIBE and investigated its strengths and weaknesses by conducting a full-scale user study. The results show that Adaptive VIBE can improve the precision and the productivity of the personalized search system while helping users to discover more diverse sets of information.     

Exploiting user experience from online customer reviews for product design

Understanding user experience (UX) becomes more important in a market-driven design paradigm because it helps designers uncover significant factors, such as user’s preference, usage context, product features, as well as their interrelations. Conventional means, such as questionnaire, survey and self-report with predefined questions and prompts, are used to collect information about users’ experience during various UX studies. However, such data is often limited and restricted by initial setups, and they won’t easily allow designers to identify all critical elements such as user profile, context, related product features, etc. Meanwhile, with widely accessible social media, the volume and velocity of customer-generated data are fast-increasing. While it is generally acknowledged that such data contains important elements in understanding and analyzing UX, extracting them to assist product design remains a challenging issue. In this study, how UX data underlying product design can be isolated and restored from customer online reviews is examined. A faceted conceptual model is proposed to elucidate the crucial factors of UX, which serves as an operational mechanism connecting to product design. A methodology of establishing a UX knowledge base from customer online reviews is then proposed to support UX-centered design activities, which consists of three stages, i.e., UX discovery to extract UX data from a single review, UX data integration to group similar data and UX network formalization to build up the causal dependencies among UX groups. Using a case study on smart mobile phone reviews, examples of UX data discovered are demonstrated and both customers and designers concerned key product features and usage situations are exemplified. This study explores the feasibility to discover valuable UX data as well as their relations automatically for product design and business strategic plan by analyzing a large volume of customer online data.     

Analysis of multiple query reformulations on the web: The interactive information retrieval context

This study examines the facets and patterns of multiple Web query reformulations with a focus on reformulation sequences. Based on IR interaction models, it was presumed that query reformulation is the product of the interaction between the user and the IR system. Query reformulation also reflects the interplay between the surface and deeper levels of user interaction. Query logs were collected from a Web search engine through the selection of search sessions in which users submitted six or more unique queries per session. The final data set was composed of 313 search sessions. Three facets of query reformulation (content, format, and resource) as well as nine sub-facets were derived from the data. In addition, analysis of modification sequences identified eight distinct patterns: specified, generalized, parallel, building-block, dynamic, multitasking, recurrent, and format reformulation. Adapting Saracevic’s stratified model, the authors develop a model of Web query reformulation based on the results of the study. The implications for Web search engine design are finally discussed and the functions of an interactive reformulation tool are suggested.     

Personalised news and scientific literature aggregation

Content-based filtering can be deployed for personalised information dissemination on the web, but this is a possibility that has been largely ignored. Nowadays, there are no successful content-based filtering applications available online. Nootropia is an immune-inspired user profiling model for content-based filtering. It has the advantageous property to be able to represent a user’s multiple interests and adapt to a variety of changes in them. In this paper we describe our early efforts to develop real world personalisation services based on Nootropia. We present, the architecture, implementation, usage and evaluation of the personalised news and paper aggregator, which aggregates news and papers that are relevant to an individual’s interests. Our user study shows that Nootropia can effectively learn a user’s interests and identify relevant information. It also indicates that information filtering is a complicated task with many factors affecting its successful application in a real situation.     

Division of labour and sharing of knowledge for synchronous collaborative information retrieval

Synchronous collaborative information retrieval (SCIR) is concerned with supporting two or more users who search together at the same time in order to satisfy a shared information need. SCIR systems represent a paradigmatic shift in the way we view information retrieval, moving from an individual to a group process and as such the development of novel IR techniques is needed to support this. In this article we present what we believe are two key concepts for the development of effective SCIR namely division of labour (DoL) and sharing of knowledge (SoK). Together these concepts enable coordinated SCIR such that redundancy across group members is reduced whilst enabling each group member to benefit from the discoveries of their collaborators. In this article we outline techniques from state-of-the-art SCIR systems which support these two concepts, primarily through the provision of awareness widgets. We then outline some of our own work into system-mediated techniques for division of labour and sharing of knowledge in SCIR. Finally we conclude with a discussion on some possible future trends for these two coordination techniques.     

An implicit feedback approach for interactive information retrieval

Searchers can face problems finding the information they seek. One reason for this is that they may have difficulty devising queries to express their information needs. In this article, we describe an approach that uses unobtrusive monitoring of interaction to proactively support searchers. The approach chooses terms to better represent information needs by monitoring searcher interaction with different representations of top-ranked documents. Information needs are dynamic and can change as a searcher views information. The approach we propose gathers evidence on potential changes in these needs and uses this evidence to choose new retrieval strategies. We present an evaluation of how well our technique estimates information needs, how well it estimates changes in these needs and the appropriateness of the interface support it offers. The results are presented and the avenues for future research identified.     

Examining the generalizability of the User Engagement Scale (UES) in exploratory search

The user experience is an integral component of interactive information retrieval (IIR). However, there is a twofold problem in its measurement. Firstly, while many IIR studies have relied on a single dimension of user feedback, that of satisfaction, experience is a much more complex concept. IIR in general, and exploratory search more specifically, are dynamic, multifaceted experiences that evoke pragmatic and hedonic needs, expectations, and outcomes that are not adequately captured by user satisfaction. Secondly, questionnaires, which are typically the means in which user’s attitudes and perceptions are measured, are not typically subjected to rigorous reliability and validity testing. To address these issues, we administered the multidimensional User Engagement Scale (UES) in an exploratory search environment to assess users’ perceptions of the Perceived Usability (PUs), Aesthetics (AE), Novelty (NO), Felt Involvement (FI), Focused Attention (FA), and Endurability (EN) aspects of the experience. In a typical laboratory-style study, 381 participants performed three relatively complex search tasks using a novel search interface, and responded to the UES immediately upon completion. We used Principal Axis Factor Analysis and Multiple Regression to examine the factor structure of UES items and the relationships amongst factors. Results showed that three of the six sub-scales (PUs, AE, FA) were stable, while NO, FI and EN merged to form a single factor. We discuss recommendations for revising and validating the UES in light of these findings.     

A study of the information needs of the users of a folk music library and the implications for the design of a digital library system

A qualitative study of user information needs is reported, based on a purposive sample of users and potential users of the Vaughan Williams Memorial Library, a small specialist folk music library in North London. The study set out to establish what the users’ (both existing and potential) information needs are, so that the library’s online service may take them into account with its design. The information needs framework proposed by Nicholas [Nicholas, D. (2000) Assessing information needs: tools, techniques and concepts for the internet age. London: ASLIB] is used as an analytical tool to achieve this end. The demographics of the users were examined in order to establish four user groups: Performer, Academic, Professional and Enthusiast. Important information needs were found to be based on social interaction, and key resources of the library were its staff, the concentration of the collection and the library’s social nature. A collection of broad design requirements are proposed based on the analysis and this study also provides some insights into the issue of musical relevance, which are discussed.     

A socio-technical approach to knowledge contribution behavior: An empirical investigation of social networking sites users

Online social networking has received increasing attention as a new phenomenon among online users. As Internet users utilize online social networking websites as a useful communication tool to maintain their social networks, this study explorers online social networking websites users’ knowledge sharing in particular. This study investigated the factors which influence knowledge contribution behaviors of social networking website users by sharing through user created contents with one another. By employing a socio-technical approach, this study discussed the roles of social system factors such as ethical culture, social tie, and a sense of belonging in online social network. Additionally, this study examined technical systems factors such as structural assurance of service providers and structural assurance of the Internet. The survey method was utilized in order to empirically test the research model. The research findings and contributions are discussed as well.     

An evaluation of adaptive filtering in the context of realistic task-based information exploration

Exploratory search increasingly becomes an important research topic. Our interests focus on task-based information exploration, a specific type of exploratory search performed by a range of professional users, such as intelligence analysts. In this paper, we present an evaluation framework designed specifically for assessing and comparing performance of innovative information access tools created to support the work of intelligence analysts in the context of task-based information exploration. The motivation for the development of this framework came from our needs for testing systems in task-based information exploration, which cannot be satisfied by existing frameworks. The new framework is closely tied with the kind of tasks that intelligence analysts perform: complex, dynamic, and multiple facets and multiple stages. It views the user rather than the information system as the center of the evaluation, and examines how well users are served by the systems in their tasks. The evaluation framework examines the support of the systems at users’ major information access stages, such as information foraging and sense-making. The framework is accompanied by a reference test collection that has 18 tasks scenarios and corresponding passage-level ground truth annotations. To demonstrate the usage of the framework and the reference test collection, we present a specific evaluation study on CAFÉ, an adaptive filtering engine designed for supporting task-based information exploration. This study is a successful use case of the framework, and the study indeed revealed various aspects of the information systems and their roles in supporting task-based information exploration.     

Quality assessment of answers with user-identified criteria and data-driven features in social Q&A

The purpose of the current study is to identify the user criteria and data-driven features, both textual and non-textual, for assessing the quality of answers posted on social questioning and answering sites (social Q&A) across four different knowledge domains—Science, Technology, Art and Recreation. A comprehensive review of literature on quality assessment of information produced in social contexts was carried out to develop the theoretical framework for the current study. A total of 23 user criteria and 24 data features were proposed and tested with high-quality answers obtained from four social Q&A sites in Stack Exchange. Findings indicate that content-related criteria and user and review features were the most frequently used in quality assessments, while the importance of user criteria and data features was variable across the knowledge domains. In the Technology Q&A site containing mostly self-help questions, the utility class was the most frequently used group of criteria. The popularity of the socio-emotional class was more apparent in discussion-oriented topic categories such as Art and Recreation, where people seek others’ opinions or advice. Users of Art and Recreation Q&A sites in Stack Exchange appear to place more value on answerers’ efforts and time, good attitudes or manners, personal experience, and the same taste. The importance of user features and the emphasis on answerer's expertise on the Science Q&A site was observed. Examining the connection or gap between user quality criteria and data features across the knowledge domains could help to better understand users’ evaluation behaviors for their preferred answers, and identify the potential of social Q&A for user education/intervention in answer quality evaluation. This examination also offers practical guidance for designing more effective social Q&A platforms, considering how to customize community support systems, motivate contributions, and control content quality.     

Search history support for finding and using information: User interface design recommendations from a user study

Recording search histories, presenting them to the searcher, and building additional interface tools on them offer many opportunities for supporting user tasks in information seeking and use. This study investigated the use of search history information in legal information seeking. Qualitative methods were used to explore how attorneys and law librarians used their memory and external memory aids while searching for information and in transferring to information use. Based on the findings, interface design recommendations were made for information systems.     

Best entry points for structured document retrieval—Part II: Types,usage and effectiveness

Structured document retrieval makes use of document components as the basis of the retrieval process, rather than complete documents. The inherent relationships between these components make it vital to support users’ natural browsing behaviour in order to offer effective and efficient access to structured documents. This paper examines the concept of best entry points, which are document components from which the user can browse to obtain optimal access to relevant document components. It investigates at the types of best entry points in structured document retrieval, and their usage and effectiveness in real information search tasks.     

Best entry points for structured document retrieval—Part I: Characteristics

Structured document retrieval makes use of document components as the basis of the retrieval process, rather than complete documents. The inherent relationships between these components make it vital to support users’ natural browsing behaviour in order to offer effective and efficient access to structured documents. This paper examines the concept of best entry points, which are document components from which the user can browse to obtain optimal access to relevant document components. In particular this paper investigates the basic characteristics of best entry points.     

Specifics of information retrieval for young users: A survey

In this paper, we present the state of the art in the field of information retrieval that is relevant for understanding how to design information retrieval systems for children. We describe basic theories of human development to explain the specifics of young users, i.e., their cognitive skills, fine motor skills, knowledge, memory and emotional states in so far as they differ from those of adults. We derive the implications these differences have on the design of information retrieval systems for children. Furthermore, we summarize the main findings about children’s search behavior from multiple user studies. These findings are important to understand children’s information needs, their search strategies and usage of information retrieval systems. We also identify several weaknesses of previous user studies about children’s information-seeking behavior. Guided by the findings of these user studies, we describe challenges for the design of information retrieval systems for young users. We give an overview of algorithms and user interface concepts. We also describe existing information retrieval systems for children, in specific web search engines and digital libraries. We conclude with a discussion of open issues and directions for further research. The survey provided in this paper is important both for designers of information retrieval systems for young users as well as for researchers who start working in this field.