Automatic question answering using the web: Beyond the Factoid

In this paper we describe and evaluate a Question Answering (QA) system that goes beyond answering factoid questions. Our approach to QA assumes no restrictions on the type of questions that are handled, and no assumption that the answers to be provided are factoids. We present an unsupervised approach for collecting question and answer pairs from FAQ pages, which we use to collect a corpus of 1 million question/answer pairs from FAQ pages available on the Web. This corpus is used to train various statistical models employed by our QA system: a statistical chunker used to transform a natural language-posed question into a phrase-based query to be submitted for exact match to an off-the-shelf search engine; an answer/question translation model, used to assess the likelihood that a proposed answer is indeed an answer to the posed question; and an answer language model, used to assess the likelihood that a proposed answer is a well-formed answer. We evaluate our QA system in a modular fashion, by comparing the performance of baseline algorithms against our proposed algorithms for various modules in our QA system. The evaluation shows that our system achieves reasonable performance in terms of answer accuracy for a large variety of complex, non-factoid questions.     

Enhancing web search in the medical domain via query clarification

The majority of Internet users search for medical information online; however, many do not have an adequate medical vocabulary. Users might have difficulties finding the most authoritative and useful information because they are unfamiliar with the appropriate medical expressions describing their condition; consequently, they are unable to adequately satisfy their information need. We investigate the utility of bridging the gap between layperson and expert vocabularies; our approach adds the most appropriate expert expression to queries submitted by users, a task we call query clarification. We evaluated the impact of query clarification. Using three different synonym mappings and conducting two task-based retrieval studies, users were asked to answer medically-related questions using interleaved results from a major search engine. Our results show that the proposed system was preferred by users and helped them answer medical concerns correctly more often, with up to a 7 % increase in correct answers over an unmodified query. Finally, we introduce a supervised classifier to select the most appropriate synonym mapping for each query, which further increased the fraction of correct answers (12 %).     

A deniable and efficient question and answer service over ad hoc social networks

When people are connected together over ad hoc social networks, it is possible to ask questions and retrieve answers using the wisdom of the crowd. However, locating a suitable candidate for answering a specific unique question within larger ad hoc groups is non-trivial, especially if we wish to respect the privacy of users by providing deniability. All members of the network wish to source the best possible answers from the network, while at the same time controlling the levels of attention required to generate them by the collective group of individuals and/or the time taken to read all the answers. Conventional expert retrieval approaches rank users for a given query in a centralised indexing process, associating users with material they have previously published. Such an approach is antithetical to privacy, so we have looked to distribute the routing of questions and answers, converting the indexing process into one of building a forwarding table. Starting from the simple operation of flooding the question to everyone, we compare a number of different routing options, where decisions must be made based on past performance and exploitation of the knowledge of our immediate neighbours. We focus on fully decentralised protocols using ant-inspired tactics to route questions towards members of the network who may be able to answer them well. Simultaneously, privacy concerns are acknowledged by allowing both question asking and answering to be plausibly deniable. We have found that via our routing method, it is possible to improve answer quality and also reduce the total amount of user attention required to generate those answers.     

Government Documents Reference Service in Canada: A Nationwide Unobtrusive Study of Public and Academic Depository Libraries

This study reports on a nationwide unobtrusive evaluation of government documents reference service at public and academic depository libraries in Canada. Fifteen questions dealing with subject matter from both the legislative and executive branches of government were asked 488 times at 104 depository libraries in 30 census metropolitan areas. Overall, depository library staff members provided complete answers to questions 29.3% of the time. When complete and partially complete answers are counted together, the success rate climbs to 42.4%. Academic full depositories achieved the highest rate of success, followed by public full depositories. In-person questions were answered more successfully than phone questions. Print materials were by far the largest single source used (45.7%) to answer questions. When print alone was used, complete answers to the test questions were found only 39.9% of the time. When World Wide Web sources alone were used, the complete answer rate was 60.7%. To improve service, extensive and periodic staff training may be needed about the structures and functions of both the legislative and executive branches of government. Staff members need to know what programs are available and who is responsible for which program in the federal government.     

Evidence‐based medicine: is the evidence out there for primary care clinicians?

Background: Finding evidence to answer clinical questions is essential to the practice of evidence‐based medicine (EBM). However, practising EBM in primary care is thought to be problematic because of concerns about whether evidence exists to answer specific questions. Objectives: To determine the highest level of evidence per question; to ascertain the number of questions unanswered because of a lack of evidence; to establish the frequency with which guidelines answered questions; and to determine the domain of websites used to answer questions. Methods: Clinical questions were identified from two primary care answering services: ATTRACT and National Library for Health (NLH) Primary Care Answering Service. The types of evidence used to answer the question were noted, including whether this was from systematic reviews or meta‐analyses (level one evidence) or from randomised controlled trials (level two). The data were collected from March to June 2008. Results: Level 1 or level 2 evidence answered 11% of questions. Sixteen per cent were unanswered because of a lack of evidence. Over 40% of questions were answered using guidelines. Forty‐three per cent of questions were answered with one type of evidence and 24% with two. Conclusion: Guidelines are useful resources for primary care clinicians, answering two‐fifths of questions.     

Multicenter randomized comparative trial of Micromedex,Micromedex with Watson,or Google to answer drug information questions

Objective:The purpose of this study was to compare pharmacy students’ ability to correctly answer drug information questions using Micromedex with Watson, Micromedex without Watson, or Google.Methods:This multicenter randomized trial compared pharmacy student responses to drug information questions using Micromedex with Watson, Micromedex without Watson, or Google from January to March of 2020. First- to fourth-year pharmacy students at two institutions were included. The primary outcome was the number of correct answers. Secondary outcomes were the time taken to answer the questions and differences in number of correct answers by pharmacy student year and institution.Results:The analysis included 162 participants: 52 students in the Micromedex group, 51 students in the Watson group, and 59 students in the Google group. There was a significant difference among groups in the total number of questions answered correctly (p=0.02). Post-hoc analysis revealed that participants in the Micromedex group answered more questions correctly than those in the Google group (p=0.015). There were no significant differences between Micromedex and Watson groups (p=0.52) or between Watson and Google groups (p=0.22). There was also no difference in time to complete the questions among groups (p=0.72).Conclusion:Utilizing Google did not save students time and led to more incorrect answers. These findings suggest that health care educators and health sciences librarians should further reinforce training on the appropriate use of drug information resources.     

Evidence-based databases versus primary medical literature: an in-house investigation on their optimal use.

OBJECTIVE: The objective is to investigate the effectiveness of evidence-based medicine (EBM) resources in providing evidence for complex clinical questions versus general care management questions to identify situations for their optimal use. METHODS: In this investigation, forty complex clinical questions were randomly selected from an in-house archival database of questions received by librarians during clinical rounds. An additional forty questions were selected from a list of general care management questions received by the library from Pathways teams. To measure the effectiveness of resources in answering the questions, a team of librarians was asked to search UpToDate, the Cochrane Database of Systematic Reviews, and HealthGate Clinical Guidelines (formerly, EBM Solutions). The team then established consensus on whether a question was fully or partially answered by one of the above-mentioned EBM resources and was instructed to record the instances in which the primary literature needed to be used to answer the question completely. RESULTS: The study found that the EBM resources completely answered 20.0% of the 40 complex clinical questions and 47.5% of the 40 general care management questions, partially answered 40.0% of the complex clinical questions and 22.5% of the general care management questions, and did not answer 40.0% of the complex clinical questions and 30.0% of the general care management questions. CONCLUSION: The pervasive use of EBM resources in answering clinical questions is making it imperative for information specialists to develop an expertise on their appropriate use. By exploring their use in answering complex clinical questions and general care management questions, this paper underlines the strengths and weakness of EBM resources and provides information specialists with some basic knowledge about how these resources can be combined with the primary literature to strengthen their effectiveness.     

UpToDate versus DynaMed: a cross-sectional study comparing the speed and accuracy of two point-of-care information tools

Objective:To compare the accuracy, time to answer, user confidence, and user satisfaction between UpToDate and DynaMed (formerly DynaMed Plus), which are two popular point-of-care information tools.Methods:A crossover study was conducted with medical residents in obstetrics and gynecology and family medicine at the University of Toronto in order to compare the speed and accuracy with which they retrieved answers to clinical questions using UpToDate and DynaMed. Experiments took place between February 2017 and December 2019. Following a short tutorial on how to use each tool and completion of a background survey, participants attempted to find answers to two clinical questions in each tool. Time to answer each question, the chosen answer, confidence score, and satisfaction score were recorded for each clinical question.Results:A total of 57 residents took part in the experiment, including 32 from family medicine and 25 from obstetrics and gynecology. Accuracy in clinical answers was equal between UpToDate (average 1.35 out of 2) and DynaMed (average 1.36 out of 2). However, time to answer was 2.5 minutes faster in UpToDate compared to DynaMed. Participants were also more confident and satisfied with their answers in UpToDate compared to DynaMed.Conclusions:Despite a preference for UpToDate and a higher confidence in responses, the accuracy of clinical answers in UpToDate was equal to those in DynaMed. Previous exposure to UpToDate likely played a major role in participants'' preferences. More research in this area is recommended.     

Effect of an evidence-based answering service on GPs and their patients: a pilot study

Objectives: For general practitioners (GPs), an important obstacle to practising evidence‐based medicine is lack of time. An evidence‐based answering service was developed that took over searching and appraisal of medical evidence from the GPs. GPs sent in questions, and the informationist formulated the answers. Our objectives were to find out if such an evidence‐based answering service was feasible, including assessing the effect of the answers on GPs and their patients, as reported by the GPs. Methods: After attending a workshop on building well‐formulated questions from daily practice, the GPs sent in questions to the informationist. The literature was searched, the relevant information was appraised, and the answers to the questions were formulated. With a questionnaire, the effect of the answers on the GP and the patient was assessed, as well as the perceived barriers to implementing the answers. Results: From 26 GPs, 61 questions were received. For 12% of questions, information was found with the highest evidence level, while for 36%, no information was found. However, for 89% of the questions for which no information could be found, the answer ‘no information found’ did have an effect on the GP concerned. In total, 81% of all the answers had an effect on the GP, and, according to the GP, 52% had an effect on the patient. Few barriers to implementing the evidence were perceived. Most of the answers were found in Pubmed/Medline, the Cochrane Library and Embase. Conclusions: This study indicates that an evidence‐based answering service can have an impact on GPs and their patients. Librarians can provide an evidence‐based answering service for GPs and their patients. The evidence‐based answering service for GPs in this study had an impact on 81% of the GPs and on 52% of their patients. Although for one‐third of the questions no evidence‐based answer was found, this message in itself had an impact on 89% of the GPs. An informationist as mediator between medical information and doctors can save doctors’ time.     

Characteristics of E-Mail Reference Services in Selected Public Libraries,Victoria, Australia

SUMMARY

An analysis of 96 question and answer pairs from the Bayside Library Ask a Librarian Service found that 54 percent of the queries were received from Bayside residents. Forty-seven percent of the e-mail reference questions were classed as research queries. Although only 25.1 percent of the queries were submitted for formal education purposes, all of these were research questions, and took longer than any other category to answer. In 2001, only 6 of the 54 questions submitted were tertiary level questions, but it took a median time of 95 minutes to answer each one. The 24 general interest category questions took a median time of 47.5 minutes to answer, which is almost half the time it took to answer a tertiary level query.

Librarians from three other public libraries in Victoria offering e-mail reference were interviewed, and compared and contrasted with the Bayside Library Service.

Issues of disproportionate labour, the appearance of the passive role of the e-mail reference user, and the wisdom of public libraries devoting significant resources to answer questions for formal education were raised.     

课堂提问框架在自然博物馆讲解中的运用

博物馆讲解中的提问互动,在吸引观众注意力、激发观众兴趣、引起观众思考、加深观众对展览理解方面发挥着重要的作用。本文利用课堂教学研究常用的提问框架分析了两位自然博物馆教育人员讲解过程中的提问类型、观众回答情况和教育人员的反馈方式。发现教育人员有比较强烈的提问互动意识,能够围绕展品和讲解主题提问,但还存在问题层次性不明显、对观众认知水平分析不够、各类问题比例不均衡、候答时间过短、提问反馈中有效追问和鼓励不足等问题。可以通过设置核心问题、基于观众认知水平控制问题难度、增加问题多样性、适时调控候答时间、在提问反馈中追问并增加评价性语言等方法提升提问的有效性。     

Analysis of Statistical Question Classification for Fact-Based Questions

Question classification systems play an important role in question answering systems and can be used in a wide range of other domains. The goal of question classification is to accurately assign labels to questions based on expected answer type. Most approaches in the past have relied on matching questions against hand-crafted rules. However, rules require laborious effort to create and often suffer from being too specific. Statistical question classification methods overcome these issues by employing machine learning techniques. We empirically show that a statistical approach is robust and achieves good performance on three diverse data sets with little or no hand tuning. Furthermore, we examine the role different syntactic and semantic features have on performance. We find that semantic features tend to increase performance more than purely syntactic features. Finally, we analyze common causes of misclassification error and provide insight into ways they may be overcome.     

Accuracy of telephone reference service in health sciences libraries

Six factual queries were unobtrusively telephoned to fifty-one U.S. academic health sciences and hospital libraries. The majority of the queries (63.4%) were answered accurately. Referrals to another library or information source were made for 25.2% of the queries. Eleven answers (3.6%) were inaccurate, and no answer was provided for 7.8% of the queries. There was a correlation between the number of accurate answers provided and the presence of at least one staff member with a master's degree in library and information science. The correlation between employing a librarian certified by the Medical Library Association (MLA) and providing accurate answers was significant. The majority of referrals were to specific sources. If these "helpful referrals" are counted with accurate answers as correct responses, they total 76.8% of the answers. In a follow-up survey, five libraries stated that they did not provide accurate answers because they did not own an appropriate source. Staff-related problems were given as reasons for other than accurate answers by two of the libraries, while eight indicated that library policy prevented them from providing answers to the public.     

FWG1法在社会化问答平台答案质量评测体系构建中的应用研究

[目的/意义]答案质量越发成为影响社会化问答(SQA)平台使用效果的重要因素。构建评测体系对SQA平台答案质量进行评价,可以优化用户体验和促进平台发展。[方法/过程]结合专家评分法及三角模糊加权平均G1法,构建包含12个指标的SQA平台答案质量评测指标体系。通过对4个领域共40个问题的评价,证实指标体系的科学性。[结果/结论]形成一套可用于整体评价、赋予指标权重的评测体系,了解了国内SQA平台的答案质量和存在的问题,为平台发展及后续研究提供参考。     

Rhetorical detailing in Cicero's Verrine orations

This article offers answers to three questions: Why, after his courtroom victory in the Verrine case, did Cicero publish his speech? What changes in his rhetorical theory are reflected in the speech? What principles of narrative suasion is Cicero using in the many stories he incorporates within the speech? The concept of rhetorical detailing is proposed as a key element in the probable answer to each question.     

一种基于短文本相似度计算的主观题自动阅卷方法

文本主观题自动阅卷的关键是提高考生答案文本和试题标准答案文本之间相似度计算结果的准确率。参考文本试题人工阅卷方法,提出一种结合人工制定文本相似标准、词语集合及词语次序和同义词的短文本相似度计算方法,设计并实现相应文本主观题阅卷系统。建立试题人工评分标准库,并在387道银行培训领域真实考题数据集上进行自动阅卷与人工阅卷结果对比实验。结果显示,文本试题自动阅卷结果与人工阅卷结果相比,完全相同的达到58%,准确率达到80%左右。     

Vendor studies redux: Evaluating the approval plan option from within

Exploring the question, “Once approval plans are established, how does one know whether they are preferable to firm ordering?”, this paper questions traditional assumptions about discounts and staff savings in a library with numerous approval plan agreements. Analyzing the acquisitions patterns of four different groups of selectors at a large research library, it concludes that approval plans profiled to obtain core essentials are preferred by many selectors over firm ordering these materials. Profiles implementing this approach appear to be more difficult to establish for European than for domestic materials. Levels and types of services expected from vendors in a budgetarily constrained environment are enumerated.     

How Many Libraries Do We Need?

ABSTRACT

According to the American Library Association, there are 119,000 libraries in the United States. But how many do we need? In this vision session report, Arizona State University Librarian James J. O’Donnell addresses this question through the lenses of discovery tools, usability, and marketing strategy. He also addresses the need for a more global and collaborative approach to collection building in libraries. This deliberately provocative talk provides several different ways to think about the answer to the title question—and several answers.     

What's in a Name?

Abstract

Many archivists believe that outreach is an important part of their job. However, how do archivists define the term outreach? How does outreach compare to basic services? What types of outreach programs are being done? This article reports the results of a survey that asked college and university archivists to answer these and similar questions. The author describes the respondents' answers, compares their opinions about outreach and basic services, and reports the types of outreach programs that are done. The article concludes with a new and more inclusive definition of outreach that is based on the respondents' answers.