Strategies in assessing the need for updating evidence-based guidelines for six clinical topics: an exploration of two search methodologies

BACKGROUND: Because of the expense of updating practice guidelines, recent attention has focused on approaches that can reliably assess any updating required. Shekelle et al. (Journal of the American Medical Association 2001, 286, 1461-7) proposed using limited literature searches with expert involvement to reduce resources used in assessing whether a guideline needs updating. OBJECTIVES: This study compared Shekelle's method and the traditional systematic review method regarding comprehensiveness and effort. METHODS: Two research teams translated critical key questions on screening test treatments and outcomes to Medical Subjects Headings (MeSH) and search strategies. They refined Shekelle's method over three iterations, seeking greater efficiency. Using both methods independently, teams assessed the need to update six topics from the 1996 Guide to Clinical Preventive Services (US Preventive Services Task Force). Outcomes included completeness of study identification, importance of missed studies and effort involved. RESULTS: The revised review approach produced fewer citations than the traditional approach and saved time, identifying fewer eligible studies than the traditional approach. None of the studies missed was rated important by the experts consulted. CONCLUSIONS: The revised review approach provides an acceptable method for judging whether a guideline requires updating. Librarians were an integral part of the research process that streamlined the searches.     

文献搜索引擎检索结果集成方法的研究

本文针对现有的文献搜索引擎检索结果不全面的问题,提出了多关键词组合加权检索及其结果集成方法。通过计算不同文献搜索引擎检索结果和多关键词组合的相关度,综合应用规范分数集成法和加权分数集成法,实现了不同文献搜索引擎检索结果的集成和综合排序。最后,通过实例分析验证了该方法的有效性。     

Use of a search summary table to improve systematic review search methods,results, and efficiency

Background:Systematic reviews are comprehensive, robust, inclusive, transparent, and reproducible when bringing together the evidence to answer a research question. Various guidelines provide recommendations on the expertise required to conduct a systematic review, where and how to search for literature, and what should be reported in the published review. However, the finer details of the search results are not typically reported to allow the search methods or search efficiency to be evaluated.Case Presentation:This case study presents a search summary table, containing the details of which databases were searched, which supplementary search methods were used, and where the included articles were found. It was developed and published alongside a recent systematic review. This simple format can be used in future systematic reviews to improve search results reporting.Conclusions:Publishing a search summary table in all systematic reviews would add to the growing evidence base about information retrieval, which would help in determining which databases to search for which type of review (in terms of either topic or scope), what supplementary search methods are most effective, what type of literature is being included, and where it is found. It would also provide evidence for future searching and search methods research.     

Developing systematic search methods for the library literature: Methods and analysis

Comprehensive yet efficient search methods are essential for any systematic or scoping review. This article outlines the stages of development of a systematic search methodology for a scoping review within the library and information science (LIS) literature. The effectiveness of the database search strategies (LISTA, LISA, ERIC, Scopus, Web of Science) and supplemental search techniques are measured through a retrospective analysis of performance metrics. Findings show that for research topics limited to the library setting, it may be more effective to search fewer databases (LISTA and Scopus only) for peer reviewed journal articles and allot more time to alternate search techniques such as web searching to identify non-journal literature. The article provides an evidence-based, methodological approach to developing a systematic search plan, unique to LIS researchers, that accounts for time and resource needs.     

Variation in number of hits for complex searches in Google Scholar

Objective

Google Scholar is often used to search for medical literature. Numbers of results reported by Google Scholar outperform the numbers reported by traditional databases. How reliable are these numbers? Why are often not all available 1,000 references shown?

Methods

For several complex search strategies used in systematic review projects, the number of citations and the total number of versions were calculated. Several search strategies were followed over a two-year period, registering fluctuations in reported search results.

Results

Changes in numbers of reported search results varied enormously between search strategies and dates. Theories for calculations of the reported and shown number of hits were not proved.

Conclusions

The number of hits reported in Google Scholar is an unreliable measure. Therefore, its repeatability is problematic, at least when equal results are needed.     

A user study of manual and MEDLINE literature searches in the hospital library

To determine possible uses for MEDLINE in the hospital library, health practitioners requesting literature searches were provided with both a manual and a MEDLINE search. Patron receptivity to MEDLINE was good, but the manual search was more often preferred. The health practitioners responding to a questionnaire apparently had delegated literature searching and evaluation of material to the hospital librarians, especially when clinically related information was needed. In the community hospitals where both the search and the selection of material are the responsibility of the librarian, MEDLINE can be used as the primary search tool or to supplement material provided.     

Searching ClinicalTrials.gov and the International Clinical Trials Registry Platform to inform systematic reviews: what are the optimal search approaches?

Background:

Since 2005, International Committee of Medical Journal Editors (ICMJE) member journals have required that clinical trials be registered in publicly available trials registers before they are considered for publication.

Objectives:

The research explores whether it is adequate, when searching to inform systematic reviews, to search for relevant clinical trials using only public trials registers and to identify the optimal search approaches in trials registers.

Methods:

A search was conducted in ClinicalTrials.gov and the International Clinical Trials Registry Platform (ICTRP) for research studies that had been included in eight systematic reviews. Four search approaches (highly sensitive, sensitive, precise, and highly precise) were performed using the basic and advanced interfaces in both resources.

Results:

On average, 84% of studies were not listed in either resource. The largest number of included studies was retrieved in ClinicalTrials.gov and ICTRP when a sensitive search approach was used in the basic interface. The use of the advanced interface maintained or improved sensitivity in 16 of 19 strategies for Clinicaltrials.gov and 8 of 18 for ICTRP. No single search approach was sensitive enough to identify all studies included in the 6 reviews.

Conclusions:

Trials registers cannot yet be relied upon as the sole means to locate trials for systematic reviews. Trials registers lag behind the major bibliographic databases in terms of their search interfaces.

Implications:

For systematic reviews, trials registers and major bibliographic databases should be searched. Trials registers should be searched using sensitive approaches, and both the registers consulted in this study should be searched.Clinical trials registers such as ClinicalTrials.gov and portals to trials registers such as the World Health Organization (WHO) International Clinical Trials Registry Platform (ICTRP) are increasingly used to identify ongoing or completed clinical trials. These resources offer important information on the methods and progress of trials likely to be of interest to a range of users, including researchers, clinicians, and patients. The extent to which these resources can be relied upon as a sole source of trials for inclusion in systematic reviews, including Cochrane systematic reviews (CSRs), is the subject of the research study reported here. This study also investigates the most efficient ways that librarians, information professionals, and other searchers can search these resources. Search efficiency was investigated by evaluating the overlap and unique yield of searches in the two resources and by testing four search approaches. The tested search approaches ranged from the very precise (single specific condition search term combined with a single specific intervention search term) to the very sensitive (at least two interventions terms).     

Database choices in endocrine systematic reviews

Objective

The choice of bibliographic database during the systematic review search process has been an ongoing conversation among information specialists. With newer information sources, such as Google Scholar and clinical trials registries, we were interested in which databases were utilized by information specialists and systematic review researchers.

Method

We retrieved 144 systematic reviews and meta-analyses from 4 clinical endocrinology journals and extracted all information sources used during the search processes.

Results

Findings indicate that traditional bibliographic databases are most often used, followed by regional databases, clinical trials registries, and gray literature databases.

Conclusions

This study informs information specialists about additional resources that may be considered during the search process.     

Do published search filters to identify diagnostic test accuracy studies perform adequately?

OBJECTIVES: To assess the performance of published search filters in finding diagnostic test accuracy studies. METHODS: Diagnostic test accuracy search filters were identified by searching medline, our own files and by requesting unpublished filters from colleagues. We applied the filters to a case study review of diagnostic test accuracy studies for urinary tract infections (UTI) in young children. The included studies with records in medline formed the gold standard. The performance of the filters in finding those gold standard records was assessed. RESULTS: We identified twenty-three diagnostic test accuracy search filters for use with medline. The case study systematic review of UTI included 179 studies of diagnostic test accuracy, of which 160 were available in medline. The filters showed a wide range of sensitivities (range: 20.6% to 86.9%) and precision (range: 1% to 9.4%). CONCLUSIONS: Our results broadly support those reported in two other studies. The search filters tested do not offer an adequate trade-off between sensitivity and precision to be used to identify studies for systematic reviews. However, there are methods available to explore whether search filters are viable based on an objective statistical analysis of the text and indexing used in records.     

Search methods for prognostic factor systematic reviews: a methodologic investigation

Objective:This study retroactively investigated the search used in a 2019 review by Hayden et al., one of the first systematic reviews of prognostic factors that was published in the Cochrane Library. The review was designed to address recognized weaknesses in reviews of prognosis by using multiple supplementary search methods in addition to traditional electronic database searching.Methods:The authors used four approaches to comprehensively assess aspects of systematic review literature searching for prognostic factor studies: (1) comparison of search recall of broad versus focused electronic search strategies, (2) linking of search methods of origin for eligible studies, (3) analysis of impact of supplementary search methods on meta-analysis conclusions, and (4) analysis of prognosis filter performance.Results:The review''s focused electronic search strategy resulted in a 91% reduction in recall, compared to a broader version. Had the team relied on the focused search strategy without using supplementary search methods, they would have missed 23 of 58 eligible studies that were indexed in MEDLINE; additionally, the number of included studies in 2 of the review''s primary outcome meta-analyses would have changed. Using a broader strategy without supplementary searches would still have missed 5 studies. The prognosis filter used in the review demonstrated the highest sensitivity of any of the filters tested.Conclusions:Our study results support recommendations for supplementary search methods made by prominent systematic review methodologists. Leaving out any supplemental search methods would have resulted in missed studies, and these omissions would not have been prevented by using a broader search strategy or any of the other prognosis filters tested.

Open in a separate windowLeah Boulos     

Consistency and accuracy of indexing systematic review articles and meta‐analyses in medline

Background: Systematic review articles support the advance of science and translation of research evidence into healthcare practice. Inaccurate retrieval from medline could limit access to reviews. Objective: To determine the quality of indexing systematic reviews and meta‐analyses in medline . Methods: The Clinical Hedges Database, containing the results of a hand search of 161 journals, was used to test medline indexing terms for their ability to retrieve systematic reviews that met predefined methodologic criteria (labelled as ‘pass’ review articles) and reviews that reported a meta‐analysis. Results: The Clinical Hedges Database contained 49 028 articles; 753 were ‘pass’ review articles (552 with a meta‐analysis). In total 758 review articles (independent of whether they passed) reported a meta‐analysis. The search strategy that retrieved the highest number of ‘pass’ systematic reviews achieved a sensitivity of 97.1%. The publication type ‘meta analysis’ had a false positive rate of 5.6% (95% CI 3.9 to 7.6), and false negative rate of 0.31% (95% CI 0.26 to 0.36) for retrieving systematic reviews that reported a meta‐analysis. Conclusions: Inaccuracies in indexing systematic reviews and meta‐analyses in medline can be partly overcome by a 5‐term search strategy. Introducing a publication type for systematic reviews of the literature could improve retrieval performance.     

The decisions and processes involved in a systematic search strategy: a hierarchical framework

Objective:The decisions and processes that may compose a systematic search strategy have not been formally identified and categorized. This study aimed to (1) identify all decisions that could be made and processes that could be used in a systematic search strategy and (2) create a hierarchical framework of those decisions and processes.Methods:The literature was searched for documents or guides on conducting a literature search for a systematic review or other evidence synthesis. The decisions or processes for locating studies were extracted from eligible documents and categorized into a structured hierarchical framework. Feedback from experts was sought to revise the framework. The framework was revised iteratively and tested using recently published literature on systematic searching.Results:Guidance documents were identified from expert organizations and a search of the literature and Internet. Data were extracted from 74 eligible documents to form the initial framework. The framework was revised based on feedback from 9 search experts and further review and testing by the authors. The hierarchical framework consists of 119 decisions or processes sorted into 17 categories and arranged under 5 topics. These topics are “Skill of the searcher,” “Selecting information to identify,” “Searching the literature electronically,” “Other ways to identify studies,” and “Updating the systematic review.”Conclusions:The work identifies and classifies the decisions and processes used in systematic searching. Future work can now focus on assessing and prioritizing research on the best methods for successfully identifying all eligible studies for a systematic review.     

Developing and Applying a Protocol for a Systematic Review in the Social Sciences

The article reports on a systematic method of undertaking a literature search on the educational impact of being a young carer (16–24 years old). The search methodology applied and described in detail will be of value to academic librarians and to other education researchers who undertake systematic literature searches. Seven bibliographic databases and Google Scholar were searched between November 2015 and January 2016. Two and three concept search structures were compared, involving 28 search terms plus truncation variants. One hundred and eighty-one relevant articles were retrieved. Sensitivity, precision, and “unique articles retrieved” were used as metrics. Social Care Online and Google Scholar had the greatest sensitivity. As well as meticulous use of AND, OR, and bracket operators, the use of NEAR and NOT operators to increase precision were tested and are recommended as useful tools for conducting systematic searches.     

Evidence-based practice: extending the search to find material for the systematic review

BACKGROUND: Cochrane-style systematic reviews increasingly require the participation of librarians. Guidelines on the appropriate search strategy to use for systematic reviews have been proposed. However, research evidence supporting these recommendations is limited. OBJECTIVE: This study investigates the effectiveness of various systematic search methods used to uncover randomized controlled trials (RCTs) for systematic reviews. Effectiveness is defined as the proportion of relevant material uncovered for the systematic review using extended systematic review search methods. The following extended systematic search methods are evaluated: searching subject-specific or specialized databases (including trial registries), hand searching, scanning reference lists, and communicating personally. METHODS: Two systematic review projects were prospectively monitored regarding the method used to identify items as well as the type of items retrieved. The proportion of RCTs identified by each systematic search method was calculated. RESULTS: The extended systematic search methods uncovered 29.2% of all items retrieved for the systematic reviews. The search of specialized databases was the most effective method, followed by scanning of reference lists, communicating personally, and hand searching. Although the number of items identified through hand searching was small, these unique items would otherwise have been missed. CONCLUSIONS: Extended systematic search methods are effective tools for uncovering material for the systematic review. The quality of the items uncovered has yet to be assessed and will be key in evaluating the value of the systematic search methods.     

社会科学信息分析中的文本挖掘

认为信息分析方法可以用于社会科学研究,具有客观、系统和定量的特点。内容分析作为一种典型的信息分析方法展示了这些特点。但它也具有抽样过程的人为性、手工标引的低效率、人工作业的低信度问题。网络时代出现的文本挖掘方法能够处理海量文献、处理非结构化数据,其研究品质远远高于手工信息分析方法。     

Clinical Evidence diagnosis: Developing a sensitive search strategy to retrieve diagnostic studies on deep vein thrombosis: a pragmatic approach

Objectives: To devise and evaluate a sensitive search strategy to retrieve diagnostic studies on specific diagnostic tests for deep vein thrombosis (DVT). Methods: Systematic reviews on diagnostic tests for DVT were identified and the studies cited by them used to produce a reference set of search results (to be used to evaluate different search strategies). Five existing diagnosis search filters were combined to produce a sensitive search. This combined search was then refined to produce a more specific strategy, which was run on medline and the results were checked against the reference set. This search was too specific and was modified to produce a more balanced final strategy, which was again tested and the results compared with the reference set. The sensitivity of this newly created strategy was compared with the existing diagnosis searches already found. Finally, studies identified by the final search strategy were critically appraised for validity and relevance and the selected articles were compared with those found in the reference set. Results: The final filter retrieved 124 out of 126 references from the reference set. From the search result, 227 cohort studies were selected and 147 of these were not cited in any of the systematic reviews on diagnostic tests for DVT. Conclusions: The search strategy had 98.8% sensitivity. The precision of 8.8%, although low, compares well with other strategies with high sensitivity. Most of the systematic reviews on diagnosing a DVT have omitted a number of high quality articles.     

A Comparison of Three Approaches to Electronic Media Diaries

This project compared the richness of reporting and reflection of three approaches to electronic media diaries. For 1 week, 145 students were randomly assigned to keep daily diaries about their media use and perceptions of media content using one of three methods: (a) a traditional daily journaling approach conducted online, (b) an experience sampling method (ESM) approach that combined daily journaling with multiple daily message prompts, and (c) an approach that combined online peer-group discussion with online daily journaling. The goal was to determine if the supplementary diary activities yielded more detailed, emotive, and cognitive responses. Diary content was analyzed using traditional manual content analysis and linguistic software. Results indicated that the ESM condition elicited less cognitive- and affective-oriented language than the other conditions. The group discussion condition evidenced some advantage over the diary-only group in terms of number of media channels and specific content mentioned.     

高校图书馆OPAC中的分面搜索对用户体验的影响——基于不同任务的对比实验分析

[目的/意义] 研究高校图书馆OPAC中引入分面搜索技术给用户带来怎样的搜索体验。[方法/过程] 主要通过对用户在两种界面下(传统列表界面和基于列表与分面的界面)进行两种搜索任务(已知任务和探寻式任务)时所表现出的行为和喜好进行实验与观察,从搜索时间、搜索结果准确度、用户评价3方面对比分析搜索界面、搜索任务对用户搜索体验的影响。[结果/结论] 通过研究发现:相对于传统列表界面,受试者在使用基于列表与分面的界面时,搜索用时更短、用户评价更高。