Evaluating information skills training in health libraries: a systematic review

Introduction: Systematic reviews have shown that there is limited evidence to demonstrate that the information literacy training health librarians provide is effective in improving clinicians’ information skills or has an impact on patient care. Studies lack measures which demonstrate validity and reliability in evaluating the impact of training. Aim: To determine what measures have been used; the extent to which they are valid and reliable; to provide guidance for health librarians who wish to evaluate the impact of their information skills training. Methods: Data sources: Systematic review methodology involved searching seven databases, and personal files. Study selection: Studies were included if they were about information skills training, used an objective measure to assess outcomes, and occurred in a health setting. Results: Fifty‐four studies were included in the review. Most outcome measures used in the studies were not tested for the key criteria of validity and reliability. Three tested for validity and reliability are described in more detail. Conclusions: Selecting an appropriate measure to evaluate the impact of training is a key factor in carrying out any evaluation. This systematic review provides guidance to health librarians by highlighting measures used in various circumstances, and those that demonstrate validity and reliability.     

Evaluating clinical librarian services: a systematic review

Background: Previous systematic reviews have indicated limited evidence and poor quality evaluations of clinical librarian (CL) services. Rigorous evaluations should demonstrate the value of CL services, but guidance is needed before this can be achieved. Objectives: To undertake a systematic review which examines models of CL services, quality, methods and perspectives of clinical librarian service evaluations. Methods: Systematic review methodology and synthesis of evidence, undertaken collaboratively by a group of 8 librarians to develop research and critical appraisal skills. Results: There are four clear models of clinical library service provision. Clinical librarians are effective in saving health professionals time, providing relevant, useful information and high quality services. Clinical librarians have a positive effect on clinical decision making by contributing to better informed decisions, diagnosis and choice of drug or therapy. The quality of CL studies is improving, but more work is needed on reducing bias and providing evidence of specific impacts on patient care. The Critical Incident Technique as part of a mixed method approach appears to offer a useful approach to demonstrating impact. Conclusions: This systematic review provides practical guidance regarding the evaluation of CL services. It also provides updated evidence regarding the effectiveness and impact of CL services. The approach used was successful in developing research and critical appraisal skills in a group of librarians.     

The Map of Medicine: a review of evidence for its impact on healthcare

Objectives: Map of Medicine is an evidence‐based online clinical knowledge resource. Procured at significant cost by healthcare providers in the UK, Sweden and Denmark, it is important to establish the beneficial impact that investment has had on healthcare practise and, ultimately, on patient outcomes. The objective of this study is to review the evidence for the impact of the Map of Medicine on clinical practice. Methods: A systematic review of peer‐reviewed and grey literature was conducted. Nine healthcare databases, Google Scholar and Google were searched for articles containing the terms ‘map of medicine’. Results: The search identified 133 articles. Eleven of the articles identified met the inclusion criteria. The main finding of the study is the paucity of evidence available on the impact of the Map of Medicine and the variable quality of that which does exist. There are some encouraging early indications for the Map of Medicine as a tool within service redesign, leading to an increase in appropriate referrals to secondary care, decreased patient waiting times and considerable cost savings. Conclusion: Further research with study designs that can generate high quality evidence for the impact of Map of Medicine is essential in order to support policy decisions.     

Cost-effective ways of delivering enquiry services: a rapid review

Background: In the recent times of recession and budget cuts, it is more important than ever for library and information services to deliver cost‐effective services. Objectives: This rapid review aims to examine the evidence for the most cost‐effective ways of delivering enquiry services. Methods: A literature search was conducted on LISA (Library and Information Sciences Abstracts) and MEDLINE. Searches were limited to 2007 onwards. Results: Eight studies met the inclusion criteria. The studies covered hospital and academic libraries in the USA and Canada. Services analysed were ‘point‐of‐care’ librarian consultations, staffing models for reference desks and virtual/digital reference services. Conclusions: Transferable lessons, relevant to health library and information services generally, can be drawn from this rapid review. These suggest that ‘point‐of‐care’ librarians for primary care practitioners are a cost‐effective way of answering questions. Reference desks can be cost‐effectively staffed by student employees or general reference staff, although librarian referral must be provided for more complex and subject‐specific enquiries. However, it is not possible to draw any conclusions on virtual/digital reference services because of the limited literature available. Further case analysis studies measuring specific services, particularly enquiry services within a health library and information context, are required.     

Peer teaching and information retrieval: the role of the NICE Evidence search student champion scheme in enhancing students’ confidence

Background

This research reports on the NICE Evidence search (ES) student champion scheme (SCS) first five years of activity (2011–2016) in terms of its impact on health care undergraduate students’ information search skills and search confidence.

Objectives

A review of students’ evaluation of the scheme was carried out to chart the changes in attitude towards NICE Evidence search as an online health care information source and to monitor students’ approach to information seeking.

Methods

This study is based on the results of questionnaires distributed to students before and after attending a training session on NICE Evidence search delivered by their own peers. The exercise was implemented in health related universities in England over a period of five consecutive academic years.

Results

(i) Students’ search confidence improved considerably after the training; (ii) ES was perceived as being an increasingly useful resource of evidence based information for their studies; (iii) the training helped students develop discerning search skills and use evidence based information sources more consistently and critically.

Conclusions

The NICE SCS improves confidence in approaching information tasks amongst health care undergraduate students. Future developments could involve offering the training at the onset of a course of study and adopting online delivery formats to expand its geographical reach.     

Sources of information on adverse effects: a systematic review

Background: Systematic reviews can provide accurate and timely information on adverse effects. An essential part of the systematic review process is a thorough search of the literature. This often requires searching many different sources. However, it is unclear which sources are most effective at providing information on adverse effects. Objective: To identify and summarise studies that have evaluated sources of information on adverse effects. Methods: Studies were located by searching in 10 databases as well as by reference checking, hand searching, citation searching and contacting experts. Results: A total of 6218 citations were retrieved yielding 19 studies which met the inclusion criteria. The included studies tended to focus on the adverse effects of drug interventions and compare the relative value of different sources using the number of relevant references retrieved from searches of each source. However, few studies were conducted recently with a large sample of references. Conclusions: This review suggests that embase , Derwent Drug File, medline and industry submissions may potentially provide the greatest number of relevant references for information on adverse effects of drugs. However, a systematic evaluation of the current value of different sources of information for adverse effects is urgently required.     

Evidence-based practice: extending the search to find material for the systematic review

BACKGROUND: Cochrane-style systematic reviews increasingly require the participation of librarians. Guidelines on the appropriate search strategy to use for systematic reviews have been proposed. However, research evidence supporting these recommendations is limited. OBJECTIVE: This study investigates the effectiveness of various systematic search methods used to uncover randomized controlled trials (RCTs) for systematic reviews. Effectiveness is defined as the proportion of relevant material uncovered for the systematic review using extended systematic review search methods. The following extended systematic search methods are evaluated: searching subject-specific or specialized databases (including trial registries), hand searching, scanning reference lists, and communicating personally. METHODS: Two systematic review projects were prospectively monitored regarding the method used to identify items as well as the type of items retrieved. The proportion of RCTs identified by each systematic search method was calculated. RESULTS: The extended systematic search methods uncovered 29.2% of all items retrieved for the systematic reviews. The search of specialized databases was the most effective method, followed by scanning of reference lists, communicating personally, and hand searching. Although the number of items identified through hand searching was small, these unique items would otherwise have been missed. CONCLUSIONS: Extended systematic search methods are effective tools for uncovering material for the systematic review. The quality of the items uncovered has yet to be assessed and will be key in evaluating the value of the systematic search methods.     

Evaluation of methodological search filters--a review

OBJECTIVES: The objectives of the study were: to provide an overview of approaches to methodological search filter development; to identify and critically review the stages of methodological search filter development; to devise a search filter appraisal checklist based on the review. METHODS: An iterative approach to searching was employed utilizing health and library databases, the world wide web and citation searching. Further systematic methods included hand searching of key journals in the field of search filter development, contacting known experts in the field and scanning reference lists of relevant papers to identify additional studies. Altogether, 51 potentially relevant papers were found, of which 20 met the inclusion criteria. RESULTS: Four stages of search filter development were identified from the literature (search term selection, identification of a gold standard, evaluation and validation). Variations in the methods used to approach these four stages were identified, most importantly in the extent to which search filters are tested and validated. CONCLUSION: Awareness of the process and limitations involved in search filter development is essential to make an informed decision on the applicability and validity of search filters. The findings of this review indicate a considerable agenda for future research, in particular, to improve the quality of reporting of search filters and to inform users on their use and application. Based on the review, guidance in the appraisal process of search filters is given in the form of a checklist.     

Bibliotherapy for mental health service users Part 1: a systematic review

Aims and objectives: UK health policy advocates a patient‐centred approach to patient care. Library services could serve the rehabilitation needs of mental health service users through bibliotherapy (the use of written, audio, or e‐learning materials to provide therapeutic support). Part 1 of a two part article describes a systematic review of the evidence for the effectiveness of bibliotherapy in mental health services. Methods: The systematic review of the literature used Cochrane guidelines, together with an overview of evaluations of bibliotherapy initiatives, and assessments of the needs of adult mental health service users for rehabilitation support. Results: The evidence strongly suggests that library‐based interventions and the provision of information could be beneficial for service users and economical for the health service in assisting treatment of a range of conditions. At present, public libraries in the UK are developing basic bibliotherapy services. Conclusions: Librarians, including librarians working for the health service, might provide more sophisticated bibliotherapy services, but the evidence to guide delivery is limited.     

Developing methods for systematic reviewing in health services delivery and organization: an example from a review of access to health care for people with learning disabilities. Part 1. Identifying the literature

OBJECTIVES: Our objectives were to identify literature on: (i) theory, evidence and gaps in knowledge relating to the help-seeking behaviour of people with learning disabilities and their carers; (ii) barriers experienced by people with learning disabilities in securing access to the full range of health services; (iii) interventions which improve access to health services by people with learning disabilities. DATA SOURCES: twenty-eight bibliographic databases, research registers, organizational websites or library catalogues; reference lists from identified studies; contact with experts; current awareness and contents alerting services in the area of learning disabilities. REVIEW METHODS: Inclusion criteria were English language literature from 1980 onwards, relating to people with learning disabilities of any age and all study designs. The main criteria for assessment was relevance to the Guilliford et al. model of access to health care (Gulliford et al. Access to health care. Report of a Scoping Exercise for the National Co-ordinating Centre for NHS Service Delivery and Organisation R & D (NCCSDO). London: NCCSDO, 2001), which was modified to the special needs of people with learning disabilities. Inclusion criteria focused on relevance to the model with initial criteria revised in light of literature identified and comments from a consultation exercise with people with learning disabilities, family and paid carers and experts in the field. Data abstraction was completed independently and selected studies were evaluated for scientific rigour and the results synthesized. RESULTS: In total, 2221 items were identified as potentially relevant and 82 studies fully evaluated. CONCLUSIONS: The process of identifying relevant literature was characterized by a process of clarifying the concept under investigation and sensitive search techniques which led to an initial over-identification of non-relevant records from database searches. Thesaurus terms were of limited value, forcing a reliance on using free-text terms and alternative methods of identifying literature to supplement and improve the recall of the database searches. A key enabler in identifying relevant literature was the depth and breadth of knowledge built up by the reviewers whilst engaged in this process.     

The value and impact of information provided through library services for patient care: a systematic review

OBJECTIVE: An updated systematic review was carried out of research studies looking at the value and impact of library services on health outcomes for patients and time saved by health professionals. METHODS: A comprehensive systematic search was undertaken of the published literature to September 2003 in ERIC, LISA, MEDLINE, PREMEDLINE, EMBASE, the Cochrane Controlled Trials Register and Google. Some handsearching was carried out, reference lists were scanned and experts in the field were contacted. Twenty-eight research studies of professionally led libraries for health-care staff, including clinical librarian projects, met the inclusion criterion of at least one health or 'time saved' outcome. Papers were critically appraised using internationally accepted criteria. Data were extracted and results were summarised using a narrative format as the studies were heterogeneous and precluded a statistical analysis. RESULTS: There is evidence of impact from both traditional and clinical librarian services. The higher quality studies of traditional services measured impacts of 37-97% on general patient care, 10-31% on diagnosis, 20-51% on choice of tests, 27-45% on choice of therapy and 10-19% on reduced length of stay. Four studies of clinical librarian projects suggested that professionals saved time as a result of clinical librarian input, and two of these studies showed evidence of cost-effectiveness. However, the clinical librarian studies were generally smaller, with poorer quality standards. CONCLUSIONS: Research studies suggest that professionally led library services have an impact on health outcomes for patients and may lead to time savings for health-care professionals. The available studies vary greatly in quality but the better quality studies also suggest positive impacts. Good practice can be gathered from these studies to guide the development of a pragmatic survey for library services that includes the direct effects for patients among the outcome measures.     

Demonstrating the positive impact of information support on patient care in primary care: a rapid literature review

AIM: To review the literature on the positive impact of information services, or information resources, on patient care in primary care. OBJECTIVES: To identify and summarize key papers on which librarians might draw in making the case for investment, and to highlight gaps in the research evidence Methodology: A rapid literature review was conducted in the summer of 2005. RESULTS: There is a small body of evidence to demonstrate the positive impact of library and information services on the direct care of patients as well as a beneficial impact on the care of future patients through the application of evidence to multiple patients. CONCLUSIONS: There is relatively limited research evidence of the impact of information, and library services, in primary care, in comparison with hospital settings and the research available is generally reliant on small samples. There is a lack of impact studies conducted with non-clinical staff. The review highlights the value of critical incident technique (CIT). It is possible to gather evidence of the potential for information services to deliver cost savings.     

Research consultations in the academic library: A scoping review on current themes in instruction,assessment and technology

This scoping review highlights the characteristics, assessments and technologies used to describe, improve and promote one-to-one research consultations as a mode of research support and instruction in academic libraries. A search for relevant studies was conducted using LISTA, LISA, ERIC, Scopus and Web of Science, limiting to empirical evidence or studies outlining the use of technology within library practice, published from 2013 to current, in the English language. Supplemental search methods included a grey literature search, handsearching, and cited reference searching. From 2268 records, 43 studies were identified for inclusion. Of these, 17 studies described using consultations in the delivery of information literacy instruction, 33 evaluation studies reported on student, faculty, and librarian outcomes, and 15 articles discussed the use of technology. Users reported an overwhelming positive experience while mixed learning outcomes were seen in performance assessment studies. The assessment methods and uses of technology outlined in this review can be used by librarians to inform service delivery and provide evidence of the value and impact of research consultations on the academic mission of the library and its institution.     

An evaluation of searching the German CCMed database for the production of systematic reviews

Objectives: National databases may be useful sources in the production of a systematic review (SR). The aim of this study was to assess the potential benefit of a systematic search in the German database ‘Current Contents Medizin’ (CCMed ). Methods: The study was conducted on the basis of published SRs that included CCMed as a literature source. Eligible SRs were identified through a systematic search in medline , embase and The Cochrane Library. The websites of German Health Technology Assessment agencies were also screened. Citations of primary studies included as relevant in the SRs were extracted and then categorised. Results: The search yielded 52 eligible SRs. A total of 1505 relevant citations were extracted. Seventy‐seven of these articles were published in journals indexed in CCMed . Thirty‐two of the 77 citations were indexed in CCMed , but only eight of the 32 were unique. Of these eight citations, seven were not identified by a systematic search, but by handsearching. Only one unique citation, an observational study, was identified in CCMed by a systematic search. Conclusions: In the production of SRs, a systematic search in CCMed identifies relevant studies only in exceptional cases. Therefore, the routine inclusion of this database in systematic searches does not appear meaningful.     

Promoting the use of open government data: Cases of training and engagement

In the last decade, governments around the world have created open government data (OGD) repositories to make government data more accessible and usable by the public, mostly motivated by values such as improved government transparency, citizen collaboration and participation, and spurring innovation. The basic assumption is that once data are more discoverable, accessible, available in alternative formats, and with licensing schemes that allow free re-use, diverse stakeholders will develop innovative data applications. Despite OGD's potential transformative value, there is limited evidence for such transformation, particularly due to scarce data use, which is partly attributable to the lack of technical skills and user training. To advance the dialogue around methods to increase awareness of OGD, improve users' skills to work with OGD, and encourage data use, the paper compares and contrasts how three training interventions in Spain, Italy, and the United States have sought to increase awareness of OGD, improve users' skills and potentially engage them in their use of OGD. We report three main findings. First, introduction and analysis skills are taught in combination to encourage use of open data. Being aware of OGD and its benefits is insufficient to promote use. Second, OGD training seems to be more effective when complemented with knowledge about context and interactions with government. Finally, embedding the training interventions in the specific contexts and considering the unique characteristics, interests, and expectations of different types of users is critical to success.