Assessing journal author guidelines for systematic reviews and meta-analyses: findings from an institutional sample

Objectives:Systematic reviews and meta-analyses (SRs/MAs) are designed to be rigorous research methodologies that synthesize information and inform practice. An increase in their publication runs parallel to quality concerns and a movement toward standards to improve reporting and methodology. With the goal of informing the guidance librarians provide to SR/MA teams, this study assesses online journal author guidelines from an institutional sample to determine whether these author guidelines address SR/MA methodological quality.Methods:A Web of Science Core Collection (Clarivate) search identified SRs/MAs published in 2014–2019 by authors affiliated with a single institution. The AMSTAR 2 checklist was used to develop an assessment tool of closed questions specific to measures for SR/MA methodological quality in author guidelines, with questions added about author guidelines in general. Multiple reviewers completed the assessment.Results:The author guidelines of 141 journals were evaluated. Less than 20% addressed at least one of the assessed measures specific to SR/MA methodological quality. There was wide variation in author guidelines between journals from the same publisher apart from the American Medical Association, which consistently offered in-depth author guidelines. Normalized Eigenfactor and Article Influence Scores did not indicate author guideline breadth.Conclusions:Most author guidelines in the institutional sample did not address SR/MA methodological quality. When consulting with teams embarking on SRs/MAs, librarians should not expect author guidelines to provide details about the requirements of the target journals. Librarians should advise teams to follow established SR/MA standards, contact journal staff, and review SRs/MAs previously published in the journal.     

Preliminary comparison of the performance of the National Library of Medicine's systematic review publication type and the sensitive clinical queries filter for systematic reviews in PubMed

Objective:The National Library of Medicine (NLM) inaugurated a “publication type” concept to facilitate searches for systematic reviews (SRs). On the other hand, clinical queries (CQs) are validated search strategies designed to retrieve scientifically sound, clinically relevant original and review articles from biomedical literature databases. We compared the retrieval performance of the SR publication type (SR[pt]) against the most sensitive CQ for systematic review articles (CQrs) in PubMed.Methods:We ran date-limited searches of SR[pt] and CQrs to compare the relative yield of articles and SRs, focusing on the differences in retrieval of SRs by SR[pt] but not CQrs (SR[pt] NOT CQrs) and CQrs NOT SR[pt]. Random samples of articles retrieved in each of these comparisons were examined for SRs until a consistent pattern became evident.Results:For SR[pt] NOT CQrs, the yield was relatively low in quantity but rich in quality, with 79% of the articles being SRs. For CQrs NOT SR[pt], the yield was high in quantity but low in quality, with only 8% being SRs. For CQrs AND SR[pt], the quality was highest, with 92% being SRs.Conclusions:We found that SR[pt] had high precision and specificity for SRs but low recall (sensitivity), whereas CQrs had much higher recall. SR[pt] OR CQrs added valid SRs to the CQrs yield at low cost (i.e., added few non-SRs). For searches that are intended to be exhaustive for SRs, SR[pt] can be added to existing sensitive search filters.     

An evaluation of searching the German CCMed database for the production of systematic reviews

Objectives: National databases may be useful sources in the production of a systematic review (SR). The aim of this study was to assess the potential benefit of a systematic search in the German database ‘Current Contents Medizin’ (CCMed ). Methods: The study was conducted on the basis of published SRs that included CCMed as a literature source. Eligible SRs were identified through a systematic search in medline , embase and The Cochrane Library. The websites of German Health Technology Assessment agencies were also screened. Citations of primary studies included as relevant in the SRs were extracted and then categorised. Results: The search yielded 52 eligible SRs. A total of 1505 relevant citations were extracted. Seventy‐seven of these articles were published in journals indexed in CCMed . Thirty‐two of the 77 citations were indexed in CCMed , but only eight of the 32 were unique. Of these eight citations, seven were not identified by a systematic search, but by handsearching. Only one unique citation, an observational study, was identified in CCMed by a systematic search. Conclusions: In the production of SRs, a systematic search in CCMed identifies relevant studies only in exceptional cases. Therefore, the routine inclusion of this database in systematic searches does not appear meaningful.     

Journals in the Core Collection

ABSTRACT

This overview of the core concept applied to journals defines the relevant terminology and cites specific examples of core lists. Ten approaches for determining core journals (subjective judgment, use, indexing coverage, overlapping library holdings, citation data, citation network/co-citation analysis, production of articles, Bradford's Law, faculty publication data, and multiple criteria methods) are reviewed and the practical applications of core journals lists are explained. Theoretical and practical problems associated with the core concept and core journal lists are discussed and a taxonomy for classifying core journal lists is outlined.     

Measuring contextual citation impact of scientific journals

This paper explores a new indicator of journal citation impact, denoted as source normalized impact per paper (SNIP). It measures a journal's contextual citation impact, taking into account characteristics of its properly defined subject field, especially the frequency at which authors cite other papers in their reference lists, the rapidity of maturing of citation impact, and the extent to which a database used for the assessment covers the field's literature. It further develops Eugene Garfield's notions of a field's ‘citation potential’ defined as the average length of references lists in a field and determining the probability of being cited, and the need in fair performance assessments to correct for differences between subject fields. A journal's subject field is defined as the set of papers citing that journal. SNIP is defined as the ratio of the journal's citation count per paper and the citation potential in its subject field. It aims to allow direct comparison of sources in different subject fields. Citation potential is shown to vary not only between journal subject categories – groupings of journals sharing a research field – or disciplines (e.g., journals in mathematics, engineering and social sciences tend to have lower values than titles in life sciences), but also between journals within the same subject category. For instance, basic journals tend to show higher citation potentials than applied or clinical journals, and journals covering emerging topics higher than periodicals in classical subjects or more general journals. SNIP corrects for such differences. Its strengths and limitations are critically discussed, and suggestions are made for further research. All empirical results are derived from Elsevier's Scopus.     

A Comparative Citation Analysis of Seven Mid-South Law Reviews

Abstract

Several studies have compared the frequency of citation to law reviews, comparing citation rates for all or most law reviews being published at the time of the study. This article compares the citation rates of general law reviews published by seven public law schools in close geographic proximity. The results show that citation rates are influenced by several factors, including whether articles published by the journals have a state or national focus, the subject matter of the articles, and whether articles are published as part of symposia.     

Expanded Assessment Study Examining the Citation Patterns from Traditional and Nontraditional Institutions and Their Effect upon the Quality of Doctoral Dissertation Reference Lists

ABSTRACT

This study used citation analysis in conjunction with a subjective rubric with five validated criteria to assess the quality of a purposive sample of 452 education dissertation reference lists produced at a nontraditional institution from six doctoral programs. These programs used a variety of distance education models. The citations were then compared with the 100 reference lists selected from doctoral education programs at 10 traditional institutions. Criteria included the breadth of resources, depth of the literature review as shown through the citing of critical historical and theoretical works, depth as demonstrated through the scholarliness of citations chosen, currency, and relevancy. The study discusses how the expanded sample of traditional reference lists continued to demonstrate consistent patterns. In contrast, although the sample of reference lists from distance education programs was much expanded, the patterns of the subsets of reference lists at the nontraditional institution were significantly diverse. The patterns of 35 doctoral reference lists from another for-profit, nontraditional institution known for providing distance education were also examined to compare the results of the two nontraditional programs. Possible reasons for differences in citation patterns by traditional and nontraditional programs are discussed, and the possible role of distance education and program criteria are examined for insights they offer into the use of citation analysis as an assessment tool of the library research skills of doctoral students.     

Efficient literature searching in diffuse topics: lessons from a systematic review of research on communicating risk to patients in primary care

Using the example of communication about risk in a primary care setting, this paper puts forward a method of developing and evaluating a detailed search strategy for locating the literature for a systematic review of a ‘diffuse’ subject. The aim of this paper is to show how to develop a search strategy that maximizes both recall and precision while keeping search outputs manageable. Six different databases were used, namely Medline, Embase, PsychLIT, CancerLIT, Cinahl and Social Science Citation Index (SSCI). The searches were augmented by hand-searching, contacting authors, citation searching and reference lists from included papers. Other databases were searched but yielded no extra references for this subject matter. Of the 99 papers included, 80 were indexed on Medline. The Medline search strategy identified 54 of them and the remaining 26 were located on other databases. The 19 further unique references were found using the other databases and methods of retrieval. A combination of several databases must be used to maximize recall and to increase the precision of searches on individual databases, thus improving the overall efficiency of the search.     

Retracted publications in pharmacy systematic reviews

Objective:Systematic reviews and other evidence syntheses, the pinnacle of the evidence pyramid, embody comprehensiveness and rigor; however, retracted data are being incorporated into these publications. This study examines the use of retracted publications in the field of pharmacy, describes characteristics of retracted publications cited by systematic reviews, and discusses factors associated with citation likelihood.Methods:Using data from Retraction Watch, we identified retracted publications in the pharmacy field. We identified all articles citing these retracted publications in Web of Science and Scopus and limited results to systematic reviews. We classified the retraction reason, determined whether the citation occurred before or after retraction, and analyzed factors associated with the likelihood of systematic reviews citing a retracted publication.Results:Of 1,396 retracted publications, 283 were cited 1,096 times in systematic reviews. Most (65.0%) (712/1096) citations occurred before retraction. Citations were most often to items retracted due to data falsification or manipulation (39.2%), followed by items retracted due to ethical misconduct including plagiarism (30.4%), or concerns about or errors in data or methods (26.2%). Compared to those not cited in systematic reviews, cited items were significantly more likely to be retracted due to data falsification and manipulation, were published in high impact factor journals, and had longer delays between publication and retraction.Conclusions:Further analysis of systematic reviews citing retracted publications is needed to determine the impact of flawed data. Librarians understand the nuances involved and can advocate for greater transparency around the retraction process and increase awareness of challenges posed by retractions.     

Measuring the Quality of Contributions of Saudi Authors to LIS Journals Using Journal Impact Factor (JIF), SCImago Journal Rank (SJR), and Google Scholar Metrics (GSM)

This research study evaluates the quality of articles published by Saudi and expatriate authors in foreign Library and Information Science (LIS) journals using three popular metrics for ranking journals—Journal Impact Factor (JIF), SCImago Journal Rank (SJR), and Google Scholar Metrics (GSM). The reason for using multiple metrics is to see how closely or differently journals are ranked by the three different methods of citation analysis. However, the 2012 JIF list of journals is too small, almost half the size of the SJR and GSM lists, which inhibited one-to-one comparison among the impact factors of the thirty-six journals selected by Saudi authors for publishing articles. Only seventeen journals were found common to all the three lists, limiting the usefulness of the data. A basic problem is that Saudi LIS authors generally lack the level of competency in the English language required to achieve publication in the most prominent LIS journals. The study will have implications for authors, directors, and deans of all types of academic libraries; chairmen and deans of library schools; and the Saudi Library Association. Hopefully these entities will take necessary steps to prepare and motivate both academics and practicing librarians to improve the quality of their research and publications and thus get published in higher ranked journals.     

A longitudinal investigation into the changing citing behavior of geomatics postgraduate students at Wuhan University,China, 1988–2004: Implications for collection development

Aim:The purpose of this study is to identify and describe the citing behavior of geomatics postgraduate students at the Wuhan University in China. These students constitute one of the main user groups of Wuhan University Library and the analysis of their use of literature (through the means of citation analysis) can yield valuable insights against which to benchmark and inform collection development policy and other services at the Information Technology Library (ITL) at Wuhan University.Design/methodology/approach:In this project, 9317 citations in 21 PhD and 180 MSc theses submitted at the Wuhan University in the years of 1988, 1996, and 2004 were analyzed. All cited materials were divided by output format (monographs, proceedings, journal articles) and the more in-depth analysis was conducted for all cited journals. The following characteristics of the cited journal literature were analyzed: subject, age, language, and dispersion of journal titles.Findings:An increased use of journal literature in bibliographies of geomatics theses was observed over time. A trend to increase citations to international conference proceedings was also noted as well as a trend to cite more journals in disciplines other than surveying and mapping. It was also noted that, over time, citations to documents in languages other than Chinese and English have significantly decreased (in some cases, disappeared). Large dispersion among the most frequently cited journals in different time periods (only 25% of titles were repeated in 1988, 1996, and 2004 top ranking journal lists) was also noted. It was also noted that the core of geomatics literature is expanding (it now takes 42% of journal titles to cover 80% of all citations).Practical implications:By analyzing citation patterns of the graduate students of geomatics in China, we can track the developments and changes within that discipline in China and can compare these trends internationally (e.g. increased role of a journal, the importance of conference proceedings, domination of the English-language sources, dispersion of journal literature, and increasing multidisciplinarity of the discipline). The techniques and methods used in this study can be replicated to study other disciplines. Also, it is hoped that the findings of this research will inform collection development planning, development of reference services, and information literacy training.     

Academic in-group bias: An empirical examination of the link between author and journal affiliation

Do academic journals favor authors who share their institutional affiliation? To answer this question we examine citation counts, as a proxy for paper quality, for articles published in four leading international relations journals during the years 2000–2015. We compare citation counts for articles written by “in-group members” (authors affiliated with the journal’s publishing institution) versus “out-group members” (authors not affiliated with that institution). Articles written by in-group authors received 18% to 49% fewer Web of Science citations when published in their home journal (International Security or World Politics) vs. an unaffiliated journal, compared to out-group authors. These results are mainly driven by authors who received their PhDs from Harvard or MIT. The findings show evidence of a bias within some journals towards publishing papers by faculty from their home institution, at the expense of paper quality.     

Beyond Title Lists

Abstract

Librarians and patrons can access many thousands of electronic journals today. Librarians who incorporate records for these journals into their OPACs will maximize the journals' access and use, but accurately tracking them can be difficult. This paper describes a commercial solution to the problem. Serials Solutions combines aggregator title lists with CONSER MARC records to improve access to these titles. This paper reviews the benefits and challenges associated with this process, and identifies several areas in which aggregators and OPAC vendors can improve the services they provide, to facilitate this process.     

An Altmetrics and citation analysis of selected predatory journals in library and information science field

PurposeThis paper aims to examine whether Altmetric data can be used as an indicator for identifying predatory journals.Design/methodology/approachThis is an applied study which uses citation and Altmetrics methods. The study selected 21 predatory journals from the Beall's list and Kscien's list, as well as 18 non-predatory open access journals from the DOAJ's list, in the field of Library and Information Science. The Altmetric score for articles published in these journals was obtained from the Altmetric Explorer, a service provided by Altmetric.com. Web of Science was used to search for citation data of articles published in these journals.FindingsThe predatory journals almost have no presence in social media, with poor Altmetric score. In contrast, non-predatory open access journals have a high presence rate and Altmetric score. There is a significant positive correlation between the number of articles cited and the number of articles having Altmetric score among non-predatory open-access journals, but not among predatory journals. Poor Altmetric score may be viewed as a potential characteristic of predatory journals, but other indicators would also need to be considered to determine whether a journal is predatory.Originality/valueDistinct from the traditional research methods, this study combined citation analysis and Altmetrics analysis. By comparing the characteristics of predatory journals and non-predatory open access journals, the findings contribute to the identification of predatory journals.     

Identifying unidentified U.S. government reports

The Library of Congress last published Popular Names of U.S. Government Reports in 1984. In its appendix this helpful reference work lists 108 unidentified reports. This article discusses sources and methods used to identify some of these reports, emphasizing the benefits of using standard reference sources in conjunction with more specialized tools and electronic databases to verify the existence, location and correct citation of government publications. For those reports the authors have further identified, they provide bibliographic information and notes.     

Assessing the roles and challenges of librarians in dental systematic and scoping reviews

Objective:The objective of this study was to determine the scope of experience, roles, and challenges that librarians face in participating in dental and oral health systematic and scoping reviews to inform outreach efforts to researchers and identify areas for librarian professional development.Methods:The authors developed a twenty-three-item survey based on the findings of two recent articles about health sciences librarians'' roles and challenges in conducting systematic and scoping reviews. The survey was distributed via electronic mailing lists to librarians who were likely to have participated in conducting dental systematic and scoping reviews.Results:While survey respondents reported participating in many dental reviews, they participated more commonly in systematic reviews than in scoping reviews. Also, they worked less commonly on dental and oral health reviews than on non-dental reviews. Librarian roles in dental reviews tended to follow traditional librarian roles: all respondents had participated in planning and information retrieval stages, whereas fewer respondents had participated in screening and assessing articles. The most frequently reported challenges involved the lead reviewer or review team rather than the librarians themselves, with time- and methodology-related challenges being most common.Conclusions:Although librarians might not be highly involved in dental and oral health systematic and scoping reviews, more librarian participation in these reviews, either as methodologists or information experts, may improve their reviews'' overall quality.     

Core Journal Lists for Behaviorally Disordered Children

Two professionals, one an academic researcher and the other a classroom teacher, each selected starting sets of journals that they found most useful in their work with behaviorally disordered children. Using citation analysis, core lists of journals were developed from these starting sets and ranked by "discipline impact factor," or DIF. Only two journals were common to both lists. The uncritical acceptance of core lists without considering the value judgements that are a part of the selection process can be misleading.     

�й����׼���ѧ�������ס���������������ڿ�

利用引文分析方法,研究了文献计量学在中国的发展情况。在进行大量文献研究和引文分析的基础上,介绍了一批文献计量学研究与应用的核心论文、核心著者与核心期刊     

Journals on Information Literacy: Citation Analysis and Social Impact Metrics

ABSTRACT

Library and information science as a discipline is undergoing rapid and extensive change, driven particularly by new areas of research. Citation analysis and information literacy are widely researched and have a rich background of growth. Publish or Perish, Buzzsumo.com, altmetrics.com, and textalyser.net have been used to analyze all metrics associated with citation analysis, social networking, and keyword analysis.

The hypothesis of this study is: “There is no significant difference in the citation pattern of the three prominent citation journals: Journal of Information Literacy, Communications in Information Literacy and Nordic Journal of Information Literacy in Higher Education”.

The hypotheses of the study were tested by assessment of the degree to which each of the journals met set of objective criteria. 697 papers from the three journals were analyzed and the results showed that the citation metrics of the Journal of Information Literacy had the highest correspondence with the criteria. The information linked to the Journals of Information Literacy was moderately posted on social networking sites (SNS) and the highest frequency of the keywords: “Information”; “Information literacy”; and “Information literacy instruction” were found in one, two and three-word searches. Some recommendations have also been offered to improve SNS presence and for further research in this field.